%matplotlib inline
%reload_ext autoreload
%autoreload 2Generate rollout grids
import sys
sys.path.append("../../../")
import os
from pathlib import Path
import geopandas as gpd
import pandas as pd
from povertymapping.iso3 import get_region_name
from povertymapping.rollout_grids import get_region_filtered_bingtile_grids/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.1-CAPI-1.17.1) is incompatible with the GEOS version PyGEOS was compiled with (3.10.1-CAPI-1.16.0). Conversions between both will be slow.
warnings.warn(Generate Rollout Grids: Philippines
This notebook generates the rollout grid tiles over the country. The output file is used as an input for Step 3, where we will run the trained model over the set of grids.
The generated grids are set at 2.4km (zoom level 14), matching the grids used during model training. The grids are also filtered to only include populated areas based on Meta’s High Resolution Settlement Layer (HRSL) data.
Set country-specific parameters
COUNTRY_CODE = "ph"
REGION = get_region_name(COUNTRY_CODE, code="alpha-2").lower()
ADMIN_LVL = "ADM2"
ZOOM_LEVEL = 14Generate Grids
admin_grids_gdf = get_region_filtered_bingtile_grids(
REGION, admin_lvl=ADMIN_LVL, quadkey_lvl=ZOOM_LEVEL, use_cache=True
)2023-03-12 17:30:19.723 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:264 - No cached grids file found. Generating grids file :/home/jace/.cache/geowrangler/quadkey_grids/philippines_14_ADM2_populated_admin_grids.geojson
2023-03-12 17:30:19.726 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region philippines and admin level ADM2
2023-03-12 17:30:19.726 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region philippines and admin level ADM2
2023-03-12 17:30:20.399 | INFO | povertymapping.geoboundaries:get_geoboundaries:41 - Downloading geoboundaries for PHL at admin level ADM2 at https://www.geoboundaries.org/gbRequest.html?ISO=PHL&ADM=ADM2
2023-03-12 17:30:22.328 | INFO | povertymapping.geoboundaries:get_geoboundaries:50 - Download path for PHL at admin level ADM2 found at https://geoboundaries.org/data/geoBoundaries-3_0_0/PHL/ADM2/geoBoundaries-3_0_0-PHL-ADM2.geojson2023-03-12 17:32:13.993 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:287 - Generating grids for region philippines and admin level ADM2 at quadkey level 142023-03-12 17:37:19.068 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:293 - Generated 59402 grids for region philippines and admin level ADM2 at quadkey level 14
2023-03-12 17:37:19.083 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:299 - Assigning grids to admin areas using metric crs epsg:3857
2023-03-12 17:37:19.083 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:299 - Assigning grids to admin areas using metric crs epsg:3857
/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/geodataframe.py:2196: UserWarning: `keep_geom_type=True` in overlay resulted in 6 dropped geometries of different geometry types than df1 has. Set `keep_geom_type=False` to retain all geometries
return geopandas.overlay(
2023-03-12 17:41:24.322 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:305 - Getting philippines population data for filtering grids2023-03-12 17:46:26.820 | INFO | povertymapping.hdx:get_hdx_file:201 - HDX Data: Unzipping the zip file /home/jace/.cache/geowrangler/hdx/phl_general_2020_geotiff.zip...
2023-03-12 17:48:20.304 | INFO | povertymapping.hdx:get_hdx_file:211 - HDX Data: Successfully downloaded and cached for philippines at /home/jace/.cache/geowrangler/hdx/phl_general_2020_geotiff.zip!
2023-03-12 17:48:20.306 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:307 - Computing population zonal stats per grid
2023-03-12 17:48:20.309 | INFO | povertymapping.rollout_grids:compute_raster_stats:98 - Creating raster zonal stats for 59402 grids for file size 19389139724
2023-03-12 18:19:54.212 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:318 - Filtering unpopulated grids based on population data
2023-03-12 18:19:54.257 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:321 - Filtered admin grid count: 46483
/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index,Explore Populated Grids
admin_grids_gdf.info()<class 'geopandas.geodataframe.GeoDataFrame'>
Int64Index: 46483 entries, 28 to 59400
Data columns (total 8 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 geometry 46483 non-null geometry
1 quadkey 46483 non-null object
2 shapeName 46483 non-null object
3 shapeISO 46483 non-null object
4 shapeID 46483 non-null object
5 shapeGroup 46483 non-null object
6 shapeType 46483 non-null object
7 pop_count 46483 non-null float64
dtypes: float64(1), geometry(1), object(6)
memory usage: 3.2+ MBadmin_grids_gdf.head(2)| geometry | quadkey | shapeName | shapeISO | shapeID | shapeGroup | shapeType | pop_count | |
|---|---|---|---|---|---|---|---|---|
| 28 | POLYGON ((118.47656 6.94824, 118.47656 6.97005... | 13232122010020 | Tawi-Tawi | None | PHL-ADM2-3_0_0-B77 | PHL | ADM2 | 11.677980 |
| 30 | POLYGON ((118.41064 7.01367, 118.41064 7.03548... | 13232120223323 | Tawi-Tawi | None | PHL-ADM2-3_0_0-B77 | PHL | ADM2 | 317.069037 |
# Uncomment to view the generated grid on an interactive map
# admin_grids_gdf.explore()Save to file
rollout_date = "-".join(os.getcwd().split("/")[-2].split("-")[:3])
grid_save_path = f"./{rollout_date}-{COUNTRY_CODE}-rollout-grids.geojson"
admin_grids_gdf.to_file(grid_save_path, driver="GeoJSON", index=False)