%matplotlib inline
%reload_ext autoreload
%autoreload 2Generate rollout grids
import sys
sys.path.append("../../../")
import os
from pathlib import Path
import geopandas as gpd
import pandas as pd
from povertymapping.iso3 import get_region_name
from povertymapping.rollout_grids import get_region_filtered_bingtile_grids/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.1-CAPI-1.17.1) is incompatible with the GEOS version PyGEOS was compiled with (3.10.1-CAPI-1.16.0). Conversions between both will be slow.
warnings.warn(Generate Rollout Grids: Cambodia
This notebook is the second step in the rollout and generates the rollout grid tiles over the country. The output file is used as an input for Step 3, where we will run the trained model over the set of grids.
The generated grids are set at 2.4km (zoom level 14), matching the grids used during model training. The grids are also filtered to only include populated areas based on Meta’s High Resolution Settlement Layer (HRSL) data.
Set country-specific parameters
COUNTRY_CODE = "kh"
REGION = get_region_name(COUNTRY_CODE, code="alpha-2").lower()
ADMIN_LVL = "ADM2"
ZOOM_LEVEL = 14Generate Grids
admin_grids_gdf = get_region_filtered_bingtile_grids(
REGION, admin_lvl=ADMIN_LVL, quadkey_lvl=ZOOM_LEVEL, use_cache=True
)2023-03-12 17:36:21.223 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:264 - No cached grids file found. Generating grids file :/home/jace/.cache/geowrangler/quadkey_grids/cambodia_14_ADM2_populated_admin_grids.geojson
2023-03-12 17:36:21.230 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region cambodia and admin level ADM2
2023-03-12 17:36:21.230 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region cambodia and admin level ADM2
2023-03-12 17:36:21.849 | INFO | povertymapping.geoboundaries:get_geoboundaries:41 - Downloading geoboundaries for KHM at admin level ADM2 at https://www.geoboundaries.org/gbRequest.html?ISO=KHM&ADM=ADM2
2023-03-12 17:36:23.378 | INFO | povertymapping.geoboundaries:get_geoboundaries:50 - Download path for KHM at admin level ADM2 found at https://geoboundaries.org/data/geoBoundaries-3_0_0/KHM/ADM2/geoBoundaries-3_0_0-KHM-ADM2.geojson2023-03-12 17:36:54.238 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:287 - Generating grids for region cambodia and admin level ADM2 at quadkey level 14
12.50% [1/8 00:00<00:00]
2023-03-12 17:37:45.227 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:293 - Generated 32580 grids for region cambodia and admin level ADM2 at quadkey level 14
2023-03-12 17:37:45.236 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:299 - Assigning grids to admin areas using metric crs epsg:3857
2023-03-12 17:37:45.236 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:299 - Assigning grids to admin areas using metric crs epsg:3857
2023-03-12 17:38:11.135 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:305 - Getting cambodia population data for filtering grids2023-03-12 17:39:42.662 | INFO | povertymapping.hdx:get_hdx_file:201 - HDX Data: Unzipping the zip file /home/jace/.cache/geowrangler/hdx/khm_general_2020_geotiff.zip...
2023-03-12 17:40:06.724 | INFO | povertymapping.hdx:get_hdx_file:211 - HDX Data: Successfully downloaded and cached for cambodia at /home/jace/.cache/geowrangler/hdx/khm_general_2020_geotiff.zip!
2023-03-12 17:40:06.726 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:307 - Computing population zonal stats per grid
2023-03-12 17:40:06.728 | INFO | povertymapping.rollout_grids:compute_raster_stats:98 - Creating raster zonal stats for 32580 grids for file size 3732653178
2023-03-12 17:54:06.391 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:318 - Filtering unpopulated grids based on population data
2023-03-12 17:54:06.429 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:321 - Filtered admin grid count: 20137
/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index,Explore Populated Grids
admin_grids_gdf.info()<class 'geopandas.geodataframe.GeoDataFrame'>
Int64Index: 20137 entries, 3 to 32579
Data columns (total 8 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 geometry 20137 non-null geometry
1 quadkey 20137 non-null object
2 shapeName 20137 non-null object
3 shapeISO 20137 non-null object
4 shapeID 20137 non-null object
5 shapeGroup 20137 non-null object
6 shapeType 20137 non-null object
7 pop_count 20137 non-null float64
dtypes: float64(1), geometry(1), object(6)
memory usage: 1.4+ MBadmin_grids_gdf.head(2)| geometry | quadkey | shapeName | shapeISO | shapeID | shapeGroup | shapeType | pop_count | |
|---|---|---|---|---|---|---|---|---|
| 3 | POLYGON ((103.16162 10.81172, 103.16162 10.833... | 13223001030111 | Botum Sakor | None | KHM-ADM2-3_0_0-B77 | KHM | ADM2 | 1.530576 |
| 4 | POLYGON ((103.18359 10.83331, 103.18359 10.854... | 13223001013222 | Botum Sakor | None | KHM-ADM2-3_0_0-B77 | KHM | ADM2 | 1.530576 |
# Uncomment to view the generated grid on an interactive map
# admin_grids_gdf.explore()Save to file
rollout_date = "-".join(os.getcwd().split("/")[-2].split("-")[:3])
grid_save_path = f"./{rollout_date}-{COUNTRY_CODE}-rollout-grids.geojson"
admin_grids_gdf.to_file(grid_save_path, driver="GeoJSON", index=False)