%matplotlib inline
%reload_ext autoreload
%autoreload 2Predict on rollout grids
import sys
sys.path.append("../../../")
import os
from pathlib import Path
import geopandas as gpd
import pandas as pd
from povertymapping.iso3 import get_region_name
from povertymapping.rollout_grids import get_region_filtered_bingtile_grids/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.11.1-CAPI-1.17.1) is incompatible with the GEOS version PyGEOS was compiled with (3.10.1-CAPI-1.16.0). Conversions between both will be slow.
warnings.warn(Generate Rollout Grids: Malaysia
This notebook is the second step in the rollout and generates the rollout grid tiles over the country. The output file is used as an input for Step 3, where we will run the trained model over the set of grids.
The generated grids are set at 2.4km (zoom level 14), matching the grids used during model training. The grids are also filtered to only include populated areas based on Meta’s High Resolution Settlement Layer (HRSL) data.
Set country-specific parameters
COUNTRY_CODE = "my"
REGION = get_region_name(COUNTRY_CODE, code="alpha-2").lower()
ADMIN_LVL = "ADM2"
ZOOM_LEVEL = 14Generate Grids
admin_grids_gdf = get_region_filtered_bingtile_grids(
REGION, admin_lvl=ADMIN_LVL, quadkey_lvl=ZOOM_LEVEL, use_cache=True
)2023-03-09 21:41:01.871 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:264 - No cached grids file found. Generating grids file :/home/jace/.cache/geowrangler/quadkey_grids/malaysia_14_ADM2_populated_admin_grids.geojson
2023-03-09 21:41:01.872 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region malaysia and admin level ADM2
2023-03-09 21:41:01.872 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region malaysia and admin level ADM2
2023-03-09 21:41:02.708 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:287 - Generating grids for region malaysia and admin level ADM2 at quadkey level 142023-03-09 21:42:01.196 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:293 - Generated 58058 grids for region malaysia and admin level ADM2 at quadkey level 14
2023-03-09 21:42:01.232 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:299 - Assigning grids to admin areas using metric crs epsg:3857
2023-03-09 21:42:17.729 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:305 - Getting malaysia population data for filtering grids
2023-03-09 21:42:18.839 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:307 - Computing population zonal stats per grid
2023-03-09 21:42:18.840 | INFO | povertymapping.rollout_grids:compute_raster_stats:98 - Creating raster zonal stats for 58058 grids for file size 17418701324
2023-03-09 22:02:34.393 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:318 - Filtering unpopulated grids based on population data
2023-03-09 22:02:34.409 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:321 - Filtered admin grid count: 30202
/home/jace/workspace/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/geopandas/io/file.py:362: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index,Explore Populated Grids
admin_grids_gdf.info()<class 'geopandas.geodataframe.GeoDataFrame'>
Int64Index: 30202 entries, 11 to 58050
Data columns (total 8 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 geometry 30202 non-null geometry
1 quadkey 30202 non-null object
2 shapeName 30202 non-null object
3 shapeISO 30202 non-null object
4 shapeID 30202 non-null object
5 shapeGroup 30202 non-null object
6 shapeType 30202 non-null object
7 pop_count 30202 non-null float64
dtypes: float64(1), geometry(1), object(6)
memory usage: 2.1+ MBadmin_grids_gdf.head(2)| geometry | quadkey | shapeName | shapeISO | shapeID | shapeGroup | shapeType | pop_count | |
|---|---|---|---|---|---|---|---|---|
| 11 | POLYGON ((101.27197 2.92110, 101.27197 2.94304... | 13223202222021 | Klang | None | MYS-ADM2-3_0_0-B57 | MYS | ADM2 | 1948.905696 |
| 12 | POLYGON ((101.27197 2.89915, 101.27197 2.92110... | 13223202222023 | Klang | None | MYS-ADM2-3_0_0-B57 | MYS | ADM2 | 5633.984424 |
# Uncomment to view the generated grid on an interactive map
# admin_grids_gdf.explore()Save to file
rollout_date = "-".join(os.getcwd().split("/")[-2].split("-")[:3])
grid_save_path = f"./{rollout_date}-{COUNTRY_CODE}-rollout-grids.geojson"
admin_grids_gdf.to_file(grid_save_path, driver="GeoJSON", index=False)