%matplotlib inline
%reload_ext autoreload
%autoreload 2Generate rollout grids
import sys
sys.path.append("../../../")
import os
from pathlib import Path
import geopandas as gpd
import pandas as pd
from povertymapping.iso3 import get_region_name
from povertymapping.rollout_grids import get_region_filtered_bingtile_gridsGenerate Roll-out Grids: Vietnam
This notebook generates the rollout grid tiles over the country. The output file is used as an input for Step 3, where we will run the trained model over the set of grids.
The generated grids are set at 2.4km (zoom level 14), matching the grids used during model training. The grids are also filtered to only include populated areas based on Meta’s High Resolution Settlement Layer (HRSL) data.
Set country-specific parameters
COUNTRY_CODE = "vn"
REGION = get_region_name(COUNTRY_CODE, code="alpha-2").lower()
ADMIN_LVL = "ADM2"
ZOOM_LEVEL = 14Generate Grids
admin_grids_gdf = get_region_filtered_bingtile_grids(
REGION,
admin_lvl=ADMIN_LVL,
quadkey_lvl=ZOOM_LEVEL,
use_cache=True,
n_workers=4,
max_batch_size=1000,
)2023-04-17 18:07:35.408 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:264 - No cached grids file found. Generating grids file :/home/alron/.cache/geowrangler/quadkey_grids/vietnam_14_ADM2_populated_admin_grids.geojson
2023-04-17 18:07:35.421 | DEBUG | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:281 - Loading boundaries for region vietnam and admin level ADM2
2023-04-17 18:07:35.956 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:287 - Generating grids for region vietnam and admin level ADM2 at quadkey level 142023-04-17 18:08:26.716 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:293 - Generated 64782 grids for region vietnam and admin level ADM2 at quadkey level 14
2023-04-17 18:08:26.723 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:299 - Assigning grids to admin areas using metric crs epsg:3857
2023-04-17 18:08:31.630 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:305 - Getting vietnam population data for filtering grids
2023-04-17 18:08:32.883 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:307 - Computing population zonal stats per grid
2023-04-17 18:08:32.884 | INFO | povertymapping.rollout_grids:compute_raster_stats:108 - Batching call to create raster_zonal stats for 64782 grids for file size 13271.962124 Mb
2023-04-17 18:08:33.000 | INFO | povertymapping.rollout_grids:compute_raster_stats:119 - Created 64 for 64 splits of 1000
2023-04-17 18:08:33.001 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 0 with index (0/1012 in 4 parallel threads/home/alron/unicef-ai4d-poverty-mapping/env/lib/python3.9/site-packages/rasterio/features.py:362: NotGeoreferencedWarning: Dataset has no geotransform, gcps, or rpcs. The identity matrix will be returned.
_rasterize(valid_shapes, out, transform, all_touched, merge_alg)
2023-04-17 18:08:41.351 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:08:41.353 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:08:41.355 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 1 with index (0/1012 in 4 parallel threads2023-04-17 18:08:49.855 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:08:49.858 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:08:49.859 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 2 with index (0/1012 in 4 parallel threads2023-04-17 18:08:57.994 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:08:57.997 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:08:57.999 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 3 with index (0/1012 in 4 parallel threads2023-04-17 18:09:06.275 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:06.279 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:06.281 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 4 with index (0/1012 in 4 parallel threads2023-04-17 18:09:14.411 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:14.413 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:14.415 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 5 with index (0/1012 in 4 parallel threads2023-04-17 18:09:22.479 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:22.482 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:22.484 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 6 with index (0/1012 in 4 parallel threads2023-04-17 18:09:30.553 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:30.557 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:30.558 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 7 with index (0/1012 in 4 parallel threads2023-04-17 18:09:39.018 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:39.021 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:39.023 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 8 with index (0/1012 in 4 parallel threads2023-04-17 18:09:47.367 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:47.370 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:47.372 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 9 with index (0/1012 in 4 parallel threads2023-04-17 18:09:55.978 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:09:55.982 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:09:55.983 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 10 with index (0/1012 in 4 parallel threads2023-04-17 18:10:04.515 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:10:04.518 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:10:04.520 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 11 with index (0/1012 in 4 parallel threads2023-04-17 18:10:13.884 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:10:13.891 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:10:13.892 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 12 with index (0/1012 in 4 parallel threads2023-04-17 18:10:23.237 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:10:23.260 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:10:23.261 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 13 with index (0/1012 in 4 parallel threads2023-04-17 18:10:32.218 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:10:32.234 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:10:32.236 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 14 with index (0/1011 in 4 parallel threads2023-04-17 18:10:40.979 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:10:40.982 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:10:40.983 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 15 with index (0/1011 in 4 parallel threads2023-04-17 18:10:50.852 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:10:50.855 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:10:50.856 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 16 with index (0/1011 in 4 parallel threads2023-04-17 18:11:01.266 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:11:01.269 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:11:01.270 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 17 with index (0/1011 in 4 parallel threads2023-04-17 18:11:10.850 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:11:10.862 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:11:10.863 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 18 with index (0/1011 in 4 parallel threads2023-04-17 18:11:20.934 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:11:20.945 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:11:20.947 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 19 with index (0/1011 in 4 parallel threads2023-04-17 18:11:31.512 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:11:31.515 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:11:31.517 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 20 with index (0/1011 in 4 parallel threads2023-04-17 18:11:47.768 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:11:47.785 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:11:47.786 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 21 with index (0/1011 in 4 parallel threads2023-04-17 18:11:59.898 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:11:59.912 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:11:59.913 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 22 with index (0/1011 in 4 parallel threads2023-04-17 18:12:13.258 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:12:13.276 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:12:13.277 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 23 with index (0/1011 in 4 parallel threads2023-04-17 18:12:30.714 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:12:30.729 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:12:30.730 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 24 with index (0/1011 in 4 parallel threads2023-04-17 18:12:49.527 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:12:49.533 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:12:49.535 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 25 with index (0/1011 in 4 parallel threads2023-04-17 18:13:09.819 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:13:09.834 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:13:09.835 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 26 with index (0/1011 in 4 parallel threads2023-04-17 18:13:34.034 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:13:34.050 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:13:34.052 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 27 with index (0/1011 in 4 parallel threads2023-04-17 18:14:01.120 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:14:01.132 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:14:01.134 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 28 with index (0/1011 in 4 parallel threads2023-04-17 18:15:54.425 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:15:54.450 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:15:54.452 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 29 with index (0/1011 in 4 parallel threads2023-04-17 18:17:13.779 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:17:13.785 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:17:13.787 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 30 with index (0/1011 in 4 parallel threads2023-04-17 18:18:08.960 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:18:08.965 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:18:08.967 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 31 with index (0/1011 in 4 parallel threads2023-04-17 18:18:23.363 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:18:23.366 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:18:23.367 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 32 with index (0/1011 in 4 parallel threads2023-04-17 18:18:33.718 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:18:33.721 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:18:33.722 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 33 with index (0/1011 in 4 parallel threads2023-04-17 18:18:42.450 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:18:42.452 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:18:42.454 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 34 with index (0/1011 in 4 parallel threads2023-04-17 18:18:50.678 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:18:50.681 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:18:50.683 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 35 with index (0/1011 in 4 parallel threads2023-04-17 18:18:59.055 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:18:59.058 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:18:59.059 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 36 with index (0/1011 in 4 parallel threads2023-04-17 18:19:07.339 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:07.344 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:07.345 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 37 with index (0/1011 in 4 parallel threads2023-04-17 18:19:15.773 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:15.775 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:15.777 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 38 with index (0/1011 in 4 parallel threads2023-04-17 18:19:23.853 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:23.856 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:23.857 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 39 with index (0/1011 in 4 parallel threads2023-04-17 18:19:31.950 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:31.953 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:31.954 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 40 with index (0/1011 in 4 parallel threads2023-04-17 18:19:40.082 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:40.084 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:40.086 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 41 with index (0/1011 in 4 parallel threads2023-04-17 18:19:48.225 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:48.227 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:48.229 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 42 with index (0/1011 in 4 parallel threads2023-04-17 18:19:56.776 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:19:56.779 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:19:56.780 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 43 with index (0/1011 in 4 parallel threads2023-04-17 18:20:05.695 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:05.697 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:05.699 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 44 with index (0/1011 in 4 parallel threads2023-04-17 18:20:14.460 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:14.462 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:14.464 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 45 with index (0/1011 in 4 parallel threads2023-04-17 18:20:23.497 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:23.511 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:23.513 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 46 with index (0/1011 in 4 parallel threads2023-04-17 18:20:31.731 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:31.733 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:31.734 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 47 with index (0/1011 in 4 parallel threads2023-04-17 18:20:39.885 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:39.888 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:39.889 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 48 with index (0/1011 in 4 parallel threads2023-04-17 18:20:47.888 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:47.891 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:47.892 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 49 with index (0/1011 in 4 parallel threads2023-04-17 18:20:56.146 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:20:56.148 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:20:56.150 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 50 with index (0/1011 in 4 parallel threads2023-04-17 18:21:04.089 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:04.091 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:04.093 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 51 with index (0/1011 in 4 parallel threads2023-04-17 18:21:12.336 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:12.339 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:12.340 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 52 with index (0/1011 in 4 parallel threads2023-04-17 18:21:20.426 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:20.429 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:20.430 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 53 with index (0/1011 in 4 parallel threads2023-04-17 18:21:28.468 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:28.471 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:28.473 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 54 with index (0/1011 in 4 parallel threads2023-04-17 18:21:36.649 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:36.651 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:36.653 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 55 with index (0/1011 in 4 parallel threads2023-04-17 18:21:44.808 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:44.811 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:44.813 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 56 with index (0/1011 in 4 parallel threads2023-04-17 18:21:53.091 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:21:53.094 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:21:53.095 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 57 with index (0/1011 in 4 parallel threads2023-04-17 18:22:01.314 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:01.317 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:01.319 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 58 with index (0/1011 in 4 parallel threads2023-04-17 18:22:09.467 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:09.469 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:09.471 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 59 with index (0/1011 in 4 parallel threads2023-04-17 18:22:17.614 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:17.617 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:17.618 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 60 with index (0/1011 in 4 parallel threads2023-04-17 18:22:25.675 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:25.678 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:25.680 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 61 with index (0/1011 in 4 parallel threads2023-04-17 18:22:33.866 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:33.869 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:33.870 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 62 with index (0/1011 in 4 parallel threads2023-04-17 18:22:40.917 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:40.920 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:40.921 | INFO | povertymapping.rollout_grids:compute_raster_stats:132 - Creating raster zonal stats for batch 63 with index (0/1011 in 4 parallel threads2023-04-17 18:22:47.041 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:207 - Completed parallel raster zonal stats for 4 threads
2023-04-17 18:22:47.047 | INFO | povertymapping.rollout_grids:compute_parallel_raster_zonal_stats:209 - Concatenated parallel raster zonal stats for 4 threads
2023-04-17 18:22:47.068 | INFO | povertymapping.rollout_grids:compute_raster_stats:145 - Completed raster zonal stats for 64 batches
2023-04-17 18:22:47.115 | INFO | povertymapping.rollout_grids:compute_raster_stats:147 - Concatenated raster zonal stats for 64 batches
2023-04-17 18:22:47.146 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:318 - Filtering unpopulated grids based on population data
2023-04-17 18:22:47.269 | INFO | povertymapping.rollout_grids:get_region_filtered_bingtile_grids:321 - Filtered admin grid count: 50880Explore Populated Grids
admin_grids_gdf.info()<class 'geopandas.geodataframe.GeoDataFrame'>
Int64Index: 50880 entries, 6 to 63293
Data columns (total 8 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 geometry 50880 non-null geometry
1 quadkey 50880 non-null object
2 shapeName 50880 non-null object
3 shapeISO 50880 non-null object
4 shapeID 50880 non-null object
5 shapeGroup 50880 non-null object
6 shapeType 50880 non-null object
7 pop_count 50880 non-null float64
dtypes: float64(1), geometry(1), object(6)
memory usage: 3.5+ MBadmin_grids_gdf.head(2)| geometry | quadkey | shapeName | shapeISO | shapeID | shapeGroup | shapeType | pop_count | |
|---|---|---|---|---|---|---|---|---|
| 6 | POLYGON ((103.46924 9.29731, 103.46924 9.31899... | 13223003120321 | Phu Quoc | None | VNM-ADM2-3_0_0-B967 | VNM | ADM2 | 500.753008 |
| 11 | POLYGON ((103.44727 9.29731, 103.44727 9.31899... | 13223003120320 | Phu Quoc | None | VNM-ADM2-3_0_0-B967 | VNM | ADM2 | 6.106744 |
# Uncomment to view the generated grid on an interactive map
# admin_grids_gdf.explore()Save to file
rollout_date = "-".join(os.getcwd().split("/")[-2].split("-")[:3])
grid_save_path = f"./{rollout_date}-{COUNTRY_CODE}-rollout-grids.geojson"
admin_grids_gdf.to_file(grid_save_path, driver="GeoJSON", index=False)