One account. Every library.
HF Data Library · Econ Data Library · more to come
FULL-COVERAGE ingest of the World Bank "Pink Sheet" (Commodity Markets). Two workbooks downloaded from https://www.worldbank.org/en/research/commodity-markets : * CMO-Historical-Data-Monthly.xlsx -> sheets "Monthly Prices", "Monthly Indices" * CMO-Historical-Data-Annual.xlsx -> sheets "Annual…
World Bank, Commodity Price Data (the Pink Sheet). Retrieved from https://www.worldbank.org/commodities. Compiled and redistributed by the Elkassabgi Data Library.
Processing: Retrieved from the official source, normalized to a long {series_key, obs_date, value} schema (period-start dates), de-duplicated, and stored as zstd Parquet. Compiled and redistributed by the Elkassabgi Data Library.
FULL-COVERAGE ingest of the World Bank "Pink Sheet" (Commodity Markets). Two workbooks downloaded from https://www.worldbank.org/en/research/commodity-markets : * CMO-Historical-Data-Monthly.xlsx -> sheets "Monthly Prices", "Monthly Indices" * CMO-Historical-Data-Annual.xlsx -> sheets "Annual Prices (Nominal)", "Annual Indices (Nominal)", "Annual Prices (Real)", "Annual Indices (Real)" We parse ALL commodity columns (energy, agriculture, fertilizers, metals, precious metals) AND all the price-index columns, for every frequency / nominal-real variant. GROUPED storage (mirrors jobs/ingest_eurostat.py + jobs/ingest_worldbank_esg.py): ONE Parquet per SHEET (the natural "dataset"); every commodity / index column is a series inside it -> long columns (series_key, obs_date, value). 6 sheets => 6 grouped Parquet files for the whole source. Never one-file-per-series. series_key = "<freq>:<variant>:<slug>" e.g. "m:price:crude_oil_brent", "m:index:energy", "a:price_real:gold", "a:index_nominal:precious_metals". The slug is a stable URL-safe id derived from the commodity / index label. SHEET LAYOUTS (observed June-2026 monthly / March-2026 annual vintage): Monthly Prices : names row 5, units row 6, data row 7+ ; period col A = "YYYYMmm". Monthly Indices : staggered header rows 6-9 (one non-blank cell per column), data row 10+ ; period col A = "YYYYMmm". Annual Prices (*) : names row 7, units row 8, data row 9+ ; period col A = year. Annual Indices(*) : staggered header rows 6-9, data row 10+ ; period col A = year. Missing values are the replacement char U+FFFD (shows as a box) or non-numeric -> skipped. License: cc-by-4.0 (reservable id from configs/sources.yaml -> worldbank_pink). Does NOT touch data/catalog.db or data/clean/ (per task constraints). Run: python jobs/ingest_worldbank_pink.py --dry # parse, print per-series summary, no writes python jobs/ingest_worldbank_pink.py # full run -> data/clean_full/worldbank_pink/ python jobs/ingest_worldbank_pink.py --no-download # reuse already-downloaded xlsx
{
"@context": "https://schema.org/",
"@type": "Dataset",
"name": "World Bank Pink Sheet (commodities)",
"description": "FULL-COVERAGE ingest of the World Bank \"Pink Sheet\" (Commodity Markets). Two workbooks downloaded from https://www.worldbank.org/en/research/commodity-markets : * CMO-Historical-Data-Monthly.xlsx -> sheets \"Monthly Prices\", \"Monthly Indices\" * CMO-Historical-Data-Annual.xlsx -> sheets \"Annual…",
"url": "https://econdatalibrary.com/worldbank_pink.html",
"identifier": "worldbank_pink",
"isAccessibleForFree": true,
"publisher": {
"@type": "Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"creator": {
"@type": "Organization",
"name": "World Bank Pink Sheet (commodities)"
},
"license": "https://creativecommons.org/licenses/by/4.0/",
"keywords": [
"commodities",
"worldbank_pink"
],
"temporalCoverage": "1960-01-01/2026-05-01",
"repeatFrequency": "P1M",
"creditText": "Source: World Bank Commodity Markets (CC BY 4.0)",
"citation": "World Bank, Commodity Price Data (the Pink Sheet). Retrieved from https://www.worldbank.org/commodities. Compiled and redistributed by the Elkassabgi Data Library.",
"sameAs": [
"https://huggingface.co/datasets/econdatalibrary-worldbank_pink",
"https://zenodo.org/communities/econdatalibrary"
],
"distribution": [
{
"@type": "DataDownload",
"encodingFormat": "text/csv",
"contentUrl": "https://econdatalibrary.com/download.html?source=worldbank_pink",
"license": "https://creativecommons.org/licenses/by/4.0/"
}
]
}{
"@context": {
"@language": "en",
"@vocab": "https://schema.org/",
"cr": "http://mlcommons.org/croissant/",
"data": {
"@id": "cr:data",
"@type": "@json"
},
"dataType": {
"@id": "cr:dataType",
"@type": "@vocab"
},
"sc": "https://schema.org/",
"conformsTo": "dct:conformsTo",
"dct": "http://purl.org/dc/terms/"
},
"@type": "sc:Dataset",
"conformsTo": "http://mlcommons.org/croissant/1.0",
"name": "worldbank_pink",
"description": "FULL-COVERAGE ingest of the World Bank \"Pink Sheet\" (Commodity Markets). Two workbooks downloaded from https://www.worldbank.org/en/research/commodity-markets : * CMO-Historical-Data-Monthly.xlsx -> sheets \"Monthly Prices\", \"Monthly Indices\" * CMO-Historical-Data-Annual.xlsx -> sheets \"Annual…",
"url": "https://econdatalibrary.com/worldbank_pink.html",
"creator": {
"@type": "sc:Organization",
"name": "World Bank Pink Sheet (commodities)"
},
"publisher": {
"@type": "sc:Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"license": "https://creativecommons.org/licenses/by/4.0/",
"keywords": [
"commodities"
],
"temporalCoverage": "1960-01-01/2026-05-01",
"citation": "World Bank, Commodity Price Data (the Pink Sheet). Retrieved from https://www.worldbank.org/commodities. Compiled and redistributed by the Elkassabgi Data Library.",
"distribution": [
{
"@type": "cr:FileObject",
"@id": "worldbank_pink-csv",
"name": "worldbank_pink-csv",
"description": "Per-series CSV, downloadable with a free API key.",
"contentUrl": "https://econdatalibrary.com/download.html?source=worldbank_pink",
"encodingFormat": "text/csv"
}
]
}