Full-coverage grouped ingest of Statistics Canada (WDS). Enumerates the ENTIRE cube catalog via getAllCubesListLite (~8,200 cubes), then for each cube pulls the full-table CSV bulk download (getFullTableDownloadCSV -> zip URL on www150), streams the data CSV, and writes ONE Parquet per cube.…
Statistics Canada. Retrieved from https://www150.statcan.gc.ca. Compiled and redistributed by the Elkassabgi Data Library.
Processing: Retrieved from the official source, normalized to a long {series_key, obs_date, value} schema (period-start dates), de-duplicated, and stored as zstd Parquet. Compiled and redistributed by the Elkassabgi Data Library.
Full-coverage grouped ingest of Statistics Canada (WDS). Enumerates the ENTIRE cube catalog via getAllCubesListLite (~8,200 cubes), then for each cube pulls the full-table CSV bulk download (getFullTableDownloadCSV -> zip URL on www150), streams the data CSV, and writes ONE Parquet per cube. Parquet columns (grouped, one file per productId): series_key : StatCan VECTOR id (e.g. "v41690973") -- the canonical series key obs_date : date32 (REF_DATE parsed: YYYY / YYYY-MM / YYYY-MM-DD) value : float64 (null when suppressed/"..") geo : geography label uom : unit of measure coordinate : StatCan dimension coordinate (e.g. "1.1.2") status : STATUS flag (".."=unavailable, E=use w/ caution, F=too unreliable, etc.) License: statcan-open (Statistics Canada Open Licence). Memory is bounded: zips stream to a temp file, the CSV is parsed row-by-row, and rows are flushed to the Parquet writer in batches (so even the 2 GB cube is fine). Resumable: a cube with an existing .parquet + .done marker is skipped. Usage: python jobs/ingest_statcan.py --dry 5 # enumerate + process 5 small cubes, print python jobs/ingest_statcan.py # full run (all cubes) python jobs/ingest_statcan.py --workers 6 # set concurrency (default 6)
{
"@context": "https://schema.org/",
"@type": "Dataset",
"name": "Statistics Canada",
"description": "Full-coverage grouped ingest of Statistics Canada (WDS). Enumerates the ENTIRE cube catalog via getAllCubesListLite (~8,200 cubes), then for each cube pulls the full-table CSV bulk download (getFullTableDownloadCSV -> zip URL on www150), streams the data CSV, and writes ONE Parquet per cube.…",
"url": "https://econdatalibrary.com/statcan.html",
"identifier": "statcan",
"isAccessibleForFree": true,
"publisher": {
"@type": "Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"creator": {
"@type": "Organization",
"name": "Statistics Canada"
},
"license": "statcan-open",
"keywords": [
"demography",
"labour",
"output",
"prices",
"rates",
"statcan"
],
"temporalCoverage": "1946-01-01/2026-05-01",
"repeatFrequency": [
"P1Y",
"P1M",
"P3M"
],
"creditText": "Source: Statistics Canada (Open Licence)",
"citation": "Statistics Canada. Retrieved from https://www150.statcan.gc.ca. Compiled and redistributed by the Elkassabgi Data Library.",
"sameAs": [
"https://huggingface.co/datasets/econdatalibrary-statcan",
"https://zenodo.org/communities/econdatalibrary"
],
"distribution": [
{
"@type": "DataDownload",
"encodingFormat": "text/csv",
"contentUrl": "https://econdatalibrary.com/download.html?source=statcan"
}
]
}{
"@context": {
"@language": "en",
"@vocab": "https://schema.org/",
"cr": "http://mlcommons.org/croissant/",
"data": {
"@id": "cr:data",
"@type": "@json"
},
"dataType": {
"@id": "cr:dataType",
"@type": "@vocab"
},
"sc": "https://schema.org/",
"conformsTo": "dct:conformsTo",
"dct": "http://purl.org/dc/terms/"
},
"@type": "sc:Dataset",
"conformsTo": "http://mlcommons.org/croissant/1.0",
"name": "statcan",
"description": "Full-coverage grouped ingest of Statistics Canada (WDS). Enumerates the ENTIRE cube catalog via getAllCubesListLite (~8,200 cubes), then for each cube pulls the full-table CSV bulk download (getFullTableDownloadCSV -> zip URL on www150), streams the data CSV, and writes ONE Parquet per cube.…",
"url": "https://econdatalibrary.com/statcan.html",
"creator": {
"@type": "sc:Organization",
"name": "Statistics Canada"
},
"publisher": {
"@type": "sc:Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"license": "statcan-open",
"keywords": [
"demography",
"labour",
"output",
"prices",
"rates"
],
"temporalCoverage": "1946-01-01/2026-05-01",
"citation": "Statistics Canada. Retrieved from https://www150.statcan.gc.ca. Compiled and redistributed by the Elkassabgi Data Library.",
"distribution": [
{
"@type": "cr:FileObject",
"@id": "statcan-csv",
"name": "statcan-csv",
"description": "Per-series CSV, downloadable with a free API key.",
"contentUrl": "https://econdatalibrary.com/download.html?source=statcan",
"encodingFormat": "text/csv"
}
]
}