One account. Every library.
HF Data Library · Econ Data Library · more to come
FULL-COVERAGE grouped ingest of U.S. Census Bureau TIME-SERIES datasets. Enumerates every dataset in https://api.census.gov/data.json, keeps the ones whose c_dataset starts with "timeseries/", and pulls each one FULLY via the Census API (api.census.gov). ONE Parquet per dataset under…
U.S. Census Bureau, Economic Indicators Time Series. Retrieved from https://www.census.gov. Compiled and redistributed by the Elkassabgi Data Library.
Processing: Retrieved from the official source, normalized to a long {series_key, obs_date, value} schema (period-start dates), de-duplicated, and stored as zstd Parquet. Compiled and redistributed by the Elkassabgi Data Library.
FULL-COVERAGE grouped ingest of U.S. Census Bureau TIME-SERIES datasets. Enumerates every dataset in https://api.census.gov/data.json, keeps the ones whose c_dataset starts with "timeseries/", and pulls each one FULLY via the Census API (api.census.gov). ONE Parquet per dataset under data/clean_full/census/, in long "wide-record" form: every retrievable variable as a column (Census returns strings), plus a constructed `series_key` (the non-time dimension columns joined) and the time/period columns. us-public-domain. Strategy per dataset (driven by data.json + variables.json + geography.json): * time-based ("time" required/present) -> one pull per no-parent geo level with time=from+1900 (full history in a single response); plus child geo levels iterated over their required parent(s). * YEAR / year+quarter only (asm area/benchmark, qwi) -> enumerate the year(s) (and quarters) discovered from the API, iterate geographies. * no predicate at all (pseo, some asm value/benchmark) -> single pull per geo. The Census API caps get= at 50 variables, so variables are chunked (<=45) with the key columns repeated in every chunk and the chunks merged on those keys. Usage: python jobs/ingest_census.py --list # just print the catalog census python jobs/ingest_census.py --only timeseries/govs # one dataset python jobs/ingest_census.py --group eits # one group python jobs/ingest_census.py # FULL run (all 93)
{
"@context": "https://schema.org/",
"@type": "Dataset",
"name": "U.S. Census Bureau",
"description": "International Trade; International Data Base; Economic Indicators. U.S. Census Bureau. 2,993 series, 1942–2026. Free download under public domain.",
"url": "https://econdatalibrary.com/census",
"identifier": "census",
"isAccessibleForFree": true,
"publisher": {
"@type": "Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"includedInDataCatalog": {
"@type": "DataCatalog",
"@id": "https://econdatalibrary.com/#catalog",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com/catalog"
},
"creator": {
"@type": "Organization",
"name": "U.S. Census Bureau"
},
"license": "https://www.usa.gov/government-works",
"keywords": [
"Economy",
"Macroeconomics",
"census"
],
"temporalCoverage": "1942-12-31/2026-12-31",
"repeatFrequency": "P1M",
"citation": "U.S. Census Bureau, Economic Indicators Time Series. Retrieved from https://www.census.gov. Compiled and redistributed by the Elkassabgi Data Library.",
"sameAs": [
"https://huggingface.co/datasets/econdatalibrary-census",
"https://zenodo.org/communities/econdatalibrary"
],
"distribution": [
{
"@type": "DataDownload",
"name": "census — CSV",
"encodingFormat": "text/csv",
"contentUrl": "https://econdatalibrary.com/download.html?source=census",
"license": "https://www.usa.gov/government-works"
}
]
}{
"@context": {
"@language": "en",
"@vocab": "https://schema.org/",
"cr": "http://mlcommons.org/croissant/",
"data": {
"@id": "cr:data",
"@type": "@json"
},
"dataType": {
"@id": "cr:dataType",
"@type": "@vocab"
},
"sc": "https://schema.org/",
"conformsTo": "dct:conformsTo",
"dct": "http://purl.org/dc/terms/"
},
"@type": "sc:Dataset",
"conformsTo": "http://mlcommons.org/croissant/1.0",
"name": "census",
"description": "International Trade; International Data Base; Economic Indicators. U.S. Census Bureau. 2,993 series, 1942–2026. Free download under public domain.",
"url": "https://econdatalibrary.com/census",
"creator": {
"@type": "sc:Organization",
"name": "U.S. Census Bureau"
},
"publisher": {
"@type": "sc:Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"license": "https://www.usa.gov/government-works",
"keywords": [
"Economy",
"Macroeconomics"
],
"temporalCoverage": "1942-12-31/2026-12-31",
"citation": "U.S. Census Bureau, Economic Indicators Time Series. Retrieved from https://www.census.gov. Compiled and redistributed by the Elkassabgi Data Library.",
"distribution": [
{
"@type": "cr:FileObject",
"@id": "census-csv",
"name": "census-csv",
"description": "Per-series CSV, downloadable with a free API key.",
"contentUrl": "https://econdatalibrary.com/download.html?source=census",
"encodingFormat": "text/csv"
}
]
}