One account. Every library.
HF Data Library · Econ Data Library · more to come
Full-coverage grouped ingest of U.S. Treasury FiscalData. Enumerates the ENTIRE FiscalData catalog (every dataset/endpoint discovered from the FiscalData dataset pages -> data/_treasury_catalog_final.json) and pulls each endpoint in full via the REST API (JSON, page[size]=10000, paginated to the…
U.S. Department of the Treasury (Fiscal Data). Retrieved from https://fiscaldata.treasury.gov. Compiled and redistributed by the Elkassabgi Data Library.
Processing: Retrieved from the official source, normalized to a long {series_key, obs_date, value} schema (period-start dates), de-duplicated, and stored as zstd Parquet. Compiled and redistributed by the Elkassabgi Data Library.
Full-coverage grouped ingest of U.S. Treasury FiscalData. Enumerates the ENTIRE FiscalData catalog (every dataset/endpoint discovered from the FiscalData dataset pages -> data/_treasury_catalog_final.json) and pulls each endpoint in full via the REST API (JSON, page[size]=10000, paginated to the last page). GROUPED storage: ONE Parquet per ENDPOINT (the "cube") under data/clean_full/treasury/<slug>__<endpoint_leaf>.parquet Each file holds the endpoint's FULL relational table (all original columns), plus: series_key : the endpoint path (e.g. "v2/accounting/od/debt_to_penny") -- the cube id obs_date : date32, the parsed primary date field (record_date / reporting_date / ...) All other source columns are preserved verbatim as strings (Treasury returns decimal strings; keeping them avoids lossy float() of multi-trillion-dollar amounts). This makes the written row count equal the source-published total-count, so coverage is verifiable. Anti-bloat: one file per endpoint (~130 files for the whole source), NOT one-per-series. License: us-public-domain (FiscalData is U.S. Government public domain). Polite UA, retry/backoff, single-stream paging per endpoint (concurrency<=6 across endpoints), bounded memory (rows flushed to the ParquetWriter in row-group batches). Usage: python jobs/ingest_treasury.py --dry # list catalog + planned files, no writes python jobs/ingest_treasury.py # full run (all endpoints) python jobs/ingest_treasury.py --workers 4 # endpoint-level concurrency (default 4) python jobs/ingest_treasury.py --only debt_to_penny # substring filter on endpoint
{
"@context": "https://schema.org/",
"@type": "Dataset",
"name": "U.S. Treasury FiscalData + Yield Curve",
"description": "Full-coverage grouped ingest of U.S. Treasury FiscalData. Enumerates the ENTIRE FiscalData catalog (every dataset/endpoint discovered from the FiscalData dataset pages -> data/_treasury_catalog_final.json) and pulls each endpoint in full via the REST API (JSON, page[size]=10000, paginated to the…",
"url": "https://econdatalibrary.com/treasury.html",
"identifier": "treasury",
"isAccessibleForFree": true,
"publisher": {
"@type": "Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"creator": {
"@type": "Organization",
"name": "U.S. Treasury FiscalData + Yield Curve"
},
"license": "https://www.usa.gov/government-works",
"keywords": [
"fiscal",
"treasury"
],
"temporalCoverage": "1993-04-01/2026-07-31",
"repeatFrequency": [
"P1D",
"P1M"
],
"citation": "U.S. Department of the Treasury (Fiscal Data). Retrieved from https://fiscaldata.treasury.gov. Compiled and redistributed by the Elkassabgi Data Library.",
"sameAs": [
"https://huggingface.co/datasets/econdatalibrary-treasury",
"https://zenodo.org/communities/econdatalibrary"
],
"distribution": [
{
"@type": "DataDownload",
"encodingFormat": "text/csv",
"contentUrl": "https://econdatalibrary.com/download.html?source=treasury",
"license": "https://www.usa.gov/government-works"
}
]
}{
"@context": {
"@language": "en",
"@vocab": "https://schema.org/",
"cr": "http://mlcommons.org/croissant/",
"data": {
"@id": "cr:data",
"@type": "@json"
},
"dataType": {
"@id": "cr:dataType",
"@type": "@vocab"
},
"sc": "https://schema.org/",
"conformsTo": "dct:conformsTo",
"dct": "http://purl.org/dc/terms/"
},
"@type": "sc:Dataset",
"conformsTo": "http://mlcommons.org/croissant/1.0",
"name": "treasury",
"description": "Full-coverage grouped ingest of U.S. Treasury FiscalData. Enumerates the ENTIRE FiscalData catalog (every dataset/endpoint discovered from the FiscalData dataset pages -> data/_treasury_catalog_final.json) and pulls each endpoint in full via the REST API (JSON, page[size]=10000, paginated to the…",
"url": "https://econdatalibrary.com/treasury.html",
"creator": {
"@type": "sc:Organization",
"name": "U.S. Treasury FiscalData + Yield Curve"
},
"publisher": {
"@type": "sc:Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"license": "https://www.usa.gov/government-works",
"keywords": [
"fiscal"
],
"temporalCoverage": "1993-04-01/2026-07-31",
"citation": "U.S. Department of the Treasury (Fiscal Data). Retrieved from https://fiscaldata.treasury.gov. Compiled and redistributed by the Elkassabgi Data Library.",
"distribution": [
{
"@type": "cr:FileObject",
"@id": "treasury-csv",
"name": "treasury-csv",
"description": "Per-series CSV, downloadable with a free API key.",
"contentUrl": "https://econdatalibrary.com/download.html?source=treasury",
"encodingFormat": "text/csv"
}
]
}