One account. Every library.
HF Data Library · Econ Data Library · more to come
FULL-coverage ingest of the Wikidata econ/finance ENTITY set (reference data; CC0). Wikidata is a knowledge graph, not a time series. We pull the ENTIRE econ/finance entity universe and store it as GROUPED Parquet -- one file per "cube", each holding every entity with a `series_key` column (the…
Wikidata. Retrieved from https://www.wikidata.org. Compiled and redistributed by the Elkassabgi Data Library.
Processing: Retrieved from the official source, normalized to a long {series_key, obs_date, value} schema (period-start dates), de-duplicated, and stored as zstd Parquet. Compiled and redistributed by the Elkassabgi Data Library.
FULL-coverage ingest of the Wikidata econ/finance ENTITY set (reference data; CC0). Wikidata is a knowledge graph, not a time series. We pull the ENTIRE econ/finance entity universe and store it as GROUPED Parquet -- one file per "cube", each holding every entity with a `series_key` column (the Wikidata QID): data/clean_full/wikidata/ companies.parquet one row per company that has a stock ticker (P249) OR an ISIN (P946) stock_exchanges.parquet one row per stock exchange (P31/P279* Q11691) currencies.parquet one row per currency (P31/P279* Q8142) _manifest.json per-cube row counts + the SPARQL-published total it targets Multi-valued attributes (tickers, ISINs, exchanges, industries, ...) are aggregated per entity with SPARQL GROUP_CONCAT (separator '|') so each result row == one entity; we paginate the DISTINCT entity set with LIMIT/OFFSET over a stable ORDER BY ?x to pull ALL of them (not a 250-row sample). License = cc0 (the reservable id in configs/sources.yaml). Polite UA, retry/backoff, single-stream paging (well under the concurrency<=6 cap; WDQS rate-limits aggressive parallelism anyway). Usage: python jobs/ingest_wikidata.py --probe # print SPARQL totals, no writes python jobs/ingest_wikidata.py --dry # pull 1 small page per cube, print, no writes python jobs/ingest_wikidata.py # FULL run -> grouped Parquet
{
"@context": "https://schema.org/",
"@type": "Dataset",
"name": "Wikidata (econ/finance subset)",
"description": "FULL-coverage ingest of the Wikidata econ/finance ENTITY set (reference data; CC0). Wikidata is a knowledge graph, not a time series. We pull the ENTIRE econ/finance entity universe and store it as GROUPED Parquet -- one file per \"cube\", each holding every entity with a `series_key` column (the…",
"url": "https://econdatalibrary.com/wikidata.html",
"identifier": "wikidata",
"isAccessibleForFree": true,
"publisher": {
"@type": "Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"creator": {
"@type": "Organization",
"name": "Wikidata (econ/finance subset)"
},
"license": "https://creativecommons.org/publicdomain/zero/1.0/",
"keywords": [
"reference",
"wikidata"
],
"temporalCoverage": "1520-11-16/2022-04-08",
"citation": "Wikidata. Retrieved from https://www.wikidata.org. Compiled and redistributed by the Elkassabgi Data Library.",
"sameAs": [
"https://huggingface.co/datasets/econdatalibrary-wikidata",
"https://zenodo.org/communities/econdatalibrary"
],
"distribution": [
{
"@type": "DataDownload",
"encodingFormat": "text/csv",
"contentUrl": "https://econdatalibrary.com/download.html?source=wikidata",
"license": "https://creativecommons.org/publicdomain/zero/1.0/"
}
]
}{
"@context": {
"@language": "en",
"@vocab": "https://schema.org/",
"cr": "http://mlcommons.org/croissant/",
"data": {
"@id": "cr:data",
"@type": "@json"
},
"dataType": {
"@id": "cr:dataType",
"@type": "@vocab"
},
"sc": "https://schema.org/",
"conformsTo": "dct:conformsTo",
"dct": "http://purl.org/dc/terms/"
},
"@type": "sc:Dataset",
"conformsTo": "http://mlcommons.org/croissant/1.0",
"name": "wikidata",
"description": "FULL-coverage ingest of the Wikidata econ/finance ENTITY set (reference data; CC0). Wikidata is a knowledge graph, not a time series. We pull the ENTIRE econ/finance entity universe and store it as GROUPED Parquet -- one file per \"cube\", each holding every entity with a `series_key` column (the…",
"url": "https://econdatalibrary.com/wikidata.html",
"creator": {
"@type": "sc:Organization",
"name": "Wikidata (econ/finance subset)"
},
"publisher": {
"@type": "sc:Organization",
"name": "Econ Data Library",
"url": "https://econdatalibrary.com"
},
"license": "https://creativecommons.org/publicdomain/zero/1.0/",
"keywords": [
"reference"
],
"temporalCoverage": "1520-11-16/2022-04-08",
"citation": "Wikidata. Retrieved from https://www.wikidata.org. Compiled and redistributed by the Elkassabgi Data Library.",
"distribution": [
{
"@type": "cr:FileObject",
"@id": "wikidata-csv",
"name": "wikidata-csv",
"description": "Per-series CSV, downloadable with a free API key.",
"contentUrl": "https://econdatalibrary.com/download.html?source=wikidata",
"encodingFormat": "text/csv"
}
]
}