Data Catalog¶

A verified inventory of every CDC data resource across the three-repo pipeline: pulse-code (explore), then health (archive), then health-charts (visualize). It records each source's live status, refresh cadence, and where it lands.

How to read this page:

• Source system: the upstream CDC/NCHS/NCI program and its API.
• Registry: the Python module in health/src/ that defines the datasets.
• Archive: where the scheduled job commits CSVs under health/data/.
• Charted: whether health-charts renders it today.
• Activity: taken from the latest data date in the committed archive and the GitHub Actions schedule, checked 2026-08-24.

The pipeline at a glance¶

pulse-code  →  health  →  health-charts
(explore)      (archive)   (visualize)

Query graduates to archive: 23 of pulse-code's 59 saved WONDER queries have a matching fetch_*.py script here. The rest stay exploration-only for now.

Source systems¶

Ten upstream systems are wired up. Six feed the scheduled archive. Three (GRASP, NSSP, NIS) are query-only in pulse/health and are not committed to data/. EPHT is cataloged with a working discovery client, but its archiver is not scheduled yet (see section 9). PLACES is the one source too large to commit at all: it is mirrored into a public Dolt database, with only small derived slices tracked here (see section 10).

Who collects it, and how¶

The center and collection method behind each system (health-charts renders the same taxonomy per chart from src/lib/sources.js):

Activity verdict. All five scheduled jobs are healthy. Weekly respiratory, wastewater, measles, and NHSN archives are current through mid-August 2026. The annual sources (WONDER, DQS, SEER) correctly sit at 2024, the newest full year those programs have released. GRASP, NSSP, and NIS carry no staleness risk because they are fetched live, not stored.

1. CDC WONDER (data/raw/wonder/)¶

CDC's public query system (mortality, births, natality, VAERS, environmental), back to 1968 via an unauthenticated XML API. 49 saved XML query templates live in src/wonder/queries/; 9 are wrapped in fetch_*.py scripts that run weekly and commit a CSV.

Exploration-only in pulse-code (no archive yet): cancer and AIDS by state, fetal deaths by cause/race, PM2.5 by state/year, STD and TB cases, unintentional injuries, plus demographic slices of COVID, opioid, and maternal mortality.

2. CDC Open Data (Socrata) (data/raw/cdc_open/, data/processed/cdc_open/)¶

The largest registry: 68 data.cdc.gov datasets, plus 1 multi-year composite (state life expectancy) and 3 CDC WCMS measles JSON endpoints. 16 are charted today; 10 are disabled in the bulk downloader because they exceed GitHub's file-size limit at full resolution and nothing charts them yet (still queryable live).

Charted (16): life expectancy, provisional mortality rates, historical death rates, quarterly birth indicators, NWSS wastewater (COVID/flu/RSV/measles/H5), respiratory % of deaths, nursing-home respiratory and vaccination (NHSN), weekly respiratory vaccination (NIS-ACM), SchoolVaxView kindergarten coverage, weekly and annual measles, and BEAM foodborne isolates.

Backlog (collected but not yet charted): leading causes of death, PLACES county/city, disability, NHANES measured obesity, VSRR overdose, NNDSS measles and STI tables (chlamydia/gonorrhea/syphilis), HAI/AMR (MRSA, CRE/CRAB/ESBL, C. diff, candidemia), ARI activity, RESP-LENS, NVSN pediatric positivity, and cumulative RSV/COVID hospitalization estimates. Run cdc-open list --uncharted for the full list.

Disabled (over GitHub size limit, uncharted): covid_conditions, rsv_net, covid_net, drug_overdose_county, nssp_ed_visits, resp_coverage_adults, covid_coverage_adults, nndss_weekly, chronic_disease_indicators. Each stays queryable via cdc_open query <id>. (epidemic_trends_rt is also skipped by the bulk downloader but is charted; see below.)

Near-real-time and forecasting (CFA)¶

The datasets closest to real time, added because they lead the survey and hospital archives rather than lag them:

The CFA nowcast (5dqz-y4ea) refreshes several times a week; the wastewater series refresh weekly. Raw pulls are git-ignored; only the small rollups commit.

3. NCHS DQS (Health, United States) (data/raw/dqs/)¶

A unified query layer over CDC's national surveys (NHANES, NHIS, NHAMCS, NVSS, NPALS, NHCS, plus BLS/CMS/MTF). 28 datasets sharing one tidy Socrata schema, across 17 topics. Only 3 are charted (drug_overdose_by_type.csv, low_birthweight_by_state.csv, national_health_spending.csv); the other 25 are the backlog: chronic disease and risk factors, nutrition, oral health, disability, self-reported health, the health-care system (beds, ED visits, utilization), workforce, spending, and long-term care.

4. WISQARS (data/raw/wisqars/)¶

Fatal injury, firearm, suicide, homicide, and overdose data. 5 datasets: injury_mortality (1999-2016), and the mapping series injury_national / injury_state / injury_county / injury_census_tract (2019-present). Charted: national monthly rates (drug OD, suicide, homicide, firearm) plus a WISQARS+WONDER suicide_by_sex blend.

5. NCI SEER (data/raw/seer/)¶

Cancer incidence and U.S. mortality by site, sex, race, and age back to 1975, via the JSON endpoints SEER*Explorer itself uses. Archived: mortality_by_year.csv, mortality_by_age.csv, mortality_by_year_age.csv. Charted: cancer deaths by type (7 sites) and by sex (lung, colorectal, pancreas, liver). Monthly refresh because SEER re-releases annually.

Known upstream drift. The SEER snapshot dropped the combined Leukemia site in favor of narrow myeloid subtypes with no aggregate total, so the cancer-sex-leukemia chart matches zero rows and is hidden until the aggregate returns.

6-8. Live-only sources (GRASP, NSSP, NIS)¶

Queryable through pulse/health but not committed to data/:

• GRASP / FluView: hantavirus cases, ILINet ILI %, WHO/NREVSS clinical-lab flu positivity, and FluSurv-NET hospitalization rates (via CMU Delphi).
• NSSP: weekly % of ED visits for COVID/flu/RSV at nation/state/HHS/county (via CMU Delphi). County flu/COVID/RSV is reachable here with --geo-type county.
• NIS: NIS-Child (19-35 mo) and NIS-Teen (13-17 yr) vaccination microdata, streamed from fixed-width .dat files without touching disk.

9. EPHT: Environmental Public Health Tracking (epht)¶

CDC's environmental-health data API (ephtracking.cdc.gov), covering 867 measures across 27 content areas (air and water quality, heat, childhood lead, asthma, birth outcomes, and more) at national, state, and county level. The epht module wraps the discovery-then-fetch REST API:

uv run python -m epht content-areas            # 27 content areas
uv run python -m epht measures --search asthma # search the 867 measures
uv run python -m epht registry                 # the curated archive set (8 measures)
uv run python -m epht fetch pm25_annual_avg    # one measure, state-level

Status. Discovery (content areas, measure search, geographic types, stratification levels) is verified and working. The archival path (epht.download, state-level series under a 5 MB guard) is implemented against CDC's documented getCoreHolder contract but not yet verified end-to-end: the API rate-limits unauthenticated requests (HTTP 429), so a scheduled job needs a free EPHT_API_TOKEN from https://ephtracking.cdc.gov/apihelp. It is deliberately not wired into a GitHub Actions workflow yet, to avoid a scheduled run committing empty files before a token-backed verification pass confirms which curated measures return state-level data (some measures are county-only). The curated registry is a starting set, not a final one.

10. CDC PLACES (data/processed/places/ + Dolt)¶

Small-area estimates of 49 measures for every US county, place, census tract and ZCTA — all seven pages of CDC's PLACES data portal.

Two CDC products, mirrored as two families. PLACES proper is 40 measures modeled from BRFSS, in six categories: Health Outcomes (12 measures), Prevention (7), Disability (7), Health-Related Social Needs (7), Health Risk Behaviors (4), Health Status (3). Non-Medical Factor Measures is the seventh page: 9 measures derived from the 5-year American Community Survey, under categoryid SDOH.

The one source too large to commit. Together, ~7.9M rows / ~1.7 GB:

The tract export alone is past GitHub's 100 MB hard limit. So PLACES is mirrored into the public Dolt database fartbagxp/cdc-places — free at any size, versioned and diffable — and data/raw/places/ is gitignored. Only these derived slices are tracked:

The mirror is ours, not CDC's. DoltHub is a commercial service run by DoltHub Inc., with no affiliation to CDC. fartbagxp/cdc-places is a copy this project maintains; CDC does not publish it and is not responsible for it. The authoritative source is always data.cdc.gov — the Socrata IDs above. The database is named cdc-places for what it holds, not for who runs it.

How it's kept small¶

Population is constant per (location, year) — zero violations across all 229,298 PLACES county rows and all 28,287 Non-Medical Factors ones — and geolocation is constant per location, so both are location attributes rather than measurements.

The two families keep separate fact and location tables, sharing only measure and category. The forcing difference is geographic, not cosmetic: ACS 2017-2021 still uses Connecticut's retired county FIPS 09001—09015 while PLACES 2025 uses the new planning regions 09110—09190, so the two do not even cover the same county universe. A county map built from one family's FIPS will have gaps in Connecticut for the other. County names differ too (— Autauga County vs Autauga, on 3,131 of 3,133 shared counties), which is harmless when joining on FIPS. See PLACES.

Live queries¶

The DoltHub SQL API sends CORS headers and needs no key, so health-charts can query it from the browser for drill-down. Two limits shape every query: a 1000-row cap (reported as a RowLimit status, not an error) and a ~55 second deadline. A single location's measures comes back in 0.3s, but any join against the 6.6M-row PLACES fact table exceeds the deadline — labels have to be resolved from the 49-row measures.csv client-side. National-scale chart payloads stay as committed CSVs. See PLACES.

Supersedes places_county in cdc_open. The older data/processed/cdc_open/places_county.csv covered 8 of the 40 PLACES measures, crude only, from a server-filtered 12 MB download. county_crude.csv uses the identical column layout and reproduces all 23,648 of its rows with no value differences, so the migration is a path change for health-charts rather than a parsing change. places_county should be set enabled=False once health-charts has moved.

places_city is not PLACES. Despite the name, dxpw-cm5u in the cdc_open registry is 500 Cities 2019 GIS-format data — a different, retired product. PLACES place-level data is eav7-hnsx, via Dolt.

State, county and local¶

Every source above is federal. Two research catalogs survey what is available below the state line; nothing in either is downloaded, by design, since data/ is already ~352 MB. See State & Local Sources (endpoints verified against live data) and State Health Data Portals (all 50 states + DC).

Candidate sources: external cross-check¶

Cross-referenced against the Federal Data Terminations Tracker (Federation of American Scientists; 375 entries as of 2026-08-17) to find CDC resources the pipeline does not yet cover, and to flag which upstream programs have been cut. None of the following are in the repo today.

Recommended next ingests. EPHT (a new environmental-health domain with a clean REST API) and RSS (drops straight into cdc_open as a Socrata ID). Both are active and API-backed. The terminated programs (DAWN, VACS, PRAMS national) are worth noting in documentation as gaps that will not refill, not as ingest targets.

Counts and dates verified 2026-08-24 against the working tree: cdc_open registry (68 + 1 + 3), DQS (28/17 topics), WISQARS (5), GRASP (4), WONDER (49 XML / 9 fetch / 13 CSV), and health-charts config (54 defined, 50 visible). Latest committed data dates sampled from the archived CSVs.

PLACES (section 10) verified 2026-08-31 against both the live exports and the published database: all four geographies loaded into fartbagxp/cdc-places and queried back at 229,298 / 2,150,438 / 3,047,284 / 1,171,563 (6,598,583 total), with 3,145 / 29,923 / 83,522 / 32,520 locations. Age-adjusted estimates exist only at county and place level, as CDC documents. All four 2025 datasets were last republished by CDC in December 2025.

Non-Medical Factors verified 2026-09-03 the same way: all four geographies loaded and queried back at 28,287 / 268,389 / 751,509 / 291,024 (1,339,209 total) across 148,801 locations, exactly 9 measures per location at every level with no primary-key collisions. The shared measure table now holds 49 rows across 7 categories — the seven pages of CDC's portal, enumerated from the ArcGIS Experience config itself rather than from the page. The published database totals 7,937,792 fact rows. CDC last republished these four datasets in October/November 2023.