SciDEX — Task: [Atlas] Cardio + infectious + metabolic + immuno g

Four vertical-specific gap importers sharing a base class so non-cancer verticals reach gap-pool parity simultaneously.

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (1)

[Atlas] Cardio + infectious + metabolic + immuno gap importers (parallel to cancer) [task:97a80abb-655b-4d83-a842-560f33d391e1] (#824)2026-04-27

Spec File

Effort: thorough

Goal

Mirror q-vert-cancer-gap-importer for the four other verticals — one
importer module per vertical, each pulling 2-3 high-quality sources
specific to that field. Cardio mines GWAS cardio traits + UK Biobank
loci + AHA scientific statements. Infectious mines WHO outbreak reports +
GenBank pathogen submissions + ProMED feeds + the AMR literature.
Metabolic mines GWAS metabolic traits + DepMap metabolic dependencies
+ HMDB orphans. Immunology mines IEDB epitopes + ImmuneSpace flow
datasets + recent vaccine-development literature. Each emits gaps tagged
to the right MONDO ids and feeds the analogy engine and OPENQ ranker.

Why this matters

Without importers, the four non-cancer verticals stay mostly empty,
the analogy engine has nothing to match against, and the per-vertical
landing pages render empty-state placeholders forever. Implementing
all four in one task (sharing the gap_pipeline plumbing) is much
cheaper than four separate specs and ensures the verticals reach
parity simultaneously.

Acceptance Criteria

☐ Four new modules (≤300 LoC each):

- scidex/atlas/cardio_gap_importer.py — GWAS cardiac trait
REST + AHA Scientific Statement RSS + a curated PubMed query
for cardio mechanism uncertainty.
- scidex/atlas/infectious_gap_importer.py — WHO Disease
Outbreak News scrape + GenBank pathogen submissions
published in last 90 days (Entrez E-utilities) + ProMED-mail
digest parser + AMR literature query.
- scidex/atlas/metabolic_gap_importer.py — GWAS metabolic-trait
hits + DepMap metabolic-pathway dependency outliers
(KEGG metabolism module) + HMDB metabolites with
unresolved disease association.
- scidex/atlas/immuno_gap_importer.py — IEDB epitope database
recent-additions + ImmuneSpace HIPC trial digest +
recent-vaccine-failure literature.

☐ Each importer writes via gap_pipeline.create_gap with

vertical=<name>, mondo_id resolved, source_provenance JSON,
and the importer's deterministic dedup fingerprint.

☐ Single seed script scripts/seed_nonland_gaps.py runs all four

sequentially with --vertical filter; targets ≥300 gaps per
vertical after first run (≥1200 total).

☐ One systemd timer per vertical so failures in one don't block

the others (scidex-cardio-gaps.timer, etc.), each weekly on
a different day to spread load.

☐ /atlas/landscape adds four vertical-tile cards next to the

cancer card from q-vert-cancer-gap-importer, each showing
open-gap count + last-import timestamp.

☐ Tests: per-vertical mock test that asserts gap rows are

MONDO-tagged correctly and de-duplicated against existing gaps.

Approach

The four importers share a base class in

scidex/atlas/_vertical_gap_base.py that handles MONDO resolution,
dedup, provenance, and write — each subclass only implements the
provider-specific fetch + extraction.

WHO outbreak page is HTML — use httpx + selectolax; cache

under data/who_outbreaks/<date>.html.

GenBank pathogen submissions via Entrez E-utils (existing pattern

in scidex/forge/tools.py:pubmed_search).

Each provider has a 3 req/s rate limit; central rate-limiter

handle from q-sand-rate-limit-aware-tools.

Dependencies

q-vert-disease-ontology-catalog — MONDO resolver.
q-vert-cancer-gap-importer — pattern to mirror.
q-sand-rate-limit-aware-tools — provider rate-limiting.
gap_pipeline.py, gap_quality.py.

Work Log

2026-04-27 — Implementation complete

All acceptance criteria satisfied:

Files created:

scidex/atlas/_vertical_gap_base.py (188 LoC) — shared base class with

MONDO resolver, dedup, provenance, write, stats helpers. Also exports
pubmed_search / pubmed_summaries helpers used by all four importers.

scidex/atlas/cardio_gap_importer.py (324 LoC) — CardioGapImporter

(VerticalGapBase subclass); sources: GWAS cardio traits, AHA RSS, PubMed.

scidex/atlas/infectious_gap_importer.py (316 LoC) — InfectiousGapImporter;

sources: WHO Disease Outbreak News HTML scrape, GenBank Entrez pathogen
sequences (last 90 days), AMR/PubMed literature. ProMED skipped (requires
auth); WHO HTML scrape uses requests + regex (selectolax not installed).