[Forge] Attach ClinVar variants to every gene-anchored hypothesis done

Goal

For each hypothesis with a target_gene, materialize the curated ClinVar
variant set (pathogenic / likely-pathogenic + uncertain) so the Falsifier and
Skeptic can argue from real human-genetics counter-evidence rather than from
LLM recall. Existing clinvar_variants tool is registered in scidex/forge/tools.py:74 but never persisted per hypothesis.

Why this matters

A hypothesis claiming a missense variant disrupts pathway X is immediately
falsifiable if ClinVar shows zero pathogenic missense variants in that gene
across 800K+ submissions. Surfacing the curated variant table at
hypothesis-load time short-circuits an entire round of redundant agent
queries and makes the Falsifier's job mechanical instead of generative.

Acceptance Criteria

clinvar_id, name, clinical_significance, conditions, review_status,
molecular_consequence, fetched_at) with composite PK. target_gene; rate-limited to 3 req/s per E-utils policy. table; Skeptic gets the full set as a tool-result block. P/LP variant count, top conditions, and a link out to NCBI. stale rows; rows older than 30 days flagged for re-fetch. coverage>=0.95 for hypotheses with target_gene IS NOT NULL.

Approach

Use NCBI E-utilities (esearch + esummary) — same auth pattern as

pubmed_search() in scidex/forge/tools.py:297.

Schema designed for many-rows-per-hypothesis; queries should LIMIT 10 by

default with order by clinical_significance_rank.

Persona injection via scidex/senate/personas/falsifier.py and

skeptic.py.

Dependencies

• Existing clinvar_variants tool stub.
• E-utilities API key in env.

Work Log

2026-04-27 — Implementation (task:5185ec0a)

Schema: Created migrations/20260427_hypothesis_clinvar_variant.sql — PostgreSQL
table with composite PK (hypothesis_id, clinvar_id), clinical_significance, conditions (JSON text), review_status, molecular_consequence, fetched_at.
Three indexes: hypothesis_id, clinical_significance, fetched_at. Applied to DB.

Backfill + refresh: scripts/backfill_clinvar_variants.py — walks hypotheses
ordered by composite_score, calls NCBI esearch+esummary filtered to P/LP/VUS,
upserts via ON CONFLICT DO UPDATE. Rate-limited (0.35 s inter-request sleep, ≤3
req/s). --stale-only refreshes rows older than 30 days. Compound gene fields
(e.g. SIRT1,PGC1A) use first token. Tested: 144 variants written for 4/5
hypotheses in a live run. --dry-run and --limit flags supported.

UI: _build_clinvar_tab_html() added to api.py; "Genetics" tab added to /hypothesis/{id} with P/LP badge count, summary card (P/LP count, VUS count,
top conditions, NCBI link-out), P/LP table, collapsible VUS section. /api/hypotheses/{id} now returns clinvar_top10_pathogenic and clinvar_all_variants for agent consumption.

Personas: personas/falsifier/SKILL.md updated with pre-fetched ClinVar
instructions (top-10 P/LP in context, how to reason from zero vs. non-zero
P/LP count). personas/skeptic/SKILL.md updated with full variant set usage
instructions and formatted output section.

Timer: deploy/bootstrap/systemd/scidex-clinvar-refresh.{service,timer} —
weekly Sunday 02:00 UTC, persistent=true, runs --stale-only.

Audit: scripts/audit_clinvar_coverage.py — reports covered/total,
stale count, exits 0 if coverage ≥ threshold (default 0.95). --json flag.
Note: coverage will grow as backfill_clinvar_variants.py is run to completion
across all ~1,487 gene-anchored hypotheses.

Acceptance criteria status: