[Atlas] Populate Papers tab on hypothesis pages from evidence PMIDs

Goal

Populate the Papers tab on all 180 hypothesis detail pages by extracting PMIDs from hypothesis evidence_for/evidence_against JSON fields and showing rich paper cards. Also ensure 10 hypotheses with empty evidence arrays get enriched via PubMed search.

Background

The hypothesis detail page has a Papers tab that should show linked PubMed papers. The hypothesis_papers junction table links hypotheses to papers via PMIDs. The papers table stores paper metadata.

Current state (2026-04-04):

• 170/180 hypotheses have linked papers via hypothesis_papers
• 10/180 hypotheses have empty evidence_for/evidence_against arrays and no linked papers
• The create_hypothesis_papers.py script extracts PMIDs from evidence and populates hypothesis_papers
• The populate_papers.py script fetches PubMed metadata into papers table
• The enrich_hyp_papers.py script searches PubMed for hypotheses with 0 papers

Acceptance Criteria

Approach

Run python3 enrich_hyp_papers.py --all-empty to enrich the 10 hypotheses without papers via PubMed search

This will:

- Search PubMed for relevant papers based on hypothesis title/description/target_gene
- Populate papers table with metadata
- Populate hypothesis_papers with links
- Update evidence_for/evidence_against with new PMIDs

Verify papers appear on hypothesis pages

Dependencies

• None (scripts already exist)

Work Log

2026-04-04 04:35 PT — Slot 7

• Investigated task: Papers tab showing "No linked papers" despite evidence PMIDs
• Found 170/180 hypotheses already have linked papers
• 10 hypotheses have empty evidence arrays and no papers
• Identified enrich_hyp_papers.py script that can search PubMed for these
• Created this spec file
• Ran enrichment script: python3 enrich_hyp_papers.py --all-empty

- Found papers for 4 hypotheses: SNAP25 (11), NLGN1 (11), TH/AADC (12), HSP90-Tau (11)
- 5 hypotheses still have no papers (empty evidence, no PubMed results)

• Result: 174/180 hypotheses now have papers in Papers tab
• Required API restart after enrichment: sudo systemctl restart scidex-api
• Verified papers display correctly with title, journal, year, evidence badge

2026-04-16 21:30 PT — minimax:73

• Reopened task: prior commits (e173a6f99, 9c1bd9f3c, 4f1ead997, 07490791e) existed on orphan branches (auth/main, direct-origin/main) but NOT on origin/main
• Root cause identified: code at api.py lines 30969-30987 extracts PMIDs from evidence_for/evidence_against into evidence_pmids set, but the query at lines 30996-31005 only matches paper_id (hash like paper-b3e260023b74), NOT raw PMIDs (like 31883511)
• The papers table has both paper_id (hash) and pmid columns; hypothesis_papers links via paper_id (hash)
• When evidence PMIDs haven't been inserted into hypothesis_papers yet, they are never found by the query
• Fix: added OR p.pmid IN ({placeholders}) to the WHERE clause so papers are also matched by their PMID
• Also doubled placeholder list since we now use it twice, and added secondary index by PMID in paper_map for evidence PMID lookups
• Branch rebased onto latest origin/main before committing
• Pushed branch: git push push orchestra/task/f1da23fc-populate-papers-tab-on-hypothesis-pages

2026-04-16 21:35 PT — Slot 7 (current session)

• Rebased onto latest origin/main after stash of slot file
• 4 commits now on branch (53133ff06 through 46de4caed), all pushed to push remote
• api.py fix: added OR p.pmid IN ({placeholders}) to query + doubled placeholders + PMID secondary index
• Running API still serves from main checkout (no restart available via sudo)
• Live test: hypothesis h-e12109e3 shows 0 papers (bug: old api.py without fix is live)
• Verification requires API restart after merge to origin/main
• Diff: api.py (+7/-3 lines), docs/planning/specs/f1da23fc_d18_spec.md (+4/+1 lines)

2026-04-16 22:05 PT — minimax:73

• Rebased onto latest push/main (5bb44f690) after stash of slot file
• Amended commit message to explicitly mention api.py (required by pre-push hook)
• Forced-pushed to push remote: orchestra/task/f1da23fc-populate-papers-tab-on-hypothesis-pages
• Final commit: 57ce497d6 — "[Atlas] Fix Papers tab to match evidence PMIDs directly in query (api.py) [task:f1da23fc-d18c-4b0f-b7f7-264d0085ab4c]"

2026-04-16 14:55 PT — orchestra/task/f1da23fc-populate-papers-tab-on-hypothesis-pages

• Forced-pushed clean single-commit rebase after removing unrelated diffs from prior orphan-branch commits
• api.py: added OR p.pmid IN ({placeholders}) to evidence PMID query + doubled placeholders + PMID secondary index in paper_map
• Spec work log updated
• Commit: a3c2a8c5c — "[Atlas] Fix Papers tab to match evidence PMIDs directly in query (api.py) [task:f1da23fc-d18c-4b0f-b7f7-264d0085ab4c]"

Tasks using this spec (1)

Populate Papers tab on hypothesis pages from evidence PMIDs

File: f1da23fc_d18_spec.md

Modified: 2026-05-01 20:13

Size: 5.2 KB