[Atlas] Score 30 open knowledge gaps with quality rubric done

Goal

Score open knowledge gaps with the landscape-gap quality rubric so SciDEX can prioritize the most actionable and scientifically useful gaps. Component scores should reflect specificity, evidence coverage, hypothesis density, debate depth, and actionability.

Acceptance Criteria

Approach

Dependencies

• quest-engine-ci - Generates this task when queue depth is low and unscored gaps exist.

Dependents

• Gap routing, missions, debate scheduling, and Exchange funding depend on quality-scored gaps.

Work Log

2026-04-28 01:43 UTC — Slot 76 (minimax) — iteration 2

• Task still current: live PostgreSQL has 2,038 open knowledge gaps with gap_quality_score IS NULL before this run; 1,085 open gaps already have in-range scores.
• Ran python3 score_30_gaps.py --limit 30 --task-id 2f2a61c5-6996-4244-8705-35d1b84416c9 to score 30 open gaps; all 30 rows updated with composite + component scores.
• Before: 2,038 open gaps unscored, 1,115 open gaps scored in range.
• After: 2,008 open gaps unscored, 1,145 open gaps scored in range; net 30 new gaps scored this iteration.
• Score range this run: composite scores in [0.635, 0.858]; all in [0,1]. No new low-quality gaps scored below 0.3, so low-quality triage remains 25 cumulative.
• Verification: 30 scored, 0 failed; remaining_unscored=2008, scored_in_range=1145, low_quality_under_review=25.

2026-04-28 01:41 UTC — Slot 76 (minimax)

• Task still current: live PostgreSQL has 2,038 open knowledge gaps with gap_quality_score IS NULL after this run; 1,115 open gaps already have in-range scores.
• Ran python3 score_30_gaps.py --limit 30 to score 30 open gaps; all 30 rows updated with composite + component scores.
• Before: 2,068 open gaps unscored, 1,085 open gaps scored in range.
• After: 2,038 open gaps unscored, 1,115 open gaps scored in range; net 30 new gaps scored this iteration.
• Score range this run: composite scores in [0.652, 0.825]; all in [0,1]. No new low-quality gaps scored below 0.3, so low-quality triage remains 25 cumulative.
• Verification: 30 scored, 0 failed; remaining_unscored=2038, scored_in_range=1115, low_quality_under_review=25.

2026-04-28 08:20 UTC — Slot 54 (codex)

• Task still current: live PostgreSQL has 2,168 open knowledge gaps with gap_quality_score IS NULL; 985 open gaps already have in-range scores.
• Prior iterations scored only part of the backlog; this iteration scored the next 100 open unscored gaps ordered by priority/importance using the existing five-component landscape-gap rubric.
• Structural note: 57 already-scored open gaps still have at least one missing component score, so a future hygiene task should backfill legacy component columns separately from this forward-scoring batch.
• Before: 2,168 open gaps unscored, 985 open gaps scored in range.
• After: 2,068 open gaps unscored, 1,085 open gaps scored in range; all 100 rows from this run have component scores populated.
• Score range this run: composite scores in [0.627, 0.883]; all in [0,1]. No new low-quality gaps scored below 0.3, so low-quality triage remains 25 cumulative.
• Verification: python3 -m py_compile score_30_gaps.py; python3 score_30_gaps.py --limit 1 --dry-run --task-id ad73537f-7d01-4ed4-99d0-acef69b7c7f1; PostgreSQL check over rows scored between 2026-04-28 01:18:00-07 and 2026-04-28 01:24:00-07 returned n=100, components_populated=100, rubric_match=100, remaining_unscored=2068.
• Small scorer hygiene improvement: composite rounding now uses decimal half-up quantization and the JSON verification output accepts --task-id/ORCHESTRA_TASK_ID instead of emitting an obsolete hardcoded task ID.

2026-04-28 00:43 UTC — Slot 47 (claude-auto)

• Task still current: 2,308 open knowledge gaps with gap_quality_score IS NULL before this run.
• Applied two concurrent scoring rounds (100 gaps each) with python3 score_30_gaps.py --limit 100.
• Before: 2,308 open gaps unscored, 845 open gaps scored.
• After: 2,168 open gaps unscored, 985 open gaps scored; net 140 new gaps scored this iteration.
• Score range this run: composite scores in [0.650, 0.823]; all in [0,1]. No gaps scored below 0.3 (low-quality triage total remains 25 cumulative).
• Verification: 100 scored, 0 failed (both runs); remaining_unscored=2168, scored_in_range=985.

2026-04-28 00:11 UTC — Slot 43 (claude-auto)

• Task still current: 2,408 open knowledge gaps with gap_quality_score IS NULL before this run.
• Applied scoring to 100 open unscored gaps with python3 score_30_gaps.py --limit 100.
• Before: 2,408 open gaps unscored, 745 open gaps scored.
• After: 2,308 open gaps unscored, 845 open gaps scored; all 100 rows have scores in [0,1].
• Score range this run: min=0.624, max=0.858. No gaps scored below 0.3 (low-quality triage total: 25 cumulative).
• Verification: all 100 wrote successfully; remaining_unscored=2308, scored_in_range=845.

2026-04-27 04:56 UTC — Slot 54 (codex)

• Task e48ce6aa remains current: live PostgreSQL has 2,527 open knowledge gaps with gap_quality_score IS NULL.
• Sibling work (35e9639c, b993d7b3, plus prior 30-gap batches) covered only part of the backlog; this task will score the next 100 open unscored gaps.
• Approach for this run: update the existing batch scorer to accept a configurable limit, align the composite with the task rubric (specificity, actionability, evidence each capped at 0.33), and mark gaps below 0.3 as quality_status='under_review' for closure/revision triage.
• Applied scoring to 100 open unscored gaps with python3 score_30_gaps.py --limit 100.
• Before: 2,527 open gaps unscored, 585 open gaps scored.
• After: 2,427 open gaps unscored, 685 open gaps scored; 100/100 recent rows have scores in [0,1] and match specificity_score0.33 + actionability0.33 + evidence_coverage*0.33.
• Low-quality triage: 2 gaps from this run scored below 0.3 and were marked quality_status='under_review' (gap-test-20260425-224949, gap-debate-20260426-011448-7c85f5dc).
• Verification: python3 -m py_compile score_30_gaps.py; PostgreSQL count/rubric check over rows scored at 2026-04-26 14:54:00-07 returned n=100, min_score=0.066, max_score=0.924, rubric_match=100.

2026-04-21 14:27 UTC — Slot 0 (minimax:73)

• Task started: needed to score 30 open gaps lacking gap_quality_score
• Found gap_quality.py scoring module exists at scidex/agora/gap_quality.py but its get_db() helper uses broken SQLite shim
• Wrote standalone score_30_gaps.py using direct psycopg connection
• Applied 5-component rubric: specificity (heuristic gene/pathway match), evidence_coverage (paper count via specific terms), hypothesis_density (debate sessions), debate_depth (quality_score of linked debates), actionability (keyword heuristics)
• 30 gaps scored successfully; all scores in [0,1]
• Before: 2858 unscored open gaps, 231 scored
• After: 2828 unscored open gaps, 261 scored
• Verification: acceptance criteria met
• Committed and pushed

2026-04-22 16:45 UTC — Slot (minimax:73)

• Task still necessary: 2558 unscored open gaps remain
• Fixed two bugs in score_30_gaps.py: (1) compute_evidence_coverage had unused total_papers = float(total) line and was missing total_papers param; (2) compute_hypothesis_density used LOWER(evidence_for) LIKE on a jsonb column which fails on PostgreSQL — removed the jsonb path, kept only analysis_id FK lookup
• Applied 5-dimension rubric: specificity (heuristic gene/pathway/cell-type match), evidence_coverage (paper count via specific term search), hypothesis_density (linked hypotheses + debate sessions), debate_depth (quality_score of linked debates), actionability (keyword heuristics)
• 30 gaps scored successfully; all composite scores in [0.205, 0.573], within [0,1] range
• Before: 2558 unscored open gaps, 531 scored
• After: 2528 unscored open gaps, 561 scored
• Verification: acceptance criteria met
• Committed and pushed

2026-04-21 21:36 UTC — Slot 73 (minimax:73)

• Fix for merge gate rejection: hardcoded PostgreSQL password in _PG_DSN
• Replaced import psycopg + _PG_DSN = "dbname=scidex user=scidex_app password=scidex_local_dev host=localhost" with from scidex.core.db_connect import get_connection
• Updated _pg_connect() to call get_connection() which uses SCIDEX_PG_DSN environment variable with safe default
• Verified: script runs successfully, 30 gaps scored with all scores in [0,1]
• Force-pushed fix to replace rejected branch