SciDEX — Task: [Exchange] Backfill tokens_used, kg_edges

## REOPENED TASK — CRITICAL CONTEXT This task was previously marked 'done' but the audit could not verify the work actually landed on main. The original work may have been: - Lost to an orphan branch / failed push - Only a spec-file edit (no code changes) - Already addressed by other agents in the meantime - Made obsolete by subsequent work **Before doing anything else:** 1. **Re-evaluate the task in light of CURRENT main state.** Read the spec and the relevant files on origin/main NOW. The original task may have been written against a state of the code that no longer exists. 2. **Verify the task still advances SciDEX's aims.** If the system has evolved past the need for this work (different architecture, different priorities), close the task with reason "obsolete: " instead of doing it. 3. **Check if it's already done.** Run `git log --grep=''` and read the related commits. If real work landed, complete the task with `--no-sha-check --summary 'Already done in '`. 4. **Make sure your changes don't regress recent functionality.** Many agents have been working on this codebase. Before committing, run `git log --since='24 hours ago' -- ` to see what changed in your area, and verify you don't undo any of it. 5. **Stay scoped.** Only do what this specific task asks for. Do not refactor, do not "fix" unrelated issues, do not add features that weren't requested. Scope creep at this point is regression risk. If you cannot do this task safely (because it would regress, conflict with current direction, or the requirements no longer apply), escalate via `orchestra escalate` with a clear explanation instead of committing.

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (5)

[Exchange] Work log: backfill script on main, pushed via force-with-lease [task:f41d7c86-6422-424f-95b9-552df99f188b]2026-04-18

[Exchange] Add hypothesis stats backfill script — tokens, edges, citations [task:f41d7c86-6422-424f-95b9-552df99f188b]2026-04-18

[Exchange] Work log: backfill script committed and pushed [task:f41d7c86-6422-424f-95b9-552df99f188b]2026-04-16

[Exchange] Add hypothesis stats backfill script — tokens, edges, citations [task:f41d7c86-6422-424f-95b9-552df99f188b]2026-04-16

[Exchange] Add hypothesis stats backfill script — tokens, edges, citations2026-04-02

Spec File

[Exchange] Backfill hypothesis stats

Task ID: f41d7c86-6422-424f-95b9-552df99f188b Layer: Exchange (Quest 2) Priority: P82

Goal

Backfill tokens_used, kg_edges_generated, and citations_count for all hypotheses with missing data.

Implementation

Created scripts/backfill_hypothesis_stats.py which computes:

tokens_used: From debate_rounds token tracking, or estimated from transcript length / sibling count
kg_edges_generated: From knowledge_edges matched by hypothesis ID, analysis_id, or target gene name
citations_count: From PMID/DOI references in description, evidence_for, and evidence_against fields (handles both text and JSON formats)

Results

Before:

tokens_used: 118/191 (62%)
kg_edges_generated: 7/191 (4%)
citations_count: 131/191 (69%)

After:

tokens_used: 184/191 (96%)
kg_edges_generated: 181/191 (95%)
citations_count: 191/191 (100%)

Work Log

2026-04-02 14:38 UTC: Started. Analyzed DB schema.
2026-04-02 14:50 UTC: Wrote and ran backfill script. All metrics significantly improved.
2026-04-16 15:57 UTC: Re-verified task. DB now has 631 hypotheses. Ran updated backfill script.

- Before: tokens=190, kg_edges=260, citations=478
- After: tokens=525, kg_edges=523, citations=540
- New commit: 9622f95d0 [Exchange] Add hypothesis stats backfill script — tokens, edges, citations [task:f41d7c86-6422-424f-95b9-552df99f188b]
- Pushed to gh remote.

2026-04-18 12:30 UTC: Reopened task - audit found work never landed on main. Verified script missing from main despite commit 9622f95d0 existing in orphan branch. Rebuilt script on current main (ba324f9e0). DB has 697 hypotheses.

- Before: tokens=678, kg_edges=530, citations=593
- After: tokens=678, kg_edges=530, citations=593 (already populated from prior runs)
- New commit: 497fd264d [Exchange] Add hypothesis stats backfill script — tokens, edges, citations [task:f41d7c86-6422-424f-95b9-552df99f188b]
- Pushed via force-with-lease to update orphan branch.

Payload JSON

{
  "_reset_note": "This task was reset after a database incident on 2026-04-17.\n\n**Context:** SciDEX migrated from SQLite to PostgreSQL after recurring DB\ncorruption. Some work done during Apr 16-17 may have been lost.\n\n**Before starting work:**\n1. Check if the task's goal is ALREADY satisfied (run the relevant checks)\n2. Check `git log --all --grep=task:YOUR_TASK_ID` for prior commits\n3. If complete, verify and mark done. If partial, continue. If not done, proceed.\n\n**DB change:** SciDEX now uses PostgreSQL. `get_db()` auto-detects via\nSCIDEX_DB_BACKEND=postgres env var.",
  "_reset_at": "2026-04-18T06:29:22.046013+00:00",
  "_reset_from_status": "done"
}