[Agora] Add 'validated' hypothesis lifecycle status — schema migration + promotion pipeline

Context

SciDEX hypotheses currently have 7 statuses: proposed, promoted, active, debated, archived, open, superseded. There is no validated status.

The hypothesis validation gate task (ca573a56, merged 2026-04-28) was intended
to promote high-scoring hypotheses to "validated" — but the schema doesn't
include that status. As a result, the platform's scientific lifecycle is
incomplete. A hypothesis can be proposed, debated, scored 0.96, and yet never
reach a "this is validated science" endpoint. This is the core value-delivery
gap on SciDEX.

Current state (2026-04-29):

• 1,974 hypotheses total
• 88 with composite_score ≥ 0.8 (potentially validation-eligible)
• 221 promoted, 156 active
• 0 validated (status doesn't exist)
• 9 hypothesis_predictions confirmed, 5 falsified (validation signal exists)
• 61 convergence_reports exist (validation evidence exists)

Goal

Add validated to the hypotheses status lifecycle in PostgreSQL

Define scientifically defensible promotion criteria for the validated status

Implement a promotion pipeline that promotes eligible hypotheses

Surface validated hypotheses prominently on the platform

What success looks like

AND has ≥2 debate sessions AND has ≥1 confirmed prediction AND evidence
chain density ≥ threshold)

What NOT to do

• Do NOT create the "validated" status with zero criteria — that would

invalidate scientific credibility

• Do NOT rely solely on composite_score ≥ 0.8 — score is model-generated;

validation should require evidence corroboration

• Do NOT change the scores of existing hypotheses — only change status
• Do NOT break the existing status flow (proposed → promoted → active/debated)

Agent guidance

Read api.py for hypothesis routes and status validation logic

Read the existing debate and scoring pipeline to understand what signals exist

Read hypothesis_predictions table — confirmed predictions are strong evidence

Read convergence_reports — these may already contain validation evidence

Write the migration, test on staging data, then apply

Post-migration, run the promotion pipeline and verify

Spec notes

• Created by quest task generator Cycle 4 (2026-04-29T03:05Z)
• Replaces the incomplete validation gate from ca573a56 which targeted

a non-existent status

• Priority 96: closest to completing SciDEX's scientific output lifecycle

Work Log

Created 2026-04-29

Spec created by ambitious quest task generator (Cycle 4). Root cause: ca573a56
shipped a validation gate but no validated status in hypotheses schema.
World model shows 88 composite_score ≥ 0.8 hypotheses and 9 confirmed
predictions — sufficient evidence for first validated cohort.

Work Log — 2026-04-29T04:18Z

Started implementation in Orchestra worktree task-0b9657cd-1826-4d1b-bc65-16fac2c0c242. Staleness check found
ca6dfe65a already added hypotheses.lifecycle='validated', validated_at, validation_notes, and /api/hypotheses/validated, but hypotheses.status still had no validated endpoint status and /api/hypotheses?status=validated returned zero rows. Live DB inspection
also found no row currently satisfies the new strict criteria composite_score >= 0.85 AND debate_count >= 2 AND >= 1 confirmed prediction
AND evidence_validation_score >= 0.7; the 9 confirmed predictions are all
attached to lower-scoring hyp_test_* rows. Plan: add the status constraint
and stricter future promotion path, expose legacy lifecycle-validated rows
through the status API/UI for continuity, and record the zero-strict-eligible
batch result rather than weakening scientific criteria.

Work Log — 2026-04-29T04:36Z

Implemented the code path for the formal hypotheses.status='validated'
endpoint without loosening the task's scientific criteria. Added migration
149 with a PostgreSQL CHECK constraint allowing only the existing statuses
plus validated. Updated scidex.agora.hypothesis_validation_gate so
future writes to status='validated' require:

• composite_score >= 0.85
• debate_count >= 2
• at least one hypothesis_predictions.status='confirmed'
• evidence_validation_score >= 0.70

The live strict batch found zero eligible rows, because the nine confirmed
predictions are currently attached to lower-scoring test hypotheses. The API
and /hypotheses UI now treat historical lifecycle='validated' rows as
validated for display/query continuity, while only the strict gate writes the
new status='validated' value going forward. Migration 149 was applied
successfully; GET /api/hypotheses?status=validated&limit=3 returned HTTP
200 with validated rows from the current live cohort.

File: quest_agora_hypothesis_validated_status.md

Modified: 2026-05-01 20:13

Size: 5.3 KB