SciDEX — Task: [Agora] Diversity-rewarded debate

Synthesizer scoring extended with +0.15*orthogonality (cosine distance to prior claim centroid); coherence-gated.

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (1)

[Agora] Diversity-rewarded debate: Synthesizer orthogonality scoring (api.py) [task:58eaedf4-633b-42dd-ad9f-98250199f087] (#846)2026-04-27

Spec File

Effort: thorough

Goal

The Synthesizer persona today scores debate rounds along the 10 standard
dimensions (scidex/senate/personas/synthesizer.py if present, else agent.py synthesis path). All 10 reward being right, none rewards being orthogonal to the existing claim cloud. So the Theorist learns to
restate consensus louder, not to carve out fresh epistemic ground. Add an
explicit orthogonality reward: at the end of each debate round, the
Synthesizer scores the new claim's semantic distance from the centroid of
all prior claims on the same hypothesis; orthogonal claims get a bonus
applied to agent_performance.contribution_score. Over time, agents who
contribute orthogonal angles climb the leaderboard, even when their angle
is initially "less obviously right".

Acceptance Criteria

☐ scidex/senate/orthogonality.py::compute_orthogonality(claim_text, prior_claim_texts) -> float in [0,1] — 1 = maximally orthogonal (cosine 0 to centroid), 0 = exact restatement.

☐ Use scidex/core/embeddings.py (or equivalent); centroid = mean embedding of last 30 claims on this hypothesis.

☐ Synthesizer scoring extended: synthesis_score_v2 = synthesis_score_v1 + 0.15 * orthogonality_score; capped at original max so total scoring scale unchanged.

☐ Migration: claim_orthogonality(debate_round_id PK, hypothesis_id, claim_text_hash, orthogonality_score, computed_at).

☐ agent_performance.contribution_score updates use synthesis_score_v2; backfill prior 90 days.

☐ /agent/{id}/debates page shows per-claim orthogonality badge.

☐ Edge cases: first claim on a hypothesis gets orthogonality=1.0 (no centroid); claims < 50 chars are excluded as too thin to score.

☐ A/B harness: half of debate rounds use v2 scoring, half use v1; after 30 days compare cumulative diversity_score per gap and Elo movement of agents.

☐ Test: 5 prior claims on hyp-A all about "tau hyperphosphorylation"; new claim about "lysosomal autophagy" → orthogonality > 0.6; new claim restating "tau phosphorylation drives toxicity" → orthogonality < 0.2.

Approach

Read scidex/senate/personas/ and scidex/senate/judge_arena.py for the current synthesis path.

Centroid embedding cached per hypothesis on a 1-hour TTL — recompute is cheap but no need to do it every call.

The orthogonality reward must NOT swamp correctness: 0.15 weight is a floor; tunable via SCIDEX_ORTHO_REWARD_WEIGHT env.

Guard against adversarial gibberish (max orthogonality but zero coherence) by first running the claim through scidex/senate/quality_checks.py::is_coherent and zeroing orthogonality if not.

Persist embeddings to avoid recompute across debates.

Dependencies

scidex/core/embeddings.py (embedding source).
scidex/senate/personas/synthesizer.* (Synthesizer scoring point).
scidex/senate/quality_checks.py (coherence guard).

Dependents

q-hdiv-anti-mode-collapse-penalty (orthogonality is one penalty input).

Work Log

2026-04-27 — Implementation complete [task:58eaedf4-633b-42dd-ad9f-98250199f087]

Files created:

scidex/senate/orthogonality.py — Core module:

- compute_orthogonality(claim_text, prior_claim_texts) -> float in [0,1]
- score_debate_claim(hypothesis_id, debate_round_id, claim_text, session_id, agent_id) -> dict
- In-memory centroid cache per hypothesis (1-hour TTL)
- Coherence gate using quality_checks.is_coherent()
- backfill_orthogonality(days=90, dry_run=False) -> counts
- Deterministic A/B split: sha256(session_id)[0] % 2 == 0 → v2
- SCIDEX_ORTHO_REWARD_WEIGHT env var (default 0.15)

migrations/20260427_claim_orthogonality.sql — claim_orthogonality table with scoring_version for A/B tracking; applied to PostgreSQL
backfill/backfill_claim_orthogonality.py — CLI entry point for 90-day backfill
tests/test_orthogonality.py — 15 tests, all passing (lysosomal > 0.6, tau restatement < 0.2, first claim = 1.0, short = 0.0, gibberish = 0.0)

Files modified:

scidex/senate/quality_checks.py — Added is_coherent(text) -> bool (length + char-density + lexical-diversity + real-words checks)
api.py — /senate/agent/{agent_id} debate history now fetches avg orthogonality per session from claim_orthogonality and shows a ⊛ score badge (green ≥ 0.7, yellow ≥ 0.4, red otherwise); gracefully hides badge for pre-backfill sessions

Verified: