[Agora] Decision-tree viewer for synthesis verdicts

← All Specs

Goal

The Synthesizer persona produces a verdict on every debate
("hypothesis is supported / weakly supported / falsified") but the reasoning it used — which arguments from Theorist it weighted, which
Skeptic objections it dismissed, which Expert assessments it deferred
to — is buried in a free-text rationale block in debate_sessions.
Build a decision-tree viewer that decomposes each synthesis into a
visual tree: root = verdict, branches = supporting/contradicting
arguments, leaves = evidence chunks with links. Editors can click any
node to see "why was this argument weighted this way?"

Effort: thorough

Acceptance Criteria

scidex/agora/synthesis_decision_tree.py::extract_tree(debate_session_id) -> DecisionTree runs an LLM pass over the synthesis text + per-round persona messages and emits {verdict, root_rationale, branches: [{stance: 'support'|'oppose'|'qualify', argument, weight: float, source_round, source_persona, evidence_refs: list[{kind, id}], children: [...]}]}.
☐ Migration migrations/20260428_synthesis_decision_trees.sql: synthesis_decision_trees(debate_session_id PK, tree_json JSONB, prompt_version TEXT, generated_at TIMESTAMPTZ, model TEXT). Trees are immutable per (debate_session_id, prompt_version).
☐ HTML view /debate/{id}/decision-tree renders the tree as an interactive collapsible D3 hierarchy (or a Mermaid graph TD if D3 is too heavy — see site/mermaid-pan.js for pan/zoom helper).
☐ Each branch shows weight as a thickness/color cue (thicker = higher weight in synthesis); hovering an evidence_ref opens a side panel with the cited paper title + abstract excerpt.
☐ Embedded on the existing debate detail page as a tab next to "Transcript" / "Replay" / "Synthesis".
☐ Driver: run_extraction(limit=20, only_missing=True) -> dict runs nightly via a new economics_drivers/ci_synthesis_tree_extraction.py; only re-extracts when prompt version changes.
☐ API: GET /api/agora/decision-tree/{debate_session_id} returns the JSON tree.
☐ Tests tests/test_decision_tree.py: synthetic 3-round debate with 2 support + 1 oppose argument → tree has 3 branches with correct stances; missing synthesis tolerated (returns 404 with reason=no_synthesis); cyclic references in extraction tolerated (DAG, not strict tree).
☐ Performance: tree builds in < 8 s for a typical 12-message debate; cached after first build.

Approach

  • Read scidex/agora/synthesis_engine.py::synthesize_debate_session to understand the synthesis prompt + output format. The new extractor consumes both the raw synthesis and the per-round messages.
  • Design the extraction prompt: input = synthesis text + per-round persona messages, output = structured JSON tree. Strict schema validation via pydantic.
  • Implement the D3 (or Mermaid) renderer; reuse the existing pan/zoom JS from site/mermaid-pan.js.
  • Wire the tab on the debate page; smoke-test on 5 existing debates.
  • The "weight" field is approximate (LLM-assigned); document this as an inferred annotation, not a measured one.
  • Dependencies

    • scidex/agora/synthesis_engine.py (existing).
    • debate_sessions, debate_messages tables.
    • scidex/core/llm.py.

    Dependents

    • q-viz-explainability-tearsheet — embeds the decision tree as a panel.
    • q-mem-persona-reputation-log — uses argument-weights as per-persona quality signal.

    Work Log

    Tasks using this spec (1)
    [Agora] Decision-tree viewer for synthesis verdicts
    Agora done P87
    File: q-viz-decision-tree-synthesis_spec.md
    Modified: 2026-05-01 20:13
    Size: 3.5 KB