[Senate] Auto-migration generation from approved schema changes done coding:8 safety:9

← Schema Governance
Generate and apply DB migrations from approved schema proposals, dry-run, test on copy, rollback plan

Git Commits (17)

Squash merge: orchestra/task/80ffb77b-quest-engine-generate-tasks-from-quests (117 commits) (#179)2026-04-26
Squash merge: orchestra/task/80ffb77b-quest-engine-generate-tasks-from-quests (116 commits) (#177)2026-04-26
Squash merge: orchestra/task/80ffb77b-quest-engine-generate-tasks-from-quests (80 commits) (#143)2026-04-26
Squash merge: orchestra/task/sen-sg-0-auto-migration-generation-from-approved (4 commits) (#90)2026-04-26
[Senate] Add schema proposal create/vote API routes; emit approval event [task:sen-sg-02-PROP]2026-04-25
[Atlas] Processing step lineage: add record_processing_step, reproducibility check, provenance graph enrichment, and /processing-history API (api.py) [task:sen-sg-06-PROC] (#43)2026-04-25
Squash merge: orchestra/task/sen-sg-0-domain-scope-enforcement-reject-out-of-s (2 commits)2026-04-25
[Senate] Update spec work log with integration work [task:sen-sg-03-VALD]2026-04-25
[Senate] Wire JSON Schema validation into register_artifact() with strict/warn/skip modes [task:sen-sg-03-VALD]2026-04-25
[Senate] Fix schema registry for PostgreSQL; add validation and compliance API [task:sen-sg-03-VALD]2026-04-25
[Atlas/Senate/Agora] Spec: notebook + artifact versioning extensions2026-04-24
Squash merge: orchestra/task/sen-sg-0-schema-registry-track-schemas-per-artifa (1 commits)2026-04-18
Squash merge: orchestra/task/47b17cbf-sen-sg-01-sreg-schema-registry-track-art (1 commits)2026-04-16
[Senate] Add schema registry API: GET /api/schemas and /api/schemas/{type} in api.py [task:sen-sg-01-SREG]2026-04-16
[Senate] Schema registry: migration, seeding, and /senate/schemas UI [task:47b17cbf-a8ac-419e-9368-7a2669da25a8]2026-04-06
[Senate] Holistic prioritization run 2: quest fixes + 3 new CI tasks [task:b4c60959-0fe9-4cba-8893-c88013e85104]2026-04-06
[Senate] Holistic prioritization: 6 tasks created for uncovered P88-P95 quests [task:b4c60959-0fe9-4cba-8893-c88013e85104]2026-04-06
Spec File

Goal

When a schema proposal is approved, automatically generate and apply the database migration.
For metadata-level changes (JSON fields), generate a backfill script. For column-level
changes, generate ALTER TABLE migrations compatible with the migration runner.

Acceptance Criteria

generate_migration(proposal_id) function that produces migration SQL
☑ Metadata-level changes: PostgreSQL jsonb_set(...) backfill SQL generated for metadata changes
☑ Column-level changes: ALTER TABLE plans generated with defaults and optional indexes
☑ New type registration: validate type name, add to ARTIFACT_TYPES constant with rollback-safe restoration on failure
☑ Dry-run mode: show what would change without applying
☑ Test on copy: validate against a temporary PostgreSQL copy table before applying
☑ Migration registered in migration_history table
☑ Schema registry updated to new version after successful migration
☑ Rollback plan generated alongside each migration

Dependencies

  • sen-sg-02-PROP — Approved proposals trigger this
  • sen-sg-03-VALD — Post-migration validation confirms integrity

Dependents

  • None (leaf task in migration chain)

Work Log

2026-04-25 22:11 PDT — Codex

  • Performed staleness review against current origin/main and live PostgreSQL schema.
  • Confirmed task is still relevant: scidex/senate/schema_governance.py finalized approved schema proposals with a log line only and did not define the migration helpers described in docs/schema_governance_guide.md.
  • Confirmed live migration_history only tracked (version, name, applied_at, execution_time_ms, status) and was missing proposal/rollback/schema-version metadata required by this task.
  • Planned a PostgreSQL-native implementation using jsonb_set and temporary-table copy validation instead of the original SQLite-style json_set / in-memory-copy wording in the spec.

2026-04-25 22:43 PDT — Codex

  • Verified the in-progress migration module against the live PostgreSQL schema and confirmed the task was still incomplete on main.
  • Ran targeted unit coverage for tests/test_schema_migration.py and reviewed the generated SQL paths for metadata, column, and add_type proposals.
  • Identified two production-hardening gaps to fix before shipping: rollback on copy-test failure and safe restoration of artifact_registry.py if an add_type migration aborts mid-transaction.

2026-04-25 23:01 PDT — Codex

  • Added failure-path hardening in scidex/senate/schema_migration.py: copy-test failures now roll back the open transaction, and add_type restores artifact_registry.py if the DB migration aborts mid-flight.
  • Added targeted regression coverage for dry-run, copy-test rollback, rollback-safe artifact-type registration, SQL-type validation, and Senate approval auto-application (8 passed).
  • Verified the real preview path against live proposal senate-prop-b54f3d4b9d61, which generated an ALTER TABLE artifacts ADD COLUMN IF NOT EXISTS effect_size DOUBLE PRECISION; plan for the approved schema shape.

2026-04-25 23:18 PDT — Codex

  • Re-ran python3 -m py_compile for scidex/senate/schema_migration.py, scidex/senate/schema_governance.py, and tests/test_schema_migration.py.
  • Re-ran pytest -q tests/test_schema_migration.py and confirmed 8 passed in 0.14s.
  • Re-checked the live PostgreSQL copy-validation path for proposal senate-prop-b54f3d4b9d61; preview SQL and temporary-table validation both succeeded against current data.

2026-04-26 — Claude Sonnet 4.6

  • Resumed task after finding all code uncommitted; rebased branch onto origin/main, resolved merge conflicts, re-ran 8 tests (0.39s pass), then synced to prior Codex commits already on the remote branch.
  • Final verification: pytest -q tests/test_schema_migration.py8 passed in 0.64s. All acceptance criteria confirmed complete.

2026-04-26 09:22 UTC — MiniMax Agent

  • Post-rebase verification: tests still pass (8/8), syntax checks clean.
  • Confirmed clean diff: 4 files only (spec, schema_governance.py, schema_migration.py, test file).
  • Restashed .orchestra-slot.json and .orchestra/audit/ (orchestration artifacts, not task code).
  • Pushed to origin/HEAD.

2026-04-26 — Claude Sonnet 4.6 (gate-fix retry)

  • Three prior merge attempts rejected with "judge returned an unparseable decision / You've hit your org's monthly usage limit" — all are judge_chain_error (judge LLM quota exhaustion), not code failures.
  • Re-verified: python3 -m py_compile clean on all 3 source files; pytest -q tests/test_schema_migration.py → 8 passed in 0.13s.
  • Branch is still 3 commits ahead of origin/main, exactly on the intended diff (4 files, 1075 net insertions).
  • All 9 acceptance criteria remain checked. No code changes needed; pushing again as directed by task instructions for judge_chain_error retries.

Payload JSON
{
  "requirements": {
    "coding": 8,
    "safety": 9
  },
  "completion_shas": [
    "5be56b1fc"
  ],
  "completion_shas_checked_at": ""
}

Sibling Tasks in Quest (Schema Governance) ↗

Task Dependencies

↓ Referenced by (downstream)