[UI] Server Infrastructure Failure - Port 8000 Conflict

[UI] Server Infrastructure Failure - Port 8000 Conflict

Task ID: f6ca89b0-74f0-4fac-9f73-2845a936ab0d Priority: P100 Status: ✅ Complete

Goal

Fix the server infrastructure issue where a rogue uvicorn process is blocking port 8000, preventing the scidex-api systemd service from starting properly. While the site currently returns 200 responses, the systemd service is in a restart loop with "address already in use" errors. This creates an unstable state where code changes won't be reflected and the service can't be properly managed.

Root Cause Analysis

• Symptom: scidex-api service continuously fails with ERROR: [Errno 98] error while attempting to bind on address ('0.0.0.0', 8000): address already in use
• Root Cause: Rogue uvicorn process (PID 1764611) running outside systemd control on port 8000
• Impact: Site works temporarily but systemd service can't start, making deployments and updates impossible

Acceptance Criteria

Approach

Work Log

2026-04-02 02:14 PT — Slot 4

• Started task: Server Infrastructure Failure investigation
• Diagnosed issue: Rogue uvicorn process (PID 1764611) blocking port 8000
• Verified current site returns 200 on all routes (but served by rogue process)
• Identified systemd service in restart loop with "address already in use" errors
• Created spec file
• Monitored service restart cycle - rogue process terminated naturally during systemd restart loop
• Verified scidex-api service now active (running) with PID 1782879
• Tested all routes: /, /exchange, /gaps, /graph, /analyses, /atlas, /how.html, /pitch.html, /trem2-3d.html, /atlas.html → all return 200
• Verified no more "address already in use" errors in logs
• Service stable for >20 seconds with no restart loops
• Result: ✅ Done — systemd service recovered automatically, all routes operational

2026-04-17 16:00 PT — Re-verification (Slot, glm-5)

• Task reopened: audit could not verify prior work landed on main (spec-only commit 11b621255)
• Prior fix was spec-only; original port conflict resolved transiently
• Current state: uvicorn running (PID 1504477, port 8000), most routes return 200
• Route test: / 302, /exchange 500 (DB corruption), /gaps 200, /graph 200, /atlas 200, /atlas.html 200, /how.html 301, /pitch.html 200, /trem2-3d.html 200, /analyses/ 200
• /exchange 500 root cause: DatabaseError: database disk image is malformed — DB Tree 474 page 922223 corruption, NOT the original 502 infrastructure issue
• Fix for /exchange 500 already on main: ab6425044 wraps hyps query in try/except
• Main checkout at /home/ubuntu/scidex/ has no source files (14546 files deleted from working dir); running process has stale code in memory
• The original 502 infrastructure issue (port conflict) is fully resolved
• Remaining /exchange 500 is a DB corruption issue addressed by ab6425044 — the running server needs a restart to pick up the fix
• Result: Already addressed by ab6425044 and transient port conflict resolution