Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1.3 KiB
1.3 KiB
1.3 — Client disconnect propagation (RED)
Phase: 1 — Proxy core Stage: RED Depends on: 1.2
- When a client closes the connection mid-response, the gateway cancels the upstream request immediately
- The stub upstream observes its request context cancelled, not a request that runs to completion
- Cancellation happens within a small bounded delay of the client close, not at the route's read timeout
- The same holds for a non-streaming request abandoned before the upstream replies
- The disconnect is logged with a reason distinguishing it from an upstream error
- No goroutine or upstream connection is left alive after the disconnect — the test asserts this, not just the response
This is load-bearing. The reasoning upstream runs 2 replicas at --max-num-seqs=4,
which is 8 concurrent sequence slots cluster-wide. An orphaned generation holds one of
those 8 until it finishes on its own, which for a 32B model on a Volta GPU can be
minutes. A handful of abandoned browser tabs can starve the entire cluster.
Verify
go test ./internal/proxy/... -run TestClientDisconnectCancelsUpstream -race -v
# expected: passes — stub upstream reports context cancellation within 1s of the client
# closing mid-stream, and the post-test goroutine count returns to baseline