Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1.6 KiB
1.6 KiB
1.4 — Per-route timeouts (GREEN)
Phase: 1 — Proxy core Stage: GREEN Depends on: 0.2, 1.1
- Connect, read and write timeouts are taken per route from configuration, never from a global default
- Chat routes use connect
10s, read1h, write1h - Embeddings and rerank routes use connect
10s, read10m, write10m - An upstream that never accepts a connection fails at the configured connect timeout, not later
- An upstream that accepts then stalls fails at the configured read timeout with a
5xxand a logged reason - A stream still emitting tokens is never cut by the read timeout — the timeout applies to inactivity, not total duration
- No code path shortens a configured proxy timeout to enforce an application-level cap
These are Kong's current values and they are deliberate. The 1-hour read timeout exists because a 32B model on a Volta GPU routinely exceeds 60 seconds per request. Any shorter application-level cap must be enforced by the gateway's own logic — a budget, a slot limit, an explicit max-generation-time — and never by shortening the proxy timeout, or long legitimate generations truncate mid-stream and callers see corrupted output rather than an error.
Verify
go test ./internal/proxy/... -run 'TestConnectTimeout|TestReadTimeout|TestLongStreamNotTruncated' -v
# expected: passes — stalled upstream errors at the configured read timeout, a stub
# emitting one event every 200ms for longer than the timeout window is not cut off,
# and a blackholed address fails at ~10s