Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1.8 KiB
1.8 KiB
5.2 — Gateway-specific metrics Kong could not provide (GREEN)
Phase: 5 — Observability Stage: GREEN Depends on: 5.1
These are the signals that justify replacing Kong. Without them the concurrency cap and budget layers are unfalsifiable — you cannot tell a saturated GPU from a broken gateway.
- In-flight request count is observable per upstream, and returns to zero when traffic stops
- Queue depth for the
reasoningconcurrency queue is observable - Occupancy of the
reasoningslot cap is observable — how many of the configured slots are held right now - The configured slot cap itself is observable, so occupancy can be read as a ratio without hardcoding the limit in a dashboard
- Rejections are counted and broken down by reason: queue full, budget exhausted, body too large, unknown model, auth failure
- Rejection reason label values match the stable reason identifiers used in the problem+json
typefield, so metrics and logs join cleanly - Time spent waiting in the queue is observable as a distribution, separately from upstream latency
- In-flight and occupancy gauges are correct after a client disconnects mid-stream — no permanent drift upward
Verify
# Hold 3 long requests open against a stub reasoning upstream, then scrape.
curl -s localhost:8080/metrics | grep -E 'inflight|queue_depth|slots'
# expected: in-flight for the reasoning upstream reads 3, queue depth reads 0, slot cap is exported
# Kill the clients mid-stream, wait, scrape again.
curl -s localhost:8080/metrics | grep -E 'inflight'
# expected: back to 0 — disconnects released their slots
curl -s localhost:8080/metrics | grep 'reason='
# expected: rejection counters split by reason, matching the problem+json type identifiers