Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
34 lines
1.8 KiB
Markdown
34 lines
1.8 KiB
Markdown
# 5.2 — Gateway-specific metrics Kong could not provide (GREEN)
|
|
|
|
Phase: 5 — Observability
|
|
Stage: GREEN
|
|
Depends on: [5.1](5.1-prometheus-parity.md)
|
|
|
|
These are the signals that justify replacing Kong. Without them the concurrency cap
|
|
and budget layers are unfalsifiable — you cannot tell a saturated GPU from a broken
|
|
gateway.
|
|
|
|
- [ ] In-flight request count is observable per upstream, and returns to zero when traffic stops
|
|
- [ ] Queue depth for the `reasoning` concurrency queue is observable
|
|
- [ ] Occupancy of the `reasoning` slot cap is observable — how many of the configured slots are held right now
|
|
- [ ] The configured slot cap itself is observable, so occupancy can be read as a ratio without hardcoding the limit in a dashboard
|
|
- [ ] Rejections are counted and broken down by reason: queue full, budget exhausted, body too large, unknown model, auth failure
|
|
- [ ] Rejection reason label values match the stable reason identifiers used in the problem+json `type` field, so metrics and logs join cleanly
|
|
- [ ] Time spent waiting in the queue is observable as a distribution, separately from upstream latency
|
|
- [ ] In-flight and occupancy gauges are correct after a client disconnects mid-stream — no permanent drift upward
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
# Hold 3 long requests open against a stub reasoning upstream, then scrape.
|
|
curl -s localhost:8080/metrics | grep -E 'inflight|queue_depth|slots'
|
|
# expected: in-flight for the reasoning upstream reads 3, queue depth reads 0, slot cap is exported
|
|
|
|
# Kill the clients mid-stream, wait, scrape again.
|
|
curl -s localhost:8080/metrics | grep -E 'inflight'
|
|
# expected: back to 0 — disconnects released their slots
|
|
|
|
curl -s localhost:8080/metrics | grep 'reason='
|
|
# expected: rejection counters split by reason, matching the problem+json type identifiers
|
|
```
|