34 lines
1.8 KiB
Markdown
34 lines
1.8 KiB
Markdown
|
# 5.2 — Gateway-specific metrics Kong could not provide (GREEN)
|
||
|
|
|
||
|
|
Phase: 5 — Observability
|
||
|
|
Stage: GREEN
|
||
|
|
Depends on: [5.1](5.1-prometheus-parity.md)
|
||
|
|
|
||
|
|
These are the signals that justify replacing Kong. Without them the concurrency cap
|
||
|
|
and budget layers are unfalsifiable — you cannot tell a saturated GPU from a broken
|
||
|
|
gateway.
|
||
|
|
|
||
|
|
- [ ] In-flight request count is observable per upstream, and returns to zero when traffic stops
|
||
|
|
- [ ] Queue depth for the `reasoning` concurrency queue is observable
|
||
|
|
- [ ] Occupancy of the `reasoning` slot cap is observable — how many of the configured slots are held right now
|
||
|
|
- [ ] The configured slot cap itself is observable, so occupancy can be read as a ratio without hardcoding the limit in a dashboard
|
||
|
|
- [ ] Rejections are counted and broken down by reason: queue full, budget exhausted, body too large, unknown model, auth failure
|
||
|
|
- [ ] Rejection reason label values match the stable reason identifiers used in the problem+json `type` field, so metrics and logs join cleanly
|
||
|
|
- [ ] Time spent waiting in the queue is observable as a distribution, separately from upstream latency
|
||
|
|
- [ ] In-flight and occupancy gauges are correct after a client disconnects mid-stream — no permanent drift upward
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Hold 3 long requests open against a stub reasoning upstream, then scrape.
|
||
|
|
curl -s localhost:8080/metrics | grep -E 'inflight|queue_depth|slots'
|
||
|
|
# expected: in-flight for the reasoning upstream reads 3, queue depth reads 0, slot cap is exported
|
||
|
|
|
||
|
|
# Kill the clients mid-stream, wait, scrape again.
|
||
|
|
curl -s localhost:8080/metrics | grep -E 'inflight'
|
||
|
|
# expected: back to 0 — disconnects released their slots
|
||
|
|
|
||
|
|
curl -s localhost:8080/metrics | grep 'reason='
|
||
|
|
# expected: rejection counters split by reason, matching the problem+json type identifiers
|
||
|
|
```
|