34 lines
1.8 KiB
Markdown
34 lines
1.8 KiB
Markdown
# 5.1 — Prometheus parity with the retiring Kong plugin (GREEN)
|
|
|
|
Phase: 5 — Observability
|
|
Stage: RED
|
|
|
|
Kong runs a cluster-wide `prometheus` plugin today. It is deleted at teardown. If the
|
|
gateway does not carry equivalent signal, observability REGRESSES at cutover and
|
|
nobody notices until an incident.
|
|
|
|
- [ ] A Prometheus text-format endpoint is exposed and scrapeable without authentication from inside the cluster
|
|
- [ ] Request rate is observable, labelled by route and by upstream
|
|
- [ ] Request latency is observable as a distribution, not a mean, labelled by route and upstream
|
|
- [ ] Response status is observable, labelled by route, upstream, and status class
|
|
- [ ] Bandwidth in both directions is observable per route and upstream
|
|
- [ ] Upstream health is observable — whether each configured upstream is currently reachable and answering
|
|
- [ ] Label values are drawn from the configured route and upstream names, never from raw request paths or user input, so cardinality cannot be driven by a caller
|
|
- [ ] Streaming responses record their full byte count and full duration, not the time to first byte
|
|
- [ ] The metrics endpoint is not reachable through the public route surface
|
|
|
|
Write the assertions against the metric families first; they must fail because the
|
|
families are absent, not because the endpoint 404s.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
curl -s localhost:8080/metrics | grep -E '^# TYPE' | awk '{print $3}' | sort -u
|
|
# expected: includes counter and histogram families covering requests, duration, bytes, upstream health
|
|
|
|
curl -s -X POST localhost:8080/v1/chat/completions -H 'content-type: application/json' \
|
|
-d '{"model":"reasoning","messages":[]}' >/dev/null
|
|
curl -s localhost:8080/metrics | grep 'upstream="reasoning-predictor"' | head
|
|
# expected: request, duration and byte samples all carry route and upstream labels
|
|
```
|