rock
c36aa56b52
Both new ServiceMonitors sat permanently OutOfSync. Cause: is the API server default for relabeling rules, so the live object carried it while the manifests omitted it. Rather than write out the default to paper over drift, removed the rules that were never needed: - vllm.yaml: the namespace/pod/service relabelings were redundant — Kubernetes service discovery already supplies all three. Only has to be derived (TEI panels aggregate by it and SD does not promote arbitrary pod labels). Kept that one, with an explicit action. - api-gateway.yaml: the metricRelabelings block was a no-op — it mapped __address__ onto instance, which is already the default. Removed. The series inventory it was documenting moved up into the file header, which is where it belonged anyway.
47 lines
1.9 KiB
YAML
47 lines
1.9 KiB
YAML
# The gateway is the only source of per-request LLM latency covering EVERY
|
|
# model, including the two Ollama ones (ornith:35b, qwen2.5:3b-instruct) that
|
|
# expose no metrics endpoint at all. Series it emits, per
|
|
# internal/observability/prometheus.go:
|
|
# llm_ttft_seconds histogram time-to-first-token
|
|
# llm_itl_seconds histogram inter-token latency
|
|
# llm_tokens_total counter SSE chunks generated
|
|
# queue_full_requests_total counter backpressure rejects
|
|
# warmup_requests_delayed_total counter vLLM slot contention
|
|
# vllm_prefix_cache_hit_tokens_total counter prefix-cache hits
|
|
# request_duration_seconds histogram full request
|
|
#
|
|
# Caveat: these are derived from the SSE byte stream, not from the engine.
|
|
# TTFT is request-start to first byte, ITL is the gap between SSE writes
|
|
# carrying "data: ", and llm_tokens_total counts SSE chunks rather than true
|
|
# tokens. They are also streaming-only — non-streaming requests record
|
|
# nothing. Treat vLLM's own histograms as authoritative for `reasoning`.
|
|
#
|
|
# The Deployment already carried prometheus.io/scrape annotations, but
|
|
# kube-prometheus-stack ignores pod annotations (no additionalScrapeConfigs,
|
|
# serviceMonitorSelectorNilUsesHelmValues: false), so none of the above was
|
|
# ever collected. This ServiceMonitor is what actually wires it up.
|
|
apiVersion: monitoring.coreos.com/v1
|
|
kind: ServiceMonitor
|
|
metadata:
|
|
name: api-gateway
|
|
namespace: monitoring
|
|
labels:
|
|
app.kubernetes.io/name: api-gateway
|
|
app.kubernetes.io/part-of: api
|
|
spec:
|
|
namespaceSelector:
|
|
matchNames:
|
|
- api
|
|
selector:
|
|
matchLabels:
|
|
app: api-gateway
|
|
endpoints:
|
|
# api-gateway Service port IS named `http` (unlike the KServe ones), so
|
|
# the name form works here.
|
|
- port: http
|
|
interval: 30s
|
|
scrapeTimeout: 10s
|
|
path: /metrics
|
|
scheme: http
|
|
|