fix(monitoring): wire llm-serving + api-gateway metric scraping

Both metric sources existed but neither was scraped:

- servicemonitors/vllm.yaml was orphaned (absent from kustomization
  resources:) and would not have matched anyway — it used 'port: http'
  but KServe-generated Services in llm-serving have UNNAMED ports.
  Switched to targetPort: 8080 (prometheus-operator v0.94.1 supports it),
  which avoids editing the InferenceServices and therefore avoids a
  predictor restart / weights reload.

- api-gw exposes llm_ttft_seconds, llm_itl_seconds, llm_tokens_total,
  queue_full_requests_total, warmup_requests_delayed_total and
  vllm_prefix_cache_hit_tokens_total, but the Deployment's
  prometheus.io/scrape annotation is ignored by kube-prometheus-stack
  (no additionalScrapeConfigs). Added a ServiceMonitor for it. This is
  the only per-request latency source covering the two Ollama models,
  which expose no metrics endpoint at all.

Ollama services are excluded from the vllm ServiceMonitor selector to
avoid permanently-DOWN targets.

Note: alerts/cag-kv-cache-rules.yaml references vllm:prefix_cache_hit_rate
derived from vllm:cache_query_hit_total/miss, which were never scraped.
Those alerts were dead; verify the metric names after the first scrape.

This commit is contained in:

rock

2026-09-29 22:05:00 +08:00

parent e6fd6f2b62

commit 6ae36981bb

3 changed files with 69 additions and 3 deletions

									
										k8s/infra/monitoring/kustomization.yaml
									
		+2
		
												View File
												
				@@ -20,5 +20,7 @@ resources:

				  - servicemonitors/forgejo.yaml

				  - servicemonitors/minio.yaml

				  - servicemonitors/ingress-nginx.yaml

				  - servicemonitors/vllm.yaml

				  - servicemonitors/api-gateway.yaml

				  - dashboards/cluster-infrastructure.yaml

				  - dashboards/api-gateway.yaml