fix(monitoring): wire llm-serving + api-gateway metric scraping

Both metric sources existed but neither was scraped:

- servicemonitors/vllm.yaml was orphaned (absent from kustomization
  resources:) and would not have matched anyway — it used 'port: http'
  but KServe-generated Services in llm-serving have UNNAMED ports.
  Switched to targetPort: 8080 (prometheus-operator v0.94.1 supports it),
  which avoids editing the InferenceServices and therefore avoids a
  predictor restart / weights reload.

- api-gw exposes llm_ttft_seconds, llm_itl_seconds, llm_tokens_total,
  queue_full_requests_total, warmup_requests_delayed_total and
  vllm_prefix_cache_hit_tokens_total, but the Deployment's
  prometheus.io/scrape annotation is ignored by kube-prometheus-stack
  (no additionalScrapeConfigs). Added a ServiceMonitor for it. This is
  the only per-request latency source covering the two Ollama models,
  which expose no metrics endpoint at all.

Ollama services are excluded from the vllm ServiceMonitor selector to
avoid permanently-DOWN targets.

Note: alerts/cag-kv-cache-rules.yaml references vllm:prefix_cache_hit_rate
derived from vllm:cache_query_hit_total/miss, which were never scraped.
Those alerts were dead; verify the metric names after the first scrape.

k8s/infra/monitoring/servicemonitors/api-gateway.yaml

@@ -0,0 +1,48 @@
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  name: api-gateway
+  namespace: monitoring
+  labels:
+    app.kubernetes.io/name: api-gateway
+    app.kubernetes.io/part-of: api
+spec:
+  namespaceSelector:
+    matchNames:
+      - api
+  selector:
+    matchLabels:
+      app: api-gateway
+  endpoints:
+    # api-gateway Service port IS named `http` (unlike the KServe ones), so
+    # the name form works here.
+    - port: http
+      interval: 30s
+      scrapeTimeout: 10s
+      path: /metrics
+      scheme: http
+
+      # The gateway is the only source of per-request LLM latency that covers
+      # EVERY model, including the two Ollama ones that expose no metrics at
+      # all. Series it produces (internal/observability/prometheus.go):
+      #   llm_ttft_seconds              histogram  time-to-first-token
+      #   llm_itl_seconds               histogram  inter-token latency
+      #   llm_tokens_total              counter    tokens generated
+      #   queue_full_requests_total     counter    backpressure rejects
+      #   warmup_requests_delayed_total counter    vLLM slot contention
+      #   vllm_prefix_cache_hit_tokens_total counter prefix-cache hits
+      #   request_duration_seconds      histogram  full request
+      #
+      # The Deployment already carried prometheus.io/scrape annotations, but
+      # kube-prometheus-stack ignores pod annotations (no additionalScrape-
+      # Configs, serviceMonitorSelectorNilUsesHelmValues: false) — so none of
+      # the above was being collected. This ServiceMonitor is what actually
+      # wires it up.
+      metricRelabelings:
+        # 3 gateway replicas each build their own in-process histogram. Without
+        # this the label set is otherwise identical, so per-instance series
+        # would collide conceptually in sum()-by aggregations; instance is
+        # retained deliberately so per-replica skew stays visible.
+        - sourceLabels: [__address__]
+          targetLabel: instance
+          replacement: $1