37 lines
2.2 KiB
Markdown
37 lines
2.2 KiB
Markdown
|
# 6.4 — Deploy unexposed alongside Kong (GREEN)
|
||
|
|
|
||
|
|
Phase: 6 — Deploy and cutover
|
||
|
|
Stage: GREEN
|
||
|
|
Depends on: [6.3](6.3-argocd-application.md)
|
||
|
|
|
||
|
|
The gateway runs in the cluster against the real predictors while Kong continues to
|
||
|
|
serve every byte of live traffic. This is the last step before anything user-visible
|
||
|
|
changes, and it is fully reversible — deleting the Application removes it.
|
||
|
|
|
||
|
|
- [ ] The gateway is running in namespace `api` and is reachable only in-cluster at `http://<svc>.api.svc.cluster.local`
|
||
|
|
- [ ] No Ingress points at the gateway. Ingress `api/api` still sends `/` on `api.riotpiao.com` to `kong-proxy:80`
|
||
|
|
- [ ] Its configured upstreams are the real Services: `reasoning-predictor.llm-serving:80`, `ornith-predictor.llm-serving:80`, `embeddings-predictor.llm-serving:80`, `reranker-predictor.llm-serving:80`, and `agent-hub.agent-pod:9090`
|
||
|
|
- [ ] Note `reasoning-predictor` listens on port 80, not 8080
|
||
|
|
- [ ] A real chat completion succeeds in-cluster against `reasoning`, returning `reasoning_content` and `content` as separate fields
|
||
|
|
- [ ] A streaming chat completion delivers tokens incrementally in-cluster, not as one buffered blob at completion
|
||
|
|
- [ ] Embeddings and rerank both answer correctly, with rerank reaching the upstream's `/rerank` path
|
||
|
|
- [ ] A client disconnect mid-stream is observed to cancel the upstream generation and release its sequence slot
|
||
|
|
- [ ] Prometheus is scraping the gateway and the metrics reflect this in-cluster traffic
|
||
|
|
- [ ] `api.riotpiao.com` is unaffected throughout — verified before and after
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
```bash
|
||
|
|
kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
|
||
|
|
# expected: kong-proxy — the gateway is still unexposed
|
||
|
|
|
||
|
|
kubectl -n api run probe --rm -it --image=curlimages/curl --restart=Never -- sh -c '
|
||
|
|
curl -sN -X POST http://homelab-frontend.api.svc.cluster.local/v1/chat/completions \
|
||
|
|
-H "content-type: application/json" \
|
||
|
|
-d "{\"model\":\"reasoning\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"count to 5\"}]}"'
|
||
|
|
# expected: multiple data: chunks arriving over time, terminated by data: [DONE]
|
||
|
|
|
||
|
|
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
|
||
|
|
# expected: 200 — public traffic still served by Kong, untouched
|
||
|
|
```
|