feat: cut api.riotpiao.com over to the Go gateway and retire Kong

Ingress api/api now backs onto api-gateway:8080; the kong Application, its
Helm values, plugins and llm-routes are removed. Gateway image v0.0.0 is in
the Forgejo registry and the pull secret is in the api namespace.
This commit is contained in:
Story Crater Bot
2026-08-19 22:51:53 -07:00
parent f8136f4e1e
commit 7e40a4baf6
13 changed files with 92 additions and 647 deletions
+8 -13
View File
@@ -3,20 +3,15 @@ kind: InferenceService
metadata:
annotations:
serving.kserve.io/deploymentMode: RawDeployment
# Kong reads its timeouts from the Kubernetes Service, not the Ingress —
# Ingress annotations configure Route entities (strip-path, methods,
# plugins), these configure the Service entity. They were on
# llm-chat-ornith's Ingress and therefore ignored, leaving Kong's 60s
# default in force. KServe propagates InferenceService annotations to the
# Service it generates, which is how they reach Kong from here.
# The konghq.com/{connect,read,write}-timeout annotations that used to live
# here went with Kong (retired 2026-08-19). They existed because Kong read
# its upstream timeouts off the Kubernetes Service, and its 60s default cut
# off the first request after any pod restart — a restart flushes VRAM and
# reloading ornith:35b takes longer than that. OLLAMA_KEEP_ALIVE=-1 hid the
# problem in steady state.
#
# This was invisible while OLLAMA_KEEP_ALIVE=-1 kept the model resident: no
# request ever waited on a cold load. A pod restart flushes VRAM, and
# loading ornith:35b takes longer than 60s, so the first request after any
# restart returned 504.
konghq.com/connect-timeout: "10000"
konghq.com/read-timeout: "3600000"
konghq.com/write-timeout: "3600000"
# The equivalent budget now belongs to the Go gateway's per-route timeout
# config in rock/homelab-frontend, not to an annotation on this object.
labels:
app.kubernetes.io/name: llm-ornith
app.kubernetes.io/part-of: llm-serving