feat: cut api.riotpiao.com over to the Go gateway and retire Kong
Ingress api/api now backs onto api-gateway:8080; the kong Application, its Helm values, plugins and llm-routes are removed. Gateway image v0.0.0 is in the Forgejo registry and the pull secret is in the api namespace.
This commit is contained in:
@@ -3,20 +3,15 @@ kind: InferenceService
|
||||
metadata:
|
||||
annotations:
|
||||
serving.kserve.io/deploymentMode: RawDeployment
|
||||
# Kong reads its timeouts from the Kubernetes Service, not the Ingress —
|
||||
# Ingress annotations configure Route entities (strip-path, methods,
|
||||
# plugins), these configure the Service entity. They were on
|
||||
# llm-chat-ornith's Ingress and therefore ignored, leaving Kong's 60s
|
||||
# default in force. KServe propagates InferenceService annotations to the
|
||||
# Service it generates, which is how they reach Kong from here.
|
||||
# The konghq.com/{connect,read,write}-timeout annotations that used to live
|
||||
# here went with Kong (retired 2026-08-19). They existed because Kong read
|
||||
# its upstream timeouts off the Kubernetes Service, and its 60s default cut
|
||||
# off the first request after any pod restart — a restart flushes VRAM and
|
||||
# reloading ornith:35b takes longer than that. OLLAMA_KEEP_ALIVE=-1 hid the
|
||||
# problem in steady state.
|
||||
#
|
||||
# This was invisible while OLLAMA_KEEP_ALIVE=-1 kept the model resident: no
|
||||
# request ever waited on a cold load. A pod restart flushes VRAM, and
|
||||
# loading ornith:35b takes longer than 60s, so the first request after any
|
||||
# restart returned 504.
|
||||
konghq.com/connect-timeout: "10000"
|
||||
konghq.com/read-timeout: "3600000"
|
||||
konghq.com/write-timeout: "3600000"
|
||||
# The equivalent budget now belongs to the Go gateway's per-route timeout
|
||||
# config in rock/homelab-frontend, not to an annotation on this object.
|
||||
labels:
|
||||
app.kubernetes.io/name: llm-ornith
|
||||
app.kubernetes.io/part-of: llm-serving
|
||||
|
||||
Reference in New Issue
Block a user