44 lines
1.6 KiB
YAML
44 lines
1.6 KiB
YAML
|
# Stable client-facing name for the chat model, deliberately decoupled from the
|
||
|
|
# process that currently serves it.
|
||
|
|
#
|
||
|
|
# poimen-memory addresses this Service, not a pod and not an InferenceService. That
|
||
|
|
# means the vLLM -> candle-vllm handoff changes only this file's selector. The client
|
||
|
|
# config never changes again, so the ingest cutover has no window in which the host
|
||
|
|
# name is wrong.
|
||
|
|
#
|
||
|
|
# Two ports on purpose:
|
||
|
|
# - 80 matches every other predictor Service in this namespace, and matches
|
||
|
|
# poimen-memory's LLM_CHAT_BASE, which carries no port number.
|
||
|
|
# - 8080 is the container port. It is also the only port llm-serving-ingress and
|
||
|
|
# poimen-memory-llm-egress permit, and the port the handoff readiness check
|
||
|
|
# uses. Publishing it directly avoids a port translation that is easy to get
|
||
|
|
# wrong in a curl.
|
||
|
|
#
|
||
|
|
# The selector names the vLLM reasoning pod today. To hand off, replace the three
|
||
|
|
# selector labels with candle-vllm's pod labels and nothing else. Do not delete and
|
||
|
|
# recreate this Service: a delete drops the ClusterIP, and a client that cached the
|
||
|
|
# name during the gap will fail until it re-resolves.
|
||
|
|
apiVersion: v1
|
||
|
|
kind: Service
|
||
|
|
metadata:
|
||
|
|
name: reasoning-long-predictor
|
||
|
|
namespace: llm-serving
|
||
|
|
labels:
|
||
|
|
app.kubernetes.io/part-of: llm-serving
|
||
|
|
app: reasoning-long-predictor
|
||
|
|
spec:
|
||
|
|
type: ClusterIP
|
||
|
|
selector:
|
||
|
|
app.kubernetes.io/name: llm-reasoning
|
||
|
|
component: predictor
|
||
|
|
serving.kserve.io/inferenceservice: reasoning
|
||
|
|
ports:
|
||
|
|
- name: http
|
||
|
|
protocol: TCP
|
||
|
|
port: 80
|
||
|
|
targetPort: 8080
|
||
|
|
- name: http-8080
|
||
|
|
protocol: TCP
|
||
|
|
port: 8080
|
||
|
|
targetPort: 8080
|