riotpiao-poimen/homelab
2
0
Fork 0
Code Issues 24 Pull Requests 1 Actions Packages Projects 1 Releases Wiki Activity

feat(llm-serving): run clef-serve so /v1/systemone answers

Browse Source 
clef-serve was parked at 0 while candle-vllm took a card. worker-1 holds
four V100s, so the coder takes one and clef takes one, leaving two free
for a TP=2 candle run.

The gateway already routes the clef-flash model to clef-serve:80 on
/v1/systemone, so this single replica count is what turns that route on.
This commit is contained in:
rock
2026-11-04 18:59:56 +08:00
parent 67760720fc
commit db97159905
1 changed files with 9 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/apps/llm-serving/clef-serve.yaml
+9 -11
View File
@@ -33,19 +33,17 @@ metadata:
app: clef-serve
app.kubernetes.io/part-of: llm-serving
spec:
# Parked at 0 to free the only V100 that was not serving anything, so candle-vllm can
# run its Phase 0 on one card. All four cards were allocated (coder 2, reasoning 1,
# this 1) and a new GPU workload had nowhere to land.
# One card. worker-1 holds four V100s: candle-vllm takes one for the coder, and
# this pod takes one, so two stay free for a TP=2 candle run.
#
# This is the only GPU consumer in the namespace that is not vLLM, which is why it is
# the first card released: the standing instruction is to keep the vLLM pods healthy
# until candle-vllm answers, and this pod is not one of them.
# Enabled 2026-11-04 to serve /v1/systemone, which the gateway already routes to
# this Service. It was parked at 0 while candle-vllm was brought up on the same
# cards, and it served no inference request between 2026-10-30T06:23:40Z and then.
#
# It was serving nothing. The last 30 log lines at park time were 30/30 kubelet
# /health probes and no inference request since start (2026-10-30T06:23:40Z). Nothing
# in git selects this service. Restore it once the rollout leaves a card free:
# 30B-A3B takes 1, Coder-Next TP=2 takes 2, so card 4 is this one.
replicas: 0
# Cold start is about four minutes: the pod pip-installs on every boot because the
# torch base image cannot be pushed to the registry (Cloudflare answers 413 above
# 100 MB per layer). startupProbe allows for that.
replicas: 1
# Recreate, not RollingUpdate. Every GPU card is allocated, so a rolling update leaves
# the new pod Pending forever waiting for a card the old pod still holds.
strategy: