rock
2c5f510951
poimen-memory will address `reasoning-long-predictor`, and the plan replaces
vLLM with candle-vllm underneath. Without an indirection, that handoff changes
a hostname the client is pinned to, and the ingest cutover has a window where
the name is wrong.
This Service is that indirection. It selects the vLLM reasoning pod today. The
handoff becomes a selector edit with no client change, so poimen-memory's
LLM_CHAT_BASE stays fixed across both engines.
Two ports, both verified through the Service from inside the poimen-memory pod:
80 -> 8080 HTTP 200 in 0.0057s matches every other predictor Service and
poimen-memory's LLM_CHAT_BASE, which has
no port in the URL
8080 -> 8080 HTTP 200 in 0.0182s the container port, the only port the two
NetworkPolicies permit, and the port the
handoff readiness check uses
Existing predictor Services publish port 80 only, so a curl to :8080 on one of
them fails to connect. Publishing both here means neither caller has to know
which convention the other follows.
Verified the selector resolves to the intended pod and nothing else: one
endpoint, 10.244.4.171, which is reasoning-predictor-59b67d5f5d-6mbbb.
Residual state, deliberately: /v1/models through this Service returns
['reasoning'], not ['reasoning-long'], because the second served name lands in
the previous commit and ArgoCD selfHeal reverts a live patch to a tracked pod
spec. Both names appear after this branch merges and syncs.
Not deleted and recreated at handoff. A delete drops the ClusterIP, and a
client that cached the name across the gap fails until it re-resolves.
44 lines
2.2 KiB
YAML
44 lines
2.2 KiB
YAML
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
# Explicit allowlist, matching k8s/apps/api. Anything added to this directory
|
|
# and not listed here is silently dropped — no error, no drift shown.
|
|
#
|
|
# These five were adopted from live state on 2026-08-15; they had been applied
|
|
# by hand and carried no ArgoCD ownership. Each was exported and verified with
|
|
# `kubectl diff -f <file>` returning empty before the Application below was
|
|
# created, so the first sync was a no-op rather than a redeploy. Re-verify that
|
|
# way after any edit here: a GPU predictor restart is a weights reload measured
|
|
# in tens of seconds, not a rolling update.
|
|
resources:
|
|
# clef-flash runs through clef-serve.yaml, not through an SGLang runtime. SGLang needs
|
|
# v0.5.22+, published only on CUDA 13, and CUDA 13 dropped Volta kernels. The model
|
|
# instead runs the implementation it ships (joint_schema_model.py) in fp16 on one V100.
|
|
- clef-serve.yaml
|
|
# coder = Qwen3-Coder-30B-A3B-Instruct, 4-bit GPTQ, TP=2 at 262,144 tokens. It serves
|
|
# both `coder` and `reasoning`, so poimen-memory needs a host change only, never a
|
|
# model-name change. reasoning.yaml below can be dropped once that host change lands.
|
|
- coder.yaml
|
|
- reasoning.yaml
|
|
# Stable client-facing name for the chat model. poimen-memory points at this Service,
|
|
# so the vLLM -> candle-vllm handoff is a selector edit here and no client change.
|
|
- reasoning-long-service.yaml
|
|
# ornith is retired: its InferenceService still reports Ready=True while running zero
|
|
# replicas, because desired equals available at 0. Its manifest was never committed,
|
|
# so it is left unmanaged and is not listed here. Delete the CR when convenient.
|
|
- embeddings.yaml
|
|
- reranker.yaml
|
|
- qwen-cpu.yaml
|
|
- networkpolicy.yaml
|
|
# No namespace transformer: every file sets its own, and the transformer would
|
|
# rewrite metadata.namespace on anything cross-namespace added later.
|
|
|
|
# app.py lives in git next to this file. The generator hashes the ConfigMap name, so an
|
|
# edit to app.py rolls the pod instead of leaving a stale copy mounted.
|
|
configMapGenerator:
|
|
- name: clef-serve-app
|
|
namespace: llm-serving
|
|
files:
|
|
- clef-serve/app.py
|
|
options:
|
|
disableNameSuffixHash: false
|