rock
41dcda819e
The Clef runtime was running from hand-applied objects that existed only in the cluster.
A rebuild would have lost it.
Three things move into git:
- clef-serve.yaml: the Deployment and the Service the gateway routes to
- configMapGenerator for clef-serve/app.py, so an edit to the code rolls the pod
instead of leaving a stale copy mounted
- clef-flash.yaml is deleted: it described an SGLang predictor that cannot run on a
V100, and its slot is now clef-serve.yaml
Recorded in the manifest, because both cost hours: SGLang needs CUDA 13 for Clef and
CUDA 13 has no sm_70 kernels, and device_map over two cards breaks the forward pass with
'indices should be either on cpu or on the same device as the indexed tensor'.
Still open and written down in the file: the image cannot be pushed, because Cloudflare
answers 413 for layers above 100 MB. So each start pip-installs and needs PyPI.
37 lines
1.7 KiB
YAML
37 lines
1.7 KiB
YAML
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
# Explicit allowlist, matching k8s/apps/api. Anything added to this directory
|
|
# and not listed here is silently dropped — no error, no drift shown.
|
|
#
|
|
# These five were adopted from live state on 2026-08-15; they had been applied
|
|
# by hand and carried no ArgoCD ownership. Each was exported and verified with
|
|
# `kubectl diff -f <file>` returning empty before the Application below was
|
|
# created, so the first sync was a no-op rather than a redeploy. Re-verify that
|
|
# way after any edit here: a GPU predictor restart is a weights reload measured
|
|
# in tens of seconds, not a rolling update.
|
|
resources:
|
|
# clef-flash runs through clef-serve.yaml, not through an SGLang runtime. SGLang needs
|
|
# v0.5.22+, published only on CUDA 13, and CUDA 13 dropped Volta kernels. The model
|
|
# instead runs the implementation it ships (joint_schema_model.py) in fp16 on one V100.
|
|
- clef-serve.yaml
|
|
- reasoning.yaml
|
|
# ornith is retired: its InferenceService still reports Ready=True while running zero
|
|
# replicas, because desired equals available at 0. Its manifest was never committed,
|
|
# so it is left unmanaged and is not listed here. Delete the CR when convenient.
|
|
- embeddings.yaml
|
|
- reranker.yaml
|
|
- qwen-cpu.yaml
|
|
- networkpolicy.yaml
|
|
# No namespace transformer: every file sets its own, and the transformer would
|
|
# rewrite metadata.namespace on anything cross-namespace added later.
|
|
|
|
# app.py lives in git next to this file. The generator hashes the ConfigMap name, so an
|
|
# edit to app.py rolls the pod instead of leaving a stale copy mounted.
|
|
configMapGenerator:
|
|
- name: clef-serve-app
|
|
namespace: llm-serving
|
|
files:
|
|
- clef-serve/app.py
|
|
options:
|
|
disableNameSuffixHash: false
|