rock
ad2720a724
Clef needs SGLang v0.5.22+, which upstream publishes only on CUDA 13. CUDA 13 dropped Volta kernels and every GPU here is a Tesla V100 (sm_70): torch.AcceleratorError: CUDA error: no kernel image is available for execution on the device SGLang also states sm75 as its floor: the README hardware table omits V100, sm75 support was removed in sgl-project/sglang#9207, and FlashInfer requires sm75+. No build has both Clef and Volta kernels, so the model is not runnable here at any dtype or flag. Remove it from the kustomization so nothing crash-loops, and record the reason in both files. The manifest stays in git for the day an Ampere-or-newer GPU arrives.
41 lines
2.2 KiB
YAML
41 lines
2.2 KiB
YAML
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
# Explicit allowlist, matching k8s/apps/api. Anything added to this directory
|
|
# and not listed here is silently dropped — no error, no drift shown.
|
|
#
|
|
# These five were adopted from live state on 2026-08-15; they had been applied
|
|
# by hand and carried no ArgoCD ownership. Each was exported and verified with
|
|
# `kubectl diff -f <file>` returning empty before the Application below was
|
|
# created, so the first sync was a no-op rather than a redeploy. Re-verify that
|
|
# way after any edit here: a GPU predictor restart is a weights reload measured
|
|
# in tens of seconds, not a rolling update.
|
|
resources:
|
|
# clef-flash.yaml is NOT listed: the model cannot run on this cluster.
|
|
#
|
|
# Clef needs SGLang v0.5.22 or newer, and SGLang publishes that only on CUDA 13
|
|
# (the last CUDA 12 tag is v0.5.19-cu129, which predates Clef support). CUDA 13
|
|
# dropped Volta kernels, and every GPU here is a Tesla V100 (sm_70):
|
|
#
|
|
# torch.AcceleratorError: CUDA error: no kernel image is available for
|
|
# execution on the device
|
|
#
|
|
# SGLang also states sm75 as its floor: the README hardware list omits V100,
|
|
# sm75 support was removed in sgl-project/sglang#9207, and FlashInfer needs
|
|
# sm75+. So no SGLang build exists that has both Clef and Volta kernels, and
|
|
# no flag or dtype change bridges that gap.
|
|
#
|
|
# Re-add it when the model gets an Ampere-or-newer GPU. The manifest itself is
|
|
# kept in git (k8s/apps/llm-serving/clef-flash.yaml) with the same reasoning.
|
|
# ornith and reasoning are live but unmanaged: their manifests were never committed
|
|
# to this directory, so listing them made `kustomize build` fail and left the whole
|
|
# Application in ComparisonError. Remove the entries rather than the models. The
|
|
# Application runs with prune=false, so an object that is no longer in the desired
|
|
# state is left alone, not deleted. Re-export both from live state when they are
|
|
# next edited.
|
|
- embeddings.yaml
|
|
- reranker.yaml
|
|
- qwen-cpu.yaml
|
|
- networkpolicy.yaml
|
|
# No namespace transformer: every file sets its own, and the transformer would
|
|
# rewrite metadata.namespace on anything cross-namespace added later.
|