Files
homelab/k8s/apps
rock 871bb8bfd4 feat(llm-serving): also serve the 30B-A3B as reasoning-long
The goal model map names this checkpoint `reasoning-long`: Qwen3-30B-A3B at
64K on one card, the model qwen-finetune wiring and the models.json overlay
target. It was deployed and healthy but only answerable as `reasoning`, so
that name did not exist.

It is a second served name on the existing pod, not a new
InferenceService. All four V100s are allocated (coder 2, this 1,
clef-serve 1), so a separate reasoning-long predictor would sit Pending
forever. It would also load the same 15.56 GiB of weights a second time to
serve a name, which buys nothing.

The two names are separate list entries. The comma form does NOT work here,
and fails silently in a way that looks like success: this image's parser
turns `--served-model-name=reasoning,reasoning-long` into
['reasoning,reasoning-long'], one model whose name contains a comma, and
api_server.py iterates args.served_model_name when it builds the model list.
Verified against vLLM 0.11.0 itself rather than assumed:

  FlexibleArgumentParser().parse_args(["--model","m",
      "--served-model-name=reasoning,reasoning-long"])
    -> ['reasoning,reasoning-long']
  FlexibleArgumentParser().parse_args(["--model","m",
      "--served-model-name","reasoning","reasoning-long"])
    -> ['reasoning', 'reasoning-long']

`reasoning` stays served from this pod on purpose. poimen-memory pins
LLM_CHAT_BASE=reasoning-predictor, so removing the name breaks ingest
outright. The duplicate with coder-predictor is known and is resolved by
moving the client host, not by deleting a name here.

Not verified end to end: a live apply was reverted by ArgoCD selfHeal before
the new pod finished loading, so the served-name list was never observed on
a running pod with both names. The parser result above is what this commit
rests on. Confirm with `GET /v1/models` after the merge syncs.
2026-10-31 19:38:26 -07:00
..
2026-10-26 03:32:18 -07:00