feat(llm-serving): also serve the 30B-A3B as reasoning-long

The goal model map names this checkpoint `reasoning-long`: Qwen3-30B-A3B at
64K on one card, the model qwen-finetune wiring and the models.json overlay
target. It was deployed and healthy but only answerable as `reasoning`, so
that name did not exist.

It is a second served name on the existing pod, not a new
InferenceService. All four V100s are allocated (coder 2, this 1,
clef-serve 1), so a separate reasoning-long predictor would sit Pending
forever. It would also load the same 15.56 GiB of weights a second time to
serve a name, which buys nothing.

The two names are separate list entries. The comma form does NOT work here,
and fails silently in a way that looks like success: this image's parser
turns `--served-model-name=reasoning,reasoning-long` into
['reasoning,reasoning-long'], one model whose name contains a comma, and
api_server.py iterates args.served_model_name when it builds the model list.
Verified against vLLM 0.11.0 itself rather than assumed:

  FlexibleArgumentParser().parse_args(["--model","m",
      "--served-model-name=reasoning,reasoning-long"])
    -> ['reasoning,reasoning-long']
  FlexibleArgumentParser().parse_args(["--model","m",
      "--served-model-name","reasoning","reasoning-long"])
    -> ['reasoning', 'reasoning-long']

`reasoning` stays served from this pod on purpose. poimen-memory pins
LLM_CHAT_BASE=reasoning-predictor, so removing the name breaks ingest
outright. The duplicate with coder-predictor is known and is resolved by
moving the client host, not by deleting a name here.

Not verified end to end: a live apply was reverted by ArgoCD selfHeal before
the new pod finished loading, so the served-name list was never observed on
a running pod with both names. The parser result above is what this commit
rests on. Confirm with `GET /v1/models` after the merge syncs.
This commit is contained in:
2026-10-31 19:38:26 -07:00
parent 42617fbd7c
commit 871bb8bfd4
+22 -1
View File
@@ -30,7 +30,28 @@ spec:
# Measured on this cluster, one V100-32GB, 64k context: 15.55 GiB of weights,
# 137k tokens of KV cache, 2.09x concurrency at 65536 tokens per request.
- --model=Qwen/Qwen3-30B-A3B-GPTQ-Int4
- --served-model-name=reasoning
# Two served names, one pod, on purpose. `reasoning-long` is the name the goal map
# uses for this model, and the name qwen-finetune wiring targets. It is NOT a
# separate InferenceService: this model already occupies the only free card, and a
# second pod on the same weights cannot schedule (4 of 4 GPUs allocated: coder 2,
# this 1, clef-serve 1). Serving both names from one process costs nothing and
# keeps the two callers working without a second weights copy.
#
# `reasoning` is deliberately still served here. poimen-memory pins
# LLM_CHAT_BASE=reasoning-predictor, so dropping the name here breaks ingest
# immediately. coder-predictor also serves `reasoning`; that ambiguity is known
# and documented in the PR. Resolve it by moving the client host, not by
# removing the name here.
#
# The two names are separate list entries, NOT one comma-joined value. This
# image's parser returns ['reasoning,reasoning-long'] for the comma form, i.e. a
# single model whose name contains a comma, and api_server.py then iterates
# args.served_model_name to build the model list. Verified against vLLM 0.11.0:
# FlexibleArgumentParser().parse_args([... "--served-model-name", "a", "b"])
# -> ['a', 'b']
- --served-model-name
- reasoning
- reasoning-long
- --quantization=gptq
- --dtype=float16
- --kv-cache-dtype=auto