36 lines
1.8 KiB
Markdown
36 lines
1.8 KiB
Markdown
|
# 2.1 — Model registry (GREEN)
|
||
|
|
|
||
|
|
Phase: 2 — LLM surface
|
||
|
|
Stage: GREEN
|
||
|
|
Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
|
||
|
|
|
||
|
|
- [ ] A model name -> upstream map is loaded from configuration at startup, never compiled in
|
||
|
|
- [ ] Each entry carries at minimum the model name clients send, the upstream address, and the upstream path to use
|
||
|
|
- [ ] Two model names may point at the same upstream address, and both resolve independently
|
||
|
|
- [ ] A duplicate model name, an empty model name, or an entry with no upstream address fails startup loudly with the offending entry named
|
||
|
|
- [ ] The registry is queryable by exact model name; lookup is case-sensitive and does no fuzzy matching or defaulting
|
||
|
|
- [ ] The set of known model names is enumerable, because `/v1/models` and unknown-model errors are both derived from it
|
||
|
|
|
||
|
|
The five entries verified live on 2026-08-19. Ports are 80, not 8080.
|
||
|
|
|
||
|
|
| model name clients send | upstream Service | engine |
|
||
|
|
|---|---|---|
|
||
|
|
| `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B |
|
||
|
|
| `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama |
|
||
|
|
| `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama, same pods |
|
||
|
|
| `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI |
|
||
|
|
| `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI |
|
||
|
|
|
||
|
|
`ornith:35b` and `qwen2.5:3b-instruct` share pods and both stay resident, so a
|
||
|
|
registry that maps them to one address is correct, not a shortcut.
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# with the five entries configured against local stubs
|
||
|
|
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 200
|
||
|
|
|
||
|
|
# duplicate model name in config must refuse to start
|
||
|
|
./gateway --config testdata/duplicate-model.yaml; echo "exit=$?" # expected: non-zero exit, stderr names the duplicated model
|
||
|
|
```
|