26 lines
1.3 KiB
Markdown
26 lines
1.3 KiB
Markdown
|
# 2.6 — `POST /v1/embeddings` passthrough (GREEN)
|
||
|
|
|
||
|
|
Phase: 2 — LLM surface
|
||
|
|
Stage: GREEN
|
||
|
|
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
|
||
|
|
|
||
|
|
- [ ] `POST /v1/embeddings` reaches `embeddings-predictor.llm-serving:80` at the path `/v1/embeddings`, unrewritten
|
||
|
|
- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
|
||
|
|
- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration rather than inherited
|
||
|
|
- [ ] The route has an explicit body size cap
|
||
|
|
- [ ] Upstream error statuses are surfaced as-is; the gateway invents no retries and no substitute response
|
||
|
|
|
||
|
|
TEI already serves the canonical path, so this route rewrites nothing. The model
|
||
|
|
served here is `nomic-ai/nomic-embed-text-v2-moe`. Port is 80, not 8080. Whether this
|
||
|
|
route dispatches on the body's `model` or is pinned to the single embeddings upstream
|
||
|
|
is a design decision for whoever works it; either way the path must not change and an
|
||
|
|
unknown `model` must not silently reach the wrong upstream.
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
```bash
|
||
|
|
curl -s -i localhost:8080/v1/embeddings -H 'content-type: application/json' \
|
||
|
|
-d '{"model":"nomic-ai/nomic-embed-text-v2-moe","input":"hello"}'
|
||
|
|
# expected: 200, embeddings stub hit at path /v1/embeddings, request body unmodified, response body byte-identical to the stub's
|
||
|
|
```
|