riotpiao-poimen/homelab-frontend
2
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gateway): dispatch /v1/systemone by model

Browse Source 
Clef-flash is a decision model, not a chat model. It serves Clef's own
System One / Jev API at /v1/systemone: a state plus a schema of typed
questions in, one probability per allowed option out, single prefill
pass, no decoding and no free-form text.

Router previously dispatched by body `model` only for
/v1/chat/completions, /v1/embeddings and /v1/rerank. A request to
/v1/systemone therefore fell through to the configured routes, found
none, and answered "no routes configured". Adding the path to the
dispatch predicate is the whole fix: routeByModel already sets
upstreamPath := path, so the upstream receives /v1/systemone unchanged.
There is deliberately no /rerank-style rewrite.

Config: `ornith:35b` is removed with the Ollama predictor, and its GPU
now serves clef-flash. `qwen2.5:3b-instruct` already pointed at qwen-cpu
and is unchanged. Callers of ornith:35b must move to `reasoning`, which
keeps its serving name while running Qwen3-30B-A3B.

Tests cover body dispatch, unknown_model parity with the other three
paths, and that the upstream path is not rewritten.
This commit is contained in:
Admin Bot
2026-10-27 23:32:24 -07:00
parent 67fe20d7a3
commit d521d7a48b
3 changed files with 177 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/proxy/router.go
+7 -4
View File
@@ -13,11 +13,14 @@ import (
)
// RouteRequest determines which upstream should handle the request.
// For /v1/chat/completions, /v1/embeddings, and /v1/rerank, it uses body-based dispatch.
// For /v1/chat/completions, /v1/embeddings, /v1/rerank and /v1/systemone, it
// uses body-based dispatch.
// For other routes, it looks up by path in the configured routes.
func (h *Handler) RouteRequest(r *http.Request) (*Route, error) {
// For /v1/chat/completions, /v1/embeddings, /v1/rerank use body-based dispatch
if r.Method == "POST" && (r.URL.Path == "/v1/chat/completions" || r.URL.Path == "/v1/embeddings" || r.URL.Path == "/v1/rerank") {
// For /v1/chat/completions, /v1/embeddings, /v1/rerank and /v1/systemone use
// body-based dispatch. /v1/systemone is Clef's decision API: same
// body-based model lookup, different request/response schema.
if r.Method == "POST" && (r.URL.Path == "/v1/chat/completions" || r.URL.Path == "/v1/embeddings" || r.URL.Path == "/v1/rerank" || r.URL.Path == "/v1/systemone") {
return h.routeByModel(r, r.URL.Path)
}
@@ -46,7 +49,7 @@ func (h *Handler) RouteRequest(r *http.Request) (*Route, error) {
// routeByModel reads the request body to find the "model" field and routes accordingly.
// The body is preserved for forwarding to the upstream.
// Returns a modelValidationError for client errors (invalid JSON, missing/unknown model).
// The path parameter indicates which endpoint is being called (/v1/chat/completions, /v1/embeddings, /v1/rerank)
// The path parameter indicates which endpoint is being called (/v1/chat/completions, /v1/embeddings, /v1/rerank, /v1/systemone)
func (h *Handler) routeByModel(r *http.Request, path string) (*Route, error) {
// If there's no body, we can't determine the model
if r.Body == nil {