feat: add upstreamModel config for model name mapping
When the upstream LLM server expects a different model name than what
clients send, the gateway now rewrites the 'model' field in the request
body before forwarding.
Config example:
models:
- name: "ornith:35b" # client-facing name
address: "ornith-predictor:80"
upstreamModel: "qwen2.5:72b" # what upstream expects
Fixes 400 'model is required' errors when upstream model names differ.
This commit is contained in:
@@ -36,6 +36,10 @@ type ModelUpstream struct {
|
||||
Address string
|
||||
// Path is the upstream path for this model (e.g., "/v1/chat/completions").
|
||||
Path string
|
||||
// UpstreamModel is the model name to send to the upstream server.
|
||||
// If empty, the client-provided model name (Name) is used as-is.
|
||||
// Use this when the upstream expects a different model name than clients send.
|
||||
UpstreamModel string
|
||||
// AuthRequired indicates whether this model requires JWT authentication.
|
||||
AuthRequired bool
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user