fix(api): unbreak >10.6KB LLM requests and actually bind key-auth to the model routes
Two independent bugs, both silent, both found while pointing an agent harness
at api.riotpiao.com.
1. Requests over ~10.6KB failed with HTTP 400
{"error":{"message":"[] is too short - 'messages'"}}.
The request-transformer plugin on the llm-chat-* routes rewrites the JSON
body, which means it reads it via kong.request.get_body(). That returns
nothing once nginx spills the body past client_body_buffer_size into a temp
file, so the plugin re-serialized a body with no `messages` and the upstream
rejected it. Measured on /v1/ornith/chat/completions: 10588 B -> 200,
11088 B -> 400. Isolated by size-sweeping /v1/embeddings, the one route with
no request-transformer, which passed an 18057 B body straight through to a
semantic 413 from TEI.
Raises the Kong http-block buffer to 16m. Any agent request carrying tool
schemas clears the old ceiling in a single turn.
2. key-auth was never applied to any model route.
The model-key-auth KongPlugin sat in namespace `api` while all five routes
that annotate it live in `llm-serving`. The ingress controller resolves
konghq.com/plugins against the annotated object's own namespace, so the
reference dangled and the plugin never bound. Verified before the fix:
unauthenticated GET /v1/models and POST /v1/ornith/chat/completions both
returned 200. A dangling plugin reference fails open and logs nothing.
Re-test both without a key after this syncs; expect 401.
Note for follow-up: llm-embeddings and llm-score carry no plugins annotation at
all, so they stay unauthenticated even after this change.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
a118195b6f
commit
826a2acda3
@@ -57,6 +57,17 @@ env:
|
||||
# nginx Ingress in ingress.yaml; both hops have to be unbuffered or the
|
||||
# buffered one dominates.
|
||||
nginx_proxy_proxy_buffering: "off"
|
||||
# Any plugin that rewrites the request body — request-transformer on the
|
||||
# llm-chat-* routes — reads it through `kong.request.get_body()`, and that
|
||||
# returns nothing once nginx has spilled the body past
|
||||
# client_body_buffer_size into a temp file. The plugin then re-serializes a
|
||||
# body with no `messages`, and the upstream answers
|
||||
# HTTP 400 {"error":{"message":"[] is too short - 'messages'"}}
|
||||
# Measured on /v1/ornith/chat/completions: 10588 B -> 200, 11088 B -> 400.
|
||||
# An agent request carrying tool schemas clears that in one turn, so the
|
||||
# buffer has to hold a whole conversation, not a chat message.
|
||||
nginx_http_client_body_buffer_size: "16m"
|
||||
nginx_http_client_max_body_size: "16m"
|
||||
|
||||
ingressController:
|
||||
enabled: true
|
||||
|
||||
Reference in New Issue
Block a user