Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
1.6 KiB
1.6 KiB
4.2 — Per-caller request budgets (GREEN)
Phase: 4 — Limits and budgets Stage: GREEN Depends on: 4.3
No rate-limiting plugin exists anywhere in the cluster today. This is net-new work,
not a migration — there is no prior behaviour to preserve.
- An identified caller gets a bounded number of requests per configured time window
- Caller identity comes from the authenticated token's subject when auth is on, and from a documented fallback attribute when auth is off
- Budget size and window length are explicit in configuration, per caller class, with no silent defaults
- Exceeding the budget is rejected with a retryable status, an
application/problem+jsonbody, and aRetry-Afternaming when the window resets - Remaining budget and reset time are observable to the caller on allowed requests, not only on rejections
- Budgets are enforced independently of the
reasoningconcurrency cap — a caller under budget can still be queued or rejected for slot pressure, and vice versa - Two distinct callers do not consume each other's budget
- Budget state is per-replica-safe: the documented behaviour with 2 replicas is stated, not accidental
Verify
# budget=5 per 60s window for the test caller
for i in $(seq 1 7); do
curl -s -o /dev/null -w '%{http_code} ' -H 'authorization: Bearer test-caller-a' \
localhost:8080/v1/models
done; echo
# expected: 200 200 200 200 200 429 429
curl -s -o /dev/null -w '%{http_code}\n' -H 'authorization: Bearer test-caller-b' \
localhost:8080/v1/models
# expected: 200 — caller B has its own budget