feat: phase 8 serviceadapter crd rollout (32/33 tasks)
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
# 0.2 — Declarative route configuration (RED)
|
||||
# 0.2 — Declarative route configuration (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: RED
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 1.2 — Streaming passthrough (RED)
|
||||
# 1.2 — Streaming passthrough (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: RED
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 1.3 — Client disconnect propagation (RED)
|
||||
# 1.3 — Client disconnect propagation (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: RED
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 1.7 — Per-route body size caps (RED)
|
||||
# 1.7 — Per-route body size caps (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: RED
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
# 2.1 — Model registry (GREEN)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: GREEN
|
||||
Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] A model name -> upstream map is loaded from configuration at startup, never compiled in
|
||||
- [ ] Each entry carries at minimum the model name clients send, the upstream address, and the upstream path to use
|
||||
- [ ] Two model names may point at the same upstream address, and both resolve independently
|
||||
- [ ] A duplicate model name, an empty model name, or an entry with no upstream address fails startup loudly with the offending entry named
|
||||
- [ ] The registry is queryable by exact model name; lookup is case-sensitive and does no fuzzy matching or defaulting
|
||||
- [ ] The set of known model names is enumerable, because `/v1/models` and unknown-model errors are both derived from it
|
||||
|
||||
The five entries verified live on 2026-08-19. Ports are 80, not 8080.
|
||||
|
||||
| model name clients send | upstream Service | engine |
|
||||
|---|---|---|
|
||||
| `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B |
|
||||
| `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama |
|
||||
| `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama, same pods |
|
||||
| `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI |
|
||||
| `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI |
|
||||
|
||||
`ornith:35b` and `qwen2.5:3b-instruct` share pods and both stay resident, so a
|
||||
registry that maps them to one address is correct, not a shortcut.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# with the five entries configured against local stubs
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 200
|
||||
|
||||
# duplicate model name in config must refuse to start
|
||||
./gateway --config testdata/duplicate-model.yaml; echo "exit=$?" # expected: non-zero exit, stderr names the duplicated model
|
||||
```
|
||||
@@ -1,53 +0,0 @@
|
||||
# 2.10 — Anthropic request translation (GREEN)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: GREEN
|
||||
Depends on: [2.9](2.9-canonical-request-model.md), [2.1](2.1-model-registry.md)
|
||||
|
||||
`POST /llm/v1/messages` accepts an Anthropic Messages request body and turns it into
|
||||
the dialect-neutral canonical request. The path carries `/v1/messages` because the
|
||||
Anthropic base-URL convention appends that suffix; the gateway prefix is `/llm`.
|
||||
|
||||
- [ ] `POST /llm/v1/messages` is accepted and selects its upstream from the body's
|
||||
`model` field, using the same registry as `/v1`: `reasoning` reaches
|
||||
`reasoning-predictor.llm-serving:80`, `ornith:35b` and `qwen2.5:3b-instruct`
|
||||
reach `ornith-predictor.llm-serving:80`
|
||||
- [ ] The top-level `system` field becomes the canonical system instruction; it is a
|
||||
distinct field in this dialect and is not a member of `messages`
|
||||
- [ ] Each message `content` may be a plain string or an array of blocks; a string and
|
||||
a single text block carrying the same characters translate identically
|
||||
- [ ] `max_tokens` is REQUIRED on this surface, matching the Anthropic contract; a
|
||||
request without it is rejected as a client error and no upstream is contacted
|
||||
- [ ] `max_tokens` above what the upstream can serve is clamped rather than rejected,
|
||||
and the clamp is logged; `reasoning` has `--max-model-len=16384`
|
||||
- [ ] Roles are restricted to `user` and `assistant`; any other role, including
|
||||
`system` inside `messages`, is a client error naming the offending role
|
||||
- [ ] `stop_sequences` becomes the canonical stop sequences, and `stream` becomes the
|
||||
canonical streaming flag
|
||||
- [ ] `tools`, `tool_choice`, any `tool_use` or `tool_result` block, any `image` block,
|
||||
any cache-control marker, and any user message with more than one content block
|
||||
are rejected as unsupported, naming the feature; none is silently dropped
|
||||
- [ ] Unknown top-level fields are rejected rather than ignored, so a client cannot
|
||||
believe an unimplemented option took effect
|
||||
- [ ] Reading the body respects the route's configured body size cap
|
||||
|
||||
`max_tokens` stays optional on `/v1/chat/completions`. The policy is per-surface: each
|
||||
dialect keeps its own contract, and the canonical request records whichever value
|
||||
resulted.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":64,"system":"be terse","messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, reasoning stub hit, system text present in the upstream request
|
||||
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 400, body names max_tokens as the missing required field, no upstream stub hit
|
||||
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"system","content":"x"}]}'
|
||||
# expected: 400, body names the rejected role, no upstream stub hit
|
||||
```
|
||||
@@ -1,50 +0,0 @@
|
||||
# 2.11 — Anthropic non-streaming response translation (GREEN)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: GREEN
|
||||
Depends on: [2.10](2.10-anthropic-request-translation.md), [2.9](2.9-canonical-request-model.md)
|
||||
|
||||
A non-streaming `POST /llm/v1/messages` gets an Anthropic Messages response, built
|
||||
from whatever the upstream returned. Upstreams speak the OpenAI chat-completion shape;
|
||||
the client on this surface must never see it.
|
||||
|
||||
- [ ] The response body is `{"id","type":"message","role":"assistant","content":[...],
|
||||
"model","stop_reason","stop_sequence","usage":{"input_tokens","output_tokens"}}`
|
||||
with `type` literally `message` and `role` literally `assistant`
|
||||
- [ ] `model` echoes the model name the client sent, not an upstream-internal name
|
||||
- [ ] Upstream `finish_reason` `stop` becomes `stop_reason` `end_turn`, and `length`
|
||||
becomes `max_tokens`
|
||||
- [ ] A generation halted by a client-supplied stop sequence reports `stop_reason`
|
||||
`stop_sequence` and puts the matched string in `stop_sequence`; otherwise
|
||||
`stop_sequence` is null and present, not omitted
|
||||
- [ ] Upstream `prompt_tokens` becomes `usage.input_tokens` and `completion_tokens`
|
||||
becomes `usage.output_tokens`; no other usage fields are invented
|
||||
- [ ] `reasoning_content`, which vLLM returns as a field separate from `content` for
|
||||
`reasoning`, becomes a `thinking` content block that PRECEDES the `text` block
|
||||
- [ ] When `reasoning_content` is absent or empty, no `thinking` block is emitted and
|
||||
`content` holds only the `text` block
|
||||
- [ ] When `content` is empty but `reasoning_content` is not, the `thinking` block is
|
||||
still returned rather than an empty `content` array
|
||||
- [ ] `Content-Type` is `application/json`, and no OpenAI field name such as `choices`,
|
||||
`finish_reason` or `object` appears anywhere in the body
|
||||
|
||||
`reasoning` runs DeepSeek-R1-Distill-Qwen-32B under vLLM with
|
||||
`--reasoning-parser=deepseek_r1`, which is why the reasoning text arrives as its own
|
||||
field and maps cleanly onto a thinking block.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, type=message, role=assistant, content[0].type=thinking, content[1].type=text
|
||||
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
| grep -c -E '"choices"|"finish_reason"|"object"'
|
||||
# expected: 0
|
||||
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"qwen2.5:3b-instruct","max_tokens":4,"messages":[{"role":"user","content":"count to fifty"}]}'
|
||||
# expected: 200, stop_reason=max_tokens, usage has input_tokens and output_tokens only
|
||||
```
|
||||
@@ -1,64 +0,0 @@
|
||||
# 2.12 — Anthropic SSE state machine (RED)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: RED
|
||||
Depends on: [2.11](2.11-anthropic-response-translation.md), [1.2](1.2-streaming-passthrough.md), [1.3](1.3-disconnect-propagation.md)
|
||||
|
||||
`POST /llm/v1/messages` with `"stream":true` must emit Anthropic SSE. Anthropic uses
|
||||
NAMED events carrying content-block indices; the upstream emits flat OpenAI data-only
|
||||
chunks. Write the failing tests against the event sequence before writing a translator.
|
||||
|
||||
- [ ] Every frame has both an `event:` line and a `data:` line; a bare `data:` frame is
|
||||
a failure on this surface
|
||||
- [ ] `Content-Type` is `text/event-stream`
|
||||
- [ ] Event order for a full response is exactly: `message_start`, then for each block
|
||||
`content_block_start`, one or more `content_block_delta`, `content_block_stop`,
|
||||
then `message_delta`, then `message_stop`
|
||||
- [ ] `message_start` carries the message envelope with the client-sent model, `role`
|
||||
`assistant`, empty `content`, and `usage.input_tokens`
|
||||
- [ ] The block carrying `reasoning_content` is index 0 with block type `thinking`, and
|
||||
its deltas are `thinking_delta`
|
||||
- [ ] The block carrying `content` is index 1 with block type `text`, and its deltas
|
||||
are `text_delta`
|
||||
- [ ] The end of reasoning is only knowable when `content` first arrives, so the
|
||||
arrival of the first `content` token MUST emit `content_block_stop` for index 0
|
||||
before `content_block_start` for index 1 — the two blocks never overlap
|
||||
- [ ] If a response has no `reasoning_content` at all, the text block is index 0 and no
|
||||
thinking block is started; indices are assigned in emission order with no gaps
|
||||
- [ ] If a response has `reasoning_content` and never any `content`, the thinking block
|
||||
is still closed before `message_delta`
|
||||
- [ ] `message_delta` carries `stop_reason` and `usage.output_tokens`; upstream
|
||||
`finish_reason` `stop` becomes `end_turn` and `length` becomes `max_tokens`
|
||||
- [ ] `message_stop` is the final frame and is emitted exactly once per response
|
||||
- [ ] Translation is streaming and unbuffered: each upstream chunk is converted and
|
||||
flushed as it arrives, and the response is never accumulated to be inspected
|
||||
- [ ] A client disconnect mid-stream cancels the upstream request immediately and
|
||||
releases the slot, rather than orphaning the generation
|
||||
- [ ] An upstream failure after `message_start` terminates the stream with an error
|
||||
frame rather than a truncated but apparently successful sequence
|
||||
- [ ] The OpenAI `data: [DONE]` sentinel is consumed by the translator and never
|
||||
forwarded to a `/llm` client
|
||||
|
||||
An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
|
||||
which is why disconnect cancellation is an acceptance criterion here and not only in
|
||||
the proxy layer.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
| grep '^event:'
|
||||
# expected: message_start, content_block_start, content_block_delta..., content_block_stop,
|
||||
# content_block_start, content_block_delta..., content_block_stop, message_delta, message_stop
|
||||
|
||||
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
| grep -n -E 'content_block_stop|"index":1' | head -3
|
||||
# expected: the index 0 content_block_stop line precedes the first line mentioning index 1
|
||||
|
||||
timeout 1 curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":512,"stream":true,"messages":[{"role":"user","content":"long"}]}' >/dev/null
|
||||
grep -c 'cancelled' /tmp/stub-reasoning.log
|
||||
# expected: 1 within a second of the client going away
|
||||
```
|
||||
@@ -1,50 +0,0 @@
|
||||
# 2.13 — Anthropic error shape on `/llm/*` (RED)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: RED
|
||||
Depends on: [2.10](2.10-anthropic-request-translation.md), [4.3](4.3-problem-json-errors.md)
|
||||
|
||||
`/v1/*` renders rejections as RFC 9457 `application/problem+json`. `/llm/*` must not.
|
||||
The same underlying rejection gets two renderings, chosen purely by which surface
|
||||
received the request. Write the failing tests against both renderings first.
|
||||
|
||||
- [ ] Every error response from a `/llm/*` path has the body
|
||||
`{"type":"error","error":{"type":"...","message":"..."}}` and
|
||||
`Content-Type: application/json`
|
||||
- [ ] No `/llm/*` response ever carries `application/problem+json`, and no `/v1/*`
|
||||
response ever carries the Anthropic error shape
|
||||
- [ ] An unknown or missing `model` returns `invalid_request_error` with a message
|
||||
naming the rejected value and listing the configured model names, derived from
|
||||
the registry rather than hardcoded
|
||||
- [ ] A request missing the required `max_tokens` returns `invalid_request_error`
|
||||
naming `max_tokens`
|
||||
- [ ] A body that is not valid JSON returns `invalid_request_error` with a message
|
||||
distinguishable from the unknown-model case
|
||||
- [ ] A request for an out-of-scope feature returns `invalid_request_error` naming the
|
||||
feature, for example tools, images or prompt caching
|
||||
- [ ] Exhausting the shared slot queue returns HTTP 429 with error type
|
||||
`rate_limit_error` and a `Retry-After` header
|
||||
- [ ] An upstream failure or timeout returns HTTP 5xx with error type `api_error`, and
|
||||
the message leaks no upstream host, port or internal path
|
||||
- [ ] No rejection reaches an upstream, and none falls back to a default model
|
||||
- [ ] Each rejection is logged with its reason and its surface; the request body is
|
||||
never logged
|
||||
- [ ] The same malformed request sent to `/v1/chat/completions` and `/llm/v1/messages`
|
||||
yields the same HTTP status with the two different body shapes
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -i localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"gpt-4","max_tokens":8,"messages":[]}'
|
||||
# expected: 4xx, content-type application/json, body {"type":"error","error":{"type":"invalid_request_error",...}}
|
||||
# listing reasoning, ornith:35b, qwen2.5:3b-instruct
|
||||
|
||||
curl -s -i localhost:8080/llm/v1/messages -H 'content-type: application/json' -d 'not json' \
|
||||
| grep -i 'content-type'
|
||||
# expected: application/json, never application/problem+json
|
||||
|
||||
curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' \
|
||||
-d '{"model":"gpt-4","messages":[]}' | grep -i 'content-type'
|
||||
# expected: application/problem+json, confirming the two surfaces render differently
|
||||
```
|
||||
@@ -1,52 +0,0 @@
|
||||
# 2.14 — Queue position event on `/llm/*` (GREEN)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: GREEN
|
||||
Depends on: [2.12](2.12-anthropic-sse-state-machine.md), [4.1](4.1-gpu-slot-semaphore.md)
|
||||
|
||||
The Anthropic event set has no way to say "you are queued". Its stream implicitly
|
||||
begins after a slot has been acquired, so a queued client sees nothing at all until
|
||||
generation starts. The `reasoning` upstream has only 8 sequence slots cluster-wide and
|
||||
the gateway caps below that, so waiting is normal and worth showing.
|
||||
|
||||
- [ ] When a streaming `/llm/v1/messages` request waits for a slot, the gateway emits a
|
||||
frame with `event: queue` before any `message_start`
|
||||
- [ ] The queue frame's data carries the caller's current position in the queue
|
||||
- [ ] Position updates are emitted as the queue drains, each as another `event: queue`
|
||||
frame, until a slot is acquired
|
||||
- [ ] Once a slot is acquired the stream continues with the standard sequence beginning
|
||||
at `message_start`, and no further queue frame is emitted for that request
|
||||
- [ ] A request that acquires a slot immediately emits no queue frame at all
|
||||
- [ ] Queue frames are flushed as they are produced, not buffered behind the first
|
||||
upstream token
|
||||
- [ ] A client that disconnects while still queued is removed from the queue, never
|
||||
reaches the upstream and never consumes a slot
|
||||
- [ ] Non-streaming requests emit no queue frames; they simply wait, then answer
|
||||
- [ ] The extension is documented in the surface's own docs as non-standard, alongside
|
||||
the fact that a strict Anthropic client ignoring unknown events degrades to
|
||||
showing nothing while queued rather than erroring
|
||||
|
||||
This is a deliberate departure from the Anthropic contract. It is safe only because
|
||||
the sole client of `/llm/*` is the first-party riotpiao frontend. It must never be
|
||||
required for correctness: dropping every `event: queue` frame leaves a valid,
|
||||
complete Anthropic stream.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# Fill the slots first, with cap=2 configured against a stub that holds each request 3s.
|
||||
seq 4 | xargs -P4 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' &
|
||||
|
||||
sleep 1
|
||||
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
| grep -m3 '^event:'
|
||||
# expected: event: queue arrives within a second, before any event: message_start
|
||||
|
||||
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"qwen2.5:3b-instruct","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
|
||||
| head -1
|
||||
# expected: event: message_start, no queue frame on an uncontended upstream
|
||||
```
|
||||
@@ -1,51 +0,0 @@
|
||||
# 2.15 — Anthropic dialect scope boundary (GREEN)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: GREEN
|
||||
Depends on: [2.10](2.10-anthropic-request-translation.md), [2.13](2.13-anthropic-error-shape.md)
|
||||
|
||||
The only client of `/llm/*` is the first-party riotpiao frontend. It is not Claude
|
||||
Code and not the Anthropic SDK, so the surface implements a deliberately narrow slice
|
||||
of the Messages API. The narrowness is the design; the risk is a half-built feature
|
||||
that appears to work.
|
||||
|
||||
- [ ] The unsupported set is enumerated in one place and covers at least: `tools` and
|
||||
`tool_choice`, `tool_use` blocks, `tool_result` turns, image content blocks,
|
||||
prompt-caching controls and cache headers, the batch API, and user messages
|
||||
carrying more than one content block
|
||||
- [ ] Each unsupported feature is detected during request translation, before any
|
||||
upstream is contacted
|
||||
- [ ] Rejection uses the Anthropic error shape
|
||||
`{"type":"error","error":{"type":"invalid_request_error","message":"..."}}` and
|
||||
the message names the specific unsupported feature, not just "unsupported"
|
||||
- [ ] No unsupported feature is silently ignored, stripped, or partially honoured; a
|
||||
request containing one never produces a 200
|
||||
- [ ] A request combining a supported and an unsupported field is rejected, not
|
||||
serviced with the unsupported part dropped
|
||||
- [ ] The `/v1/*` OpenAI surface is unaffected: tool calling continues to work there
|
||||
exactly as it does today
|
||||
- [ ] Every entry in the unsupported set has a test asserting the rejection, so
|
||||
widening scope forces a deliberate test change rather than a quiet code change
|
||||
- [ ] The list is documented on the surface, so the frontend can see the boundary
|
||||
without reading gateway code
|
||||
|
||||
Enabling any of these later must be an explicit config or code change accompanied by
|
||||
its own translation work and tests. A silent partial implementation is the specific
|
||||
failure this task exists to prevent.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":8,"tools":[{"name":"x"}],"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 4xx, error.message names tools, no upstream stub hit
|
||||
|
||||
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"user","content":[{"type":"image","source":{}}]}]}'
|
||||
# expected: 4xx, error.message names image content blocks, no upstream stub hit
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}],"tools":[{"type":"function","function":{"name":"x"}}]}'
|
||||
# expected: 200, tool calling still works on the OpenAI surface
|
||||
```
|
||||
@@ -1,39 +0,0 @@
|
||||
# 2.2 — Body-based model dispatch (GREEN)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: GREEN
|
||||
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md), [1.2](1.2-streaming-passthrough.md)
|
||||
|
||||
- [ ] `POST /v1/chat/completions` selects its upstream from the `model` field of the JSON request body
|
||||
- [ ] The body reaching the upstream is byte-identical to the body received, `model` included — dispatch reads, it does not rewrite
|
||||
- [ ] The upstream sees the canonical path `/v1/chat/completions`
|
||||
- [ ] `"model":"reasoning"` reaches `reasoning-predictor.llm-serving:80`; `"model":"ornith:35b"` and `"model":"qwen2.5:3b-instruct"` both reach `ornith-predictor.llm-serving:80`
|
||||
- [ ] `"stream":true` streams unbuffered — chunks reach the client as the upstream emits them, and are not accumulated in order to inspect the body
|
||||
- [ ] A client disconnect mid-stream cancels the upstream request rather than orphaning it
|
||||
- [ ] `reasoning_content` is passed through untouched alongside `content`; the gateway does not merge, reorder or strip either
|
||||
- [ ] Reading the body to find `model` respects the route's body size cap and does not load an unbounded request into memory
|
||||
|
||||
This is the single capability Kong OSS lacked — `ai-proxy-advanced` is Enterprise-only —
|
||||
and the entire reason this gateway exists. Everything else in this phase is a
|
||||
consequence of it. Ports are 80, not 8080.
|
||||
|
||||
An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
|
||||
which is why disconnect cancellation belongs in the acceptance criteria of this task
|
||||
and not only in the proxy layer.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, stub for reasoning-predictor recorded the hit with path /v1/chat/completions and an unmodified body
|
||||
|
||||
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
|
||||
-d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, ornith-predictor stub hit, reasoning-predictor stub not hit
|
||||
|
||||
curl -N -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, content-type text/event-stream, first data: chunk arrives before the stub finishes emitting
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# 2.3 — Unknown and missing model errors (RED)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: RED
|
||||
Depends on: [2.1](2.1-model-registry.md), [2.2](2.2-body-based-dispatch.md)
|
||||
|
||||
- [ ] `POST /v1/chat/completions` with a `model` that is not in the configured registry returns a 4xx client error, never 5xx
|
||||
- [ ] A request with no `model` field, a null `model`, or an empty-string `model` is the same class of client error
|
||||
- [ ] A body that is not valid JSON is also a client error, distinguishable from an unknown model
|
||||
- [ ] The response is RFC 9457 with `Content-Type: application/problem+json`
|
||||
- [ ] The problem body names the rejected value and enumerates every currently configured model name, derived from the registry rather than written out by hand
|
||||
- [ ] No fallback to a default model happens under any of these conditions, and no upstream is contacted
|
||||
- [ ] The rejection is logged with its reason; the request body is not logged
|
||||
|
||||
Write the failing tests first. A silent fallback to a default model is the specific
|
||||
failure mode this task exists to prevent: it turns a client typo into a bill against
|
||||
the wrong GPU and hides the mistake from the caller.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' \
|
||||
-d '{"model":"gpt-4","messages":[]}'
|
||||
# expected: 400 (or 404), content-type: application/problem+json, body lists reasoning, ornith:35b, qwen2.5:3b-instruct, and the embedding/rerank models
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"messages":[]}'
|
||||
# expected: 4xx, no upstream stub recorded any hit
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d 'not json'
|
||||
# expected: 400, problem+json, reason distinct from unknown-model
|
||||
```
|
||||
@@ -1,30 +0,0 @@
|
||||
# 2.4 — Legacy path aliases (GREEN)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: GREEN
|
||||
Depends on: [2.2](2.2-body-based-dispatch.md)
|
||||
|
||||
- [ ] `POST /v1/reasoning/chat/completions` behaves as the canonical endpoint with `model` forced to `reasoning`
|
||||
- [ ] `POST /v1/ornith/chat/completions` behaves as the canonical endpoint with `model` forced to `ornith:35b`
|
||||
- [ ] `POST /v1/qwen/chat/completions` behaves as the canonical endpoint with `model` forced to `qwen2.5:3b-instruct`
|
||||
- [ ] The forced value overrides whatever `model` the body carries, including a conflicting one, and the upstream receives the forced value
|
||||
- [ ] The upstream sees the canonical path `/v1/chat/completions`, not the alias path
|
||||
- [ ] Streaming, disconnect cancellation and `reasoning_content` passthrough behave identically to the canonical endpoint
|
||||
- [ ] The alias set is configuration, so removal is a config change and needs no code change
|
||||
- [ ] Each alias is marked temporary where it is defined, with the condition for removal stated: all callers migrated
|
||||
|
||||
These three paths exist only because Kong OSS could not dispatch on the request body.
|
||||
They keep the cutover reversible — pi is a live caller today. They are deleted once
|
||||
callers have moved to `POST /v1/chat/completions` with `model` in the body.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/reasoning/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, reasoning-predictor stub hit at /v1/chat/completions with body model=reasoning
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/qwen/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}'
|
||||
# expected: 200, ornith-predictor stub hit with body model=qwen2.5:3b-instruct — the body's "reasoning" is overridden
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# 2.5 — `GET /v1/models` (GREEN)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: GREEN
|
||||
Depends on: [2.1](2.1-model-registry.md)
|
||||
|
||||
- [ ] `GET /v1/models` returns 200 with `Content-Type: application/json`
|
||||
- [ ] The response shape is `{"object":"list","data":[{"id","object":"model","owned_by","created"}]}`
|
||||
- [ ] Every entry's `object` is the literal string `model`
|
||||
- [ ] The `id` values are exactly the model names the configured registry will accept for dispatch — no more, no fewer
|
||||
- [ ] Adding or removing a model in configuration changes this response with no code change
|
||||
- [ ] No model list is hardcoded anywhere; the list cannot disagree with what routing accepts
|
||||
- [ ] The endpoint contacts no upstream and stays cheap
|
||||
|
||||
Kong served a static list via `request-termination`, and its own manifest flags that
|
||||
the list can drift from what the engines actually serve. Deriving from the registry
|
||||
makes that drift structurally impossible: the same source answers this endpoint and
|
||||
decides which `model` values dispatch.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s localhost:8080/v1/models
|
||||
# expected: 200, {"object":"list","data":[...]} with ids reasoning, ornith:35b, qwen2.5:3b-instruct,
|
||||
# nomic-ai/nomic-embed-text-v2-moe, BAAI/bge-reranker-base
|
||||
|
||||
# every advertised id must dispatch; nothing advertised may 404
|
||||
for m in $(curl -s localhost:8080/v1/models | grep -o '"id":"[^"]*"' | cut -d'"' -f4); do
|
||||
curl -s -o /dev/null -w "$m %{http_code}\n" localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d "{\"model\":\"$m\",\"messages\":[]}"
|
||||
done
|
||||
# expected: no unknown-model rejection for any advertised id
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# 2.6 — `POST /v1/embeddings` passthrough (GREEN)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: GREEN
|
||||
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] `POST /v1/embeddings` reaches `embeddings-predictor.llm-serving:80` at the path `/v1/embeddings`, unrewritten
|
||||
- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
|
||||
- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration rather than inherited
|
||||
- [ ] The route has an explicit body size cap
|
||||
- [ ] Upstream error statuses are surfaced as-is; the gateway invents no retries and no substitute response
|
||||
|
||||
TEI already serves the canonical path, so this route rewrites nothing. The model
|
||||
served here is `nomic-ai/nomic-embed-text-v2-moe`. Port is 80, not 8080. Whether this
|
||||
route dispatches on the body's `model` or is pinned to the single embeddings upstream
|
||||
is a design decision for whoever works it; either way the path must not change and an
|
||||
unknown `model` must not silently reach the wrong upstream.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -i localhost:8080/v1/embeddings -H 'content-type: application/json' \
|
||||
-d '{"model":"nomic-ai/nomic-embed-text-v2-moe","input":"hello"}'
|
||||
# expected: 200, embeddings stub hit at path /v1/embeddings, request body unmodified, response body byte-identical to the stub's
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# 2.7 — `POST /v1/rerank` path rewrite (GREEN)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: GREEN
|
||||
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] `POST /v1/rerank` reaches `reranker-predictor.llm-serving:80` at the path `/rerank`
|
||||
- [ ] The rewrite is expressed in the route configuration, not special-cased in dispatch code
|
||||
- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
|
||||
- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration
|
||||
- [ ] The route has an explicit body size cap
|
||||
- [ ] A request to `/v1/rerank` never reaches the upstream as `/v1/rerank`
|
||||
|
||||
TEI does not serve `/v1/rerank`: probing it returned 404, while `/rerank` returned 405
|
||||
for the wrong method — that is how the correct upstream path was established. The
|
||||
model served here is `BAAI/bge-reranker-base`. Port is 80, not 8080. This is the only
|
||||
route in the LLM surface that rewrites its path.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -i localhost:8080/v1/rerank -H 'content-type: application/json' \
|
||||
-d '{"model":"BAAI/bge-reranker-base","query":"q","texts":["a","b"]}'
|
||||
# expected: 200, reranker stub recorded exactly one hit at path /rerank and zero at /v1/rerank
|
||||
```
|
||||
@@ -1,31 +0,0 @@
|
||||
# 2.8 — Kong parity test (RED)
|
||||
|
||||
Phase: 2 — LLM surface
|
||||
Stage: RED
|
||||
Depends on: [2.2](2.2-body-based-dispatch.md), [2.4](2.4-legacy-path-aliases.md), [2.5](2.5-models-endpoint.md), [2.6](2.6-embeddings-passthrough.md), [2.7](2.7-rerank-rewrite.md)
|
||||
|
||||
- [ ] A repeatable check issues the same request to Kong and to the gateway and compares the real HTTP responses: status, meaningful headers, and body
|
||||
- [ ] It covers every migrated route: `GET /v1/models`, the three `/v1/{reasoning,ornith,qwen}/chat/completions` aliases, `POST /v1/embeddings`, `POST /v1/rerank`, and the new canonical `POST /v1/chat/completions`
|
||||
- [ ] It covers a streaming chat request and asserts equivalent chunk sequencing, not just an equal final concatenation
|
||||
- [ ] It covers a mid-stream client disconnect and asserts the upstream request is cancelled rather than left running
|
||||
- [ ] It asserts `reasoning_content` and `content` are both present and untouched for the `reasoning` model
|
||||
- [ ] Fields that legitimately differ per request — ids, timestamps, generated text — are normalised before comparison, and the normalisation is explicit rather than a blanket ignore
|
||||
- [ ] Any difference fails the check loudly and names the route and the field
|
||||
- [ ] `GET /v1/models` is expected to differ from Kong's static list only where the gateway's derived list is more accurate; that difference is recorded deliberately, not normalised away
|
||||
- [ ] The check passes before cutover ([6.5](6.5-cutover.md)) and blocks it if it does not
|
||||
|
||||
Kong is serving live traffic while this runs. The check reads only; it changes no
|
||||
cluster state. Eleven ReplicaSets exist on the Kong Deployment, so re-run this
|
||||
immediately before cutover rather than trusting an old result.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
./parity --kong https://api.riotpiao.com --gateway http://homelab-frontend.api.svc.cluster.local
|
||||
# expected: exit 0, one PASS line per route, explicit report of the /v1/models difference
|
||||
|
||||
# spot-check by hand, same body to both
|
||||
curl -s -o /dev/null -w 'kong %{http_code}\n' https://api.riotpiao.com/v1/rerank \
|
||||
-H 'content-type: application/json' -d '{"model":"BAAI/bge-reranker-base","query":"q","texts":["a"]}'
|
||||
# expected: identical status from both endpoints
|
||||
```
|
||||
@@ -1,48 +0,0 @@
|
||||
# 2.9 — Dialect-neutral canonical request (GREEN)
|
||||
|
||||
Phase: 2 — LLM surfaces
|
||||
Stage: GREEN
|
||||
Depends on: [2.1](2.1-model-registry.md), [2.2](2.2-body-based-dispatch.md)
|
||||
|
||||
The gateway serves two permanent protocol dialects: `/v1/*` is OpenAI-compatible and
|
||||
`/llm/*` is Anthropic Messages. Both translate into one internal shape before anything
|
||||
else happens to them, and both translate back out of it on the way to the client.
|
||||
|
||||
- [ ] A single internal request representation exists that carries at minimum: the
|
||||
resolved upstream, the model name as the client sent it, the ordered turns, an
|
||||
optional system instruction, a maximum output token count, stop sequences, and a
|
||||
streaming flag
|
||||
- [ ] The representation names no dialect: nothing in it is called openai or anthropic,
|
||||
and no field exists solely because one dialect happens to spell it that way
|
||||
- [ ] Model dispatch, the shared slot controller, per-caller budgets, logging and
|
||||
metrics all read the canonical request and never the raw dialect body
|
||||
- [ ] The surface that received a request is recorded as one field on the canonical
|
||||
request, used only to choose the response and error rendering, never to choose an
|
||||
upstream or a slot
|
||||
- [ ] An identical prompt sent to `/v1/chat/completions` and to `/llm/v1/messages`
|
||||
produces the same resolved upstream, the same slot accounting and the same log
|
||||
fields apart from that one surface label
|
||||
- [ ] Adding a third dialect requires a new translator only; the slot controller,
|
||||
dispatch and budget code are untouched
|
||||
- [ ] Translation failure is a client error at the surface boundary, and no partially
|
||||
populated canonical request ever reaches an upstream
|
||||
|
||||
`reasoning` maps to `reasoning-predictor.llm-serving:80`; `ornith:35b` and
|
||||
`qwen2.5:3b-instruct` both map to `ornith-predictor.llm-serving:80`. Ports are 80.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, reasoning stub hit once
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200, reasoning stub hit once, same upstream and same slot counter as the /v1 call
|
||||
|
||||
grep -h 'upstream=' /tmp/gateway.log | tail -2
|
||||
# expected: both lines show upstream=reasoning-predictor and model=reasoning, differing only in the surface label
|
||||
```
|
||||
@@ -1,34 +0,0 @@
|
||||
# 3.1 — JWKS fetch and rotation (GREEN)
|
||||
|
||||
Phase: 3 — Authentication
|
||||
Stage: GREEN
|
||||
Depends on: [0.2](0.2-route-configuration.md), [0.3](0.3-health-endpoints.md)
|
||||
|
||||
- [ ] The signing key set is fetched from Authentik at `https://authentik.riotpiao.com` at runtime; no public key is pinned in configuration, in an image, or in a manifest
|
||||
- [ ] The JWKS URL is configuration, so a local stub issuer can be pointed at instead — no cluster and no credentials needed to verify this task
|
||||
- [ ] Fetched keys are cached and reused; a token verification does not trigger a network call per request
|
||||
- [ ] A token whose key id is not in the cache triggers a refetch, and the refetch is rate-limited so an unknown-key flood cannot hammer Authentik
|
||||
- [ ] After a key rotates at the issuer, tokens signed by the new key verify without restarting, redeploying, or editing configuration
|
||||
- [ ] Tokens signed by a key that is no longer published stop verifying once the cache reflects that
|
||||
- [ ] `GET /readyz` fails while JWKS has never been fetched successfully, and `GET /healthz` is unaffected
|
||||
- [ ] A JWKS fetch failure while a valid cache exists does not take the gateway down; it is logged and retried
|
||||
- [ ] No key material, token, or fetched secret appears in logs
|
||||
|
||||
This deletes the rotation runbook `AUTH-PLAN.md` was forced to propose. That runbook
|
||||
existed only because Kong OSS needed a pinned `rsa_public_key`; it must not be
|
||||
carried forward.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# stub issuer serving a JWKS, gateway pointed at it
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 200 once JWKS has been fetched
|
||||
|
||||
# stub returns 500 for JWKS from a cold start
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 503
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz # expected: 200
|
||||
|
||||
# rotate the stub's key, then present a token signed by the new key, no restart
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models -H "authorization: Bearer $NEW_TOKEN"
|
||||
# expected: 200 — refetch happened on the unknown key id
|
||||
```
|
||||
@@ -1,37 +0,0 @@
|
||||
# 3.2 — Bearer token validation (GREEN)
|
||||
|
||||
Phase: 3 — Authentication
|
||||
Stage: GREEN
|
||||
Depends on: [3.1](3.1-jwks-fetch-and-rotation.md)
|
||||
|
||||
- [ ] `Authorization: Bearer <jwt>` is the credential the gateway accepts on protected routes
|
||||
- [ ] The scheme match is case-insensitive, as the HTTP spec requires — `bearer`, `Bearer` and `BEARER` all work
|
||||
- [ ] A valid, unexpired token signed by a currently published Authentik key returns the upstream response
|
||||
- [ ] Missing header, wrong scheme, malformed token, bad signature, expired token, and wrong issuer or audience each return 401 with `WWW-Authenticate` set
|
||||
- [ ] Rejections are RFC 9457 `application/problem+json`, and the reason is logged without logging the token
|
||||
- [ ] The `Authorization` header is not forwarded to upstreams
|
||||
- [ ] The validated caller identity is available to later stages, since [3.5](3.5-capability-authorization.md) authorizes on it
|
||||
- [ ] Signature verification is real: a token with a valid-looking payload and a forged signature is rejected
|
||||
|
||||
This is precisely what Kong OSS `key-auth` could not do. Verified live: a raw
|
||||
`apikey:` header succeeded with 200 while `Authorization: Bearer` failed with 401,
|
||||
which hard-blocked every OpenAI-compatible client and is why authentication is OFF
|
||||
on the model API today.
|
||||
|
||||
Authentik is at `https://authentik.riotpiao.com`. A local stub issuer must be enough
|
||||
to work this task — no cluster, no credentials.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H "authorization: Bearer $VALID_TOKEN" -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[]}'
|
||||
# expected: 200
|
||||
|
||||
curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{"model":"reasoning"}'
|
||||
# expected: 401, WWW-Authenticate present, content-type application/problem+json
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models -H "apikey: $OLD_KONG_KEY"
|
||||
# expected: 401 — the retired Kong credential form is not accepted
|
||||
```
|
||||
@@ -1,30 +0,0 @@
|
||||
# 3.3 — Authentik service account and token provider (GREEN)
|
||||
|
||||
Phase: 3 — Authentication
|
||||
Stage: GREEN
|
||||
Depends on: [3.2](3.2-bearer-validation.md)
|
||||
|
||||
- [ ] A service account exists in Authentik at `https://authentik.riotpiao.com` for machine callers of the model API
|
||||
- [ ] An OAuth2 provider is configured so that account can obtain a token non-interactively, with no browser step
|
||||
- [ ] The token endpoint returns a signed JWT whose issuer and audience match what the gateway validates, and which the gateway accepts on a protected route
|
||||
- [ ] Token lifetime is set deliberately and recorded, not left at whatever the default is
|
||||
- [ ] The client secret is stored as a Kubernetes Secret referenced from git, never committed in plaintext
|
||||
- [ ] The Authentik configuration is captured in the repo as reproducible steps or declarative config, so it can be rebuilt rather than clicked together again
|
||||
- [ ] The token carries whatever claim [3.5](3.5-capability-authorization.md) authorizes on
|
||||
|
||||
Open risk recorded in `AUTH-PLAN.md` and unresolved: Authentik 2026.x may require an
|
||||
app-password or JWT-assertion flow rather than a plain `client_secret` POST to the
|
||||
token endpoint. Verify which flow the running version accepts before wiring anything
|
||||
that depends on it, and record the answer here.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -i -X POST https://authentik.riotpiao.com/application/o/token/ \
|
||||
-d grant_type=client_credentials -d "client_id=$CID" -d "client_secret=$CSEC"
|
||||
# expected: 200, JSON with access_token; if 400/401, the app-password / JWT-assertion risk above is real — record which
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' http://homelab-frontend.api.svc.cluster.local/v1/models \
|
||||
-H "authorization: Bearer $ACCESS_TOKEN"
|
||||
# expected: 200 — the gateway accepts a real Authentik-issued token, not only a stub one
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# 3.4 — Flag-gated auth rollout (GREEN)
|
||||
|
||||
Phase: 3 — Authentication
|
||||
Stage: GREEN
|
||||
Depends on: [3.2](3.2-bearer-validation.md)
|
||||
|
||||
- [ ] Authentication is controlled by an explicit flag in configuration, and its default is OFF
|
||||
- [ ] With the flag off, every route answers exactly as it did before Phase 3 existed — no 401s, no `WWW-Authenticate`, no behaviour change
|
||||
- [ ] With the flag on, protected routes require `Authorization: Bearer` and reject anything else with 401
|
||||
- [ ] `GET /healthz` and `GET /readyz` never require authentication in either state
|
||||
- [ ] Which routes are protected is per-route configuration, so auth can be turned on for one surface at a time
|
||||
- [ ] Turning the flag on is a git change synced by Argo; it is never toggled by hand against the cluster
|
||||
- [ ] The flag's current state is visible in logs at startup and in metrics, so nobody has to guess whether auth is on
|
||||
- [ ] Turning the flag off again fully restores unauthenticated access, making the rollout reversible
|
||||
|
||||
The model API is unauthenticated today — verified live, a request with no credentials
|
||||
returns 200. Enabling this flag breaks every current caller until they hold a token,
|
||||
pi included. That is why it defaults off and is enabled deliberately, after
|
||||
[3.6](3.6-pi-client-migration.md).
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# flag off
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models # expected: 200
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' # expected: 200
|
||||
|
||||
# flag on, no credentials
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' # expected: 401
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz # expected: 200
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# 3.5 — Capability authorization (RED)
|
||||
|
||||
Phase: 3 — Authentication
|
||||
Stage: RED
|
||||
Depends on: [3.2](3.2-bearer-validation.md), [3.4](3.4-flag-gated-rollout.md)
|
||||
|
||||
- [ ] Beyond proving who the caller is, the gateway checks the token is permitted to invoke the capability being called
|
||||
- [ ] Each capability prefix — `/v1/*` for the model surface, and the future `/sqs/*`, `/workflow/*`, `/cluster/*`, `/db/*` — maps to a required grant, and the mapping is configuration
|
||||
- [ ] A token carrying a queue grant but no model grant is rejected on `POST /v1/chat/completions` with 403, not 401 — it authenticated fine, it is not permitted
|
||||
- [ ] A token with no recognised grant at all is rejected on every protected route
|
||||
- [ ] Rejections are RFC 9457 `application/problem+json` and state which capability was denied, without echoing the token or its claims verbatim
|
||||
- [ ] Denials are logged with the caller identity and the capability, and counted as a distinct rejection reason
|
||||
- [ ] A route with no required grant configured while auth is on fails startup rather than defaulting to allow-all
|
||||
|
||||
Write the failing tests first: a queue token must not invoke a GPU. The default on an
|
||||
unconfigured route is deny, because a fail-open authorization layer is worse than none
|
||||
— it reads as protection while granting everything.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -i localhost:8080/v1/chat/completions -H "authorization: Bearer $QUEUE_ONLY_TOKEN" \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}'
|
||||
# expected: 403, application/problem+json naming the denied capability, no upstream stub hit
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
|
||||
-H "authorization: Bearer $MODEL_TOKEN" -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[]}'
|
||||
# expected: 200
|
||||
|
||||
./gateway --config testdata/auth-on-route-without-grant.yaml; echo "exit=$?"
|
||||
# expected: non-zero exit, stderr names the route missing a required grant
|
||||
```
|
||||
@@ -1,37 +0,0 @@
|
||||
# 3.6 — Migrate pi to Bearer and one provider (REFACTOR)
|
||||
|
||||
Phase: 3 — Authentication
|
||||
Stage: REFACTOR
|
||||
Depends on: [2.2](2.2-body-based-dispatch.md), [3.3](3.3-authentik-service-account.md), [3.4](3.4-flag-gated-rollout.md)
|
||||
|
||||
- [ ] `~/.pi/agent/models.json` no longer contains a `customHeaders: {apikey: ...}` block anywhere
|
||||
- [ ] pi authenticates with `Authorization: Bearer` carrying an Authentik-issued token
|
||||
- [ ] The three provider entries `homelab-reasoning`, `homelab-ornith` and `homelab-qwen` collapse into ONE provider entry
|
||||
- [ ] That single provider's base URL is the canonical OpenAI base, and the three models are listed under it — body-based dispatch makes per-model base URLs unnecessary
|
||||
- [ ] pi reaches all three models through `POST /v1/chat/completions` with `model` set to `reasoning`, `ornith:35b`, and `qwen2.5:3b-instruct`
|
||||
- [ ] Streaming still works from pi, and interrupting a generation cancels it upstream rather than leaving it running
|
||||
- [ ] The old config is captured before editing, so a revert is one file restore
|
||||
- [ ] Verified with auth ON, because that is the state the migration exists to survive
|
||||
|
||||
The `apikey` header exists only because Kong OSS `key-auth` rejected
|
||||
`Authorization: Bearer`. Once this lands, nothing depends on the legacy aliases in
|
||||
[2.4](2.4-legacy-path-aliases.md) and they can be removed.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
grep -c apikey ~/.pi/agent/models.json # expected: 0
|
||||
grep -c '"homelab' ~/.pi/agent/models.json # expected: 1 provider entry, not 3
|
||||
|
||||
for m in reasoning ornith:35b qwen2.5:3b-instruct; do
|
||||
curl -s -o /dev/null -w "$m %{http_code}\n" https://api.riotpiao.com/v1/chat/completions \
|
||||
-H "authorization: Bearer $ACCESS_TOKEN" -H 'content-type: application/json' \
|
||||
-d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}"
|
||||
done
|
||||
# expected: 200 for all three, with auth enabled
|
||||
|
||||
curl -N -s https://api.riotpiao.com/v1/chat/completions -H "authorization: Bearer $ACCESS_TOKEN" \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: text/event-stream chunks arriving incrementally; reasoning_content and content both present
|
||||
```
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Phase: 4 — Limits and budgets
|
||||
Stage: GREEN
|
||||
Depends on: [4.3](4.3-problem-json-errors.md), [2.9](2.9-canonical-request-model.md)
|
||||
Depends on: [4.3](4.3-problem-json-errors.md)
|
||||
|
||||
The `reasoning` upstream is 2 replicas x `--max-num-seqs=4` = 8 concurrent sequence
|
||||
slots cluster-wide. That 8 is a vLLM concurrency setting, not a GPU count — worker-1
|
||||
@@ -10,9 +10,9 @@ has 4 physical GPUs and the two numbers are unrelated. Exceeding 8 does not fail
|
||||
fast; it silently queues inside vLLM where the gateway has no visibility and no
|
||||
ability to cancel.
|
||||
|
||||
The gateway serves two permanent dialects over the same predictors: `/v1/*` is
|
||||
OpenAI-compatible and `/llm/*` is Anthropic Messages. There is ONE controller, keyed by
|
||||
upstream, sitting below both.
|
||||
The gateway serves `/v1/*` (OpenAI-compatible) over the same predictors. The
|
||||
Anthropic `/llm/*` dialect was dropped (see `tasks/INDEX.md` Phase 2) — this
|
||||
controller only ever needs to know about `/v1/*`.
|
||||
|
||||
- [ ] Concurrent in-flight requests to `reasoning` are capped at a configured limit strictly below 8, leaving operator headroom
|
||||
- [ ] The cap, the queue depth, and the queue wait timeout are all explicit in configuration with no silent defaults
|
||||
@@ -21,13 +21,8 @@ upstream, sitting below both.
|
||||
- [ ] A slot is released when the response completes, when the upstream errors, and when the client disconnects mid-stream — no path leaks a slot
|
||||
- [ ] A client that disconnects while still queued never reaches the upstream and never consumes a slot
|
||||
- [ ] The cap applies only to `reasoning`; other upstreams are unaffected and are not blocked behind its queue
|
||||
- [ ] The controller is keyed by UPSTREAM, never by route, path prefix or dialect — `reasoning-predictor.llm-serving:80` has exactly one cap and one queue
|
||||
- [ ] A request arriving on `/v1/chat/completions` and one arriving on `/llm/v1/messages` contend for the same slots and the same queue, admitted in arrival order
|
||||
- [ ] Total in-flight requests against `reasoning` never exceed the configured cap regardless of which surface they arrived through, including when both surfaces are saturated at once
|
||||
- [ ] Per-dialect semaphores are explicitly wrong and must not exist: the 8 sequence slots are physical, so two independent gates would each believe they were within budget while together exceeding it
|
||||
- [ ] The controller reads the dialect-neutral canonical request and has no knowledge of which surface produced it; adding a third dialect requires no change here
|
||||
- [ ] Slot occupancy and queue depth are reported per upstream, and requests are attributable to a surface for logging only, never for admission
|
||||
- [ ] Rejection at a full queue renders per surface — `application/problem+json` on `/v1`, the Anthropic error shape on `/llm` — from one shared decision
|
||||
- [ ] The controller is keyed by UPSTREAM, never by route or path prefix — `reasoning-predictor.llm-serving:80` has exactly one cap and one queue
|
||||
- [ ] Slot occupancy and queue depth are reported per upstream
|
||||
|
||||
## Verify
|
||||
|
||||
@@ -46,14 +41,4 @@ grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
|
||||
curl -s -D - -o /dev/null -X POST localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' | grep -i retry-after
|
||||
# expected: Retry-After present on the rejected request
|
||||
|
||||
# Same cap=6 stub, but split the load across both dialects: 20 on /v1 and 20 on /llm.
|
||||
seq 20 | xargs -P20 -I{} curl -s -o /dev/null -X POST localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' &
|
||||
seq 20 | xargs -P20 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' &
|
||||
wait
|
||||
grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
|
||||
# expected: 0 — the two surfaces share one cap, they do not get 6 each
|
||||
```
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 4.3 — RFC 9457 problem+json rejections (RED)
|
||||
# 4.3 — RFC 9457 problem+json rejections (GREEN)
|
||||
|
||||
Phase: 4 — Limits and budgets
|
||||
Stage: RED
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 5.1 — Prometheus parity with the retiring Kong plugin (RED)
|
||||
# 5.1 — Prometheus parity with the retiring Kong plugin (GREEN)
|
||||
|
||||
Phase: 5 — Observability
|
||||
Stage: RED
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
# 6.1 — Hardened container image (GREEN)
|
||||
|
||||
Phase: 6 — Deploy and cutover
|
||||
Stage: GREEN
|
||||
|
||||
The gateway is the public edge process. It gets no shell, no package manager, no
|
||||
writable filesystem and no privileges it does not need.
|
||||
|
||||
- [ ] The runtime image is distroless or scratch — no shell, no package manager, no busybox
|
||||
- [ ] The container runs as a non-root user, enforced by `runAsNonRoot` and an explicit non-zero UID
|
||||
- [ ] The root filesystem is read-only; any writable path the process genuinely needs is an explicitly mounted volume, not a relaxed rootfs
|
||||
- [ ] All Linux capabilities are dropped, and none are added back
|
||||
- [ ] `seccompProfile` is `RuntimeDefault`
|
||||
- [ ] Privilege escalation is disabled
|
||||
- [ ] Image tags are the commit SHA of the source that built them. Never `:latest`, never a moving tag — Argo `selfHeal` cannot reconcile a mutable tag reliably, and a rollback needs a tag that still means what it meant yesterday
|
||||
- [ ] The image is reproducible from a committed build definition; nothing is built by hand
|
||||
- [ ] The image contains no credentials, kubeconfig or service-account token baked in (G2)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
docker run --rm --entrypoint sh <image>:<sha>
|
||||
# expected: fails — no shell in the image
|
||||
|
||||
docker run --rm --read-only --user 65532 <image>:<sha> --version
|
||||
# expected: starts and exits cleanly under a read-only rootfs as a non-root user
|
||||
|
||||
docker inspect <image>:<sha> --format '{{.Config.User}}'
|
||||
# expected: a non-zero numeric UID, not empty and not "root"
|
||||
|
||||
grep -rn 'image:' k8s/ | grep -v '@sha256\|:[0-9a-f]\{7,\}'
|
||||
# expected: no matches — every image reference is pinned to a SHA
|
||||
```
|
||||
@@ -1,35 +0,0 @@
|
||||
# 6.2 — Deployment, Service, NetworkPolicy (GREEN)
|
||||
|
||||
Phase: 6 — Deploy and cutover
|
||||
Stage: GREEN
|
||||
Depends on: [6.1](6.1-hardened-image.md)
|
||||
|
||||
Manifests only. Nothing here exposes the gateway publicly — that is the cutover, and
|
||||
it is a separate step.
|
||||
|
||||
- [ ] A Deployment runs the gateway with at least 2 replicas, matching Kong's current replica count
|
||||
- [ ] The pod spec carries the hardening from 6.1: non-root, read-only rootfs, all capabilities dropped, `seccompProfile: RuntimeDefault`
|
||||
- [ ] Liveness probes hit `/healthz` and readiness probes hit `/readyz`; readiness fails while config is invalid or JWKS has never been fetched
|
||||
- [ ] Route configuration is mounted from a ConfigMap sourced from git — not a CRD, and not baked into the image
|
||||
- [ ] A config change rolls the pods; a stale ConfigMap cannot be silently served by a long-lived pod
|
||||
- [ ] Termination allows in-flight requests to drain, with a grace period long enough for the streaming timeouts in use
|
||||
- [ ] A Service exposes the gateway in-cluster with a named port, resolvable at `http://<svc>.api.svc.cluster.local`
|
||||
- [ ] No ServiceAccount with any RBAC is bound; the pod does not need or receive an API-server token (G2)
|
||||
- [ ] A NetworkPolicy allows ingress only from `ingress-nginx`
|
||||
- [ ] The same NetworkPolicy allows egress only to the proxied upstreams plus Authentik, plus DNS. Everything else is denied
|
||||
- [ ] Resource requests and limits are set explicitly
|
||||
- [ ] Every manifest is committed to git and applied by Argo. No `kubectl apply`, no `helm upgrade` (G7)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n api get deploy homelab-frontend -o jsonpath='{.spec.replicas} {.spec.template.spec.securityContext}{"\n"}'
|
||||
# expected: 2, with runAsNonRoot true and seccompProfile RuntimeDefault
|
||||
|
||||
kubectl -n api exec deploy/homelab-frontend -- true 2>&1
|
||||
# expected: fails — distroless image has no shell to exec into
|
||||
|
||||
kubectl -n llm-serving run np-probe --rm -it --image=curlimages/curl --restart=Never -- \
|
||||
curl -s -m 5 http://homelab-frontend.api.svc.cluster.local/healthz
|
||||
# expected: times out or is refused — ingress is restricted to ingress-nginx only
|
||||
```
|
||||
@@ -1,32 +0,0 @@
|
||||
# 6.3 — Argo Application in the homelab-root GitOps repo (GREEN)
|
||||
|
||||
Phase: 6 — Deploy and cutover
|
||||
Stage: GREEN
|
||||
Depends on: [6.2](6.2-kubernetes-manifests.md)
|
||||
|
||||
Verified live 2026-08-19: zero Argo Applications anywhere in the cluster source from
|
||||
any Forgejo URL. `github.com/Riotpiaole/riotpiao.homelab.com` is authoritative — do
|
||||
not introduce a second source of truth.
|
||||
|
||||
- [ ] An Argo Application for the gateway is committed under `k8s/argocd/apps/` in `github.com/Riotpiaole/riotpiao.homelab.com`
|
||||
- [ ] `repoURL` is that GitHub repo. No Forgejo URL, no local path, no second remote
|
||||
- [ ] It targets the `api` namespace, alongside the existing Kong deployment
|
||||
- [ ] Its sync wave orders it so the gateway is healthy before anything that depends on it, and does not disturb Kong's existing wave-7 Application
|
||||
- [ ] Automated sync with prune and selfHeal is enabled, so drift is corrected without a human
|
||||
- [ ] The Application reaches `Synced` and `Healthy` and stays there across a resync
|
||||
- [ ] Adding it changes nothing about live traffic — Kong still serves `api.riotpiao.com` after this lands
|
||||
- [ ] The commit is pushed and Argo picks it up on its own. No `kubectl apply` of the Application itself (G7)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n argocd get app homelab-frontend \
|
||||
-o jsonpath='{.spec.source.repoURL}{" "}{.status.sync.status}{" "}{.status.health.status}{"\n"}'
|
||||
# expected: the GitHub repo URL, Synced, Healthy
|
||||
|
||||
kubectl -n argocd get app -o json | grep -ci forgejo
|
||||
# expected: 0 — GitHub remains the only Application source
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
|
||||
# expected: 200, still served by Kong — this task changed no live traffic
|
||||
```
|
||||
@@ -1,36 +0,0 @@
|
||||
# 6.4 — Deploy unexposed alongside Kong (GREEN)
|
||||
|
||||
Phase: 6 — Deploy and cutover
|
||||
Stage: GREEN
|
||||
Depends on: [6.3](6.3-argocd-application.md)
|
||||
|
||||
The gateway runs in the cluster against the real predictors while Kong continues to
|
||||
serve every byte of live traffic. This is the last step before anything user-visible
|
||||
changes, and it is fully reversible — deleting the Application removes it.
|
||||
|
||||
- [ ] The gateway is running in namespace `api` and is reachable only in-cluster at `http://<svc>.api.svc.cluster.local`
|
||||
- [ ] No Ingress points at the gateway. Ingress `api/api` still sends `/` on `api.riotpiao.com` to `kong-proxy:80`
|
||||
- [ ] Its configured upstreams are the real Services: `reasoning-predictor.llm-serving:80`, `ornith-predictor.llm-serving:80`, `embeddings-predictor.llm-serving:80`, `reranker-predictor.llm-serving:80`, and `agent-hub.agent-pod:9090`
|
||||
- [ ] Note `reasoning-predictor` listens on port 80, not 8080
|
||||
- [ ] A real chat completion succeeds in-cluster against `reasoning`, returning `reasoning_content` and `content` as separate fields
|
||||
- [ ] A streaming chat completion delivers tokens incrementally in-cluster, not as one buffered blob at completion
|
||||
- [ ] Embeddings and rerank both answer correctly, with rerank reaching the upstream's `/rerank` path
|
||||
- [ ] A client disconnect mid-stream is observed to cancel the upstream generation and release its sequence slot
|
||||
- [ ] Prometheus is scraping the gateway and the metrics reflect this in-cluster traffic
|
||||
- [ ] `api.riotpiao.com` is unaffected throughout — verified before and after
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
|
||||
# expected: kong-proxy — the gateway is still unexposed
|
||||
|
||||
kubectl -n api run probe --rm -it --image=curlimages/curl --restart=Never -- sh -c '
|
||||
curl -sN -X POST http://homelab-frontend.api.svc.cluster.local/v1/chat/completions \
|
||||
-H "content-type: application/json" \
|
||||
-d "{\"model\":\"reasoning\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"count to 5\"}]}"'
|
||||
# expected: multiple data: chunks arriving over time, terminated by data: [DONE]
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
|
||||
# expected: 200 — public traffic still served by Kong, untouched
|
||||
```
|
||||
@@ -1,45 +0,0 @@
|
||||
# 6.5 — Cutover: repoint Ingress `api/api` (GREEN)
|
||||
|
||||
Phase: 6 — Deploy and cutover
|
||||
Stage: GREEN
|
||||
Depends on: [2.8](2.8-kong-parity-test.md), [6.4](6.4-deploy-alongside-kong.md)
|
||||
|
||||
THIS is the cutover. Ingress `api/api` catch-alls `/` on `api.riotpiao.com`; today its
|
||||
backend is `kong-proxy:80`. Changing that one backend moves all live traffic to the
|
||||
gateway. Reverting is the same one-line change back to `kong-proxy:80`, committed and
|
||||
synced — Kong stays running and untouched, so the revert takes effect in seconds.
|
||||
|
||||
Preconditions, all required before starting:
|
||||
|
||||
- [ ] 2.8 Kong parity is passing — gateway and Kong return equivalent responses for every route in the migration inventory, including a streaming request and a mid-stream disconnect
|
||||
- [ ] 6.4 is complete: the gateway is healthy in-cluster against the real predictors, and Prometheus is scraping it
|
||||
- [ ] The inventory has been re-verified immediately beforehand. Kong's Deployment has eleven ReplicaSets and is being actively iterated; a stale inventory is a stale plan
|
||||
- [ ] Kong remains deployed and serving-capable throughout. Nothing about Kong is deleted in this task
|
||||
|
||||
The change itself:
|
||||
|
||||
- [ ] The `api/api` Ingress backend becomes the gateway Service, changed in git and synced by Argo. No `kubectl apply`, no `kubectl edit` (G7)
|
||||
- [ ] The nginx annotations stay exactly as they are: `proxy-read-timeout: 3600`, `proxy-send-timeout: 3600`, `proxy-buffering: off`, `proxy-body-size: 0`. These are what make token streaming work and the gateway needs the same treatment
|
||||
- [ ] The revert is a single-line commit reverting the backend to `kong-proxy:80`, and it has been rehearsed at least once
|
||||
- [ ] After the change, public streaming works end to end through nginx, unbuffered
|
||||
- [ ] pi keeps working on the legacy `/v1/{reasoning,ornith,qwen}/chat/completions` aliases
|
||||
- [ ] A soak period follows, watching gateway metrics, rejection counters and pi traffic. Do not proceed to 6.6 until the soak is clean
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
|
||||
# expected: the gateway Service, no longer kong-proxy
|
||||
|
||||
curl -sN -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"count to 5"}]}'
|
||||
# expected: incremental data: chunks over time through nginx, ending in data: [DONE]
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://api.riotpiao.com/v1/reasoning/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}'
|
||||
# expected: 200 — pi's legacy alias still works
|
||||
|
||||
kubectl -n api get deploy kong-kong -o jsonpath='{.status.readyReplicas}{"\n"}'
|
||||
# expected: 2 — Kong still running and ready, so the revert is one commit away
|
||||
```
|
||||
@@ -1,45 +0,0 @@
|
||||
# 6.6 — Kong teardown (REFACTOR)
|
||||
|
||||
Phase: 6 — Deploy and cutover
|
||||
Stage: REFACTOR
|
||||
Depends on: [6.5](6.5-cutover.md)
|
||||
|
||||
IRREVERSIBLE. Every step before this one could be undone in seconds by repointing
|
||||
Ingress `api/api` back to `kong-proxy:80`. Once the Kong Application is removed and
|
||||
Argo prunes the Helm release and its CRDs, that escape hatch is gone — recovery means
|
||||
reinstalling Kong from scratch and rebuilding six plugin CRs.
|
||||
|
||||
Do not start until the soak after 6.5 has been clean for a deliberate, agreed period.
|
||||
|
||||
Preconditions:
|
||||
|
||||
- [ ] 6.5 is complete: `api.riotpiao.com` has been served entirely by the gateway through the soak, with no reverts
|
||||
- [ ] Gateway metrics over the soak show no elevated 5xx rate and no unexplained rejections
|
||||
- [ ] pi and every other known caller have been confirmed working against the gateway
|
||||
- [ ] The inventory has been re-verified immediately beforehand — Kong's config has been actively iterated
|
||||
|
||||
Order matters. Delete the routing objects first, the Application last:
|
||||
|
||||
- [ ] The 7 `ingressClassName: kong` Ingresses are deleted from git — 6 in `llm-serving`, 1 in `agent-pod` for `/console`, `/run`, `/sessions`
|
||||
- [ ] The 6 `KongPlugin` CRs are deleted from git: the three `llm-rewrite-*` chat rewrites, `llm-rewrite-rerank`, `llm-models-list`, and the cluster-wide `prometheus` plugin
|
||||
- [ ] Removing the cluster-wide `prometheus` plugin does not blind any dashboard, because 5.1 parity metrics are already being scraped from the gateway
|
||||
- [ ] The `kong` Application is then removed from `k8s/argocd/apps/55-api-gateway.yaml`
|
||||
- [ ] Argo prunes the Helm release, the Kong CRDs and the namespace leftovers on its own. No `helm uninstall`, no `kubectl delete` (G7)
|
||||
- [ ] Public traffic is verified unaffected after each deletion, not only at the end
|
||||
- [ ] No orphaned Kong CRDs, ReplicaSets or Services remain in namespace `api`
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl get ingress -A -o jsonpath='{range .items[*]}{.spec.ingressClassName}{"\n"}{end}' | sort | uniq -c
|
||||
# expected: no kong entries remain; nginx and istio counts unchanged
|
||||
|
||||
kubectl get kongplugins -A 2>&1; kubectl -n argocd get app kong 2>&1
|
||||
# expected: CRD not found, and Application "kong" not found
|
||||
|
||||
kubectl -n api get all | grep -i kong
|
||||
# expected: no output — nothing Kong-related left
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
|
||||
# expected: 200, served by the gateway
|
||||
```
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: [6.5](6.5-cutover.md)
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
atlas lives in a separate repo, `riotpiao-backend`. It keeps its own informers and its
|
||||
own RBAC. The gateway proxies to it and holds no Kubernetes credentials of its own —
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: [6.5](6.5-cutover.md)
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
`management-service` already exposes gRPC at `kmsvc.riotpiao.com`. This prefix is a
|
||||
NEW surface, not a replacement — that gRPC endpoint stays exactly as it is and nothing
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: [6.5](6.5-cutover.md)
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
Temporal runs in the `temporal` namespace. Temporal namespace registration is
|
||||
automatic via queue-operator and is NEVER done manually — this route must not create,
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: [6.5](6.5-cutover.md)
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
The most dangerous prefix in this phase. G2 says the gateway holds no cluster
|
||||
credentials — so it cannot be the thing that authenticates to CloudNativePG, MinIO or
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
# 8.1 — `ServiceAdapter` CRD, informer, RBAC (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §1.
|
||||
|
||||
**G2 note — read before objecting.** G2 says "the gateway holds no Kubernetes
|
||||
credentials, config comes from git not a CRD." This task deliberately supersedes G2
|
||||
for this one narrow purpose; the acknowledged-supersession rationale is written out
|
||||
in API_ROUTING_HYBRID_DESIGN.md's Context section (top of that doc) and in
|
||||
`INDEX.md`'s G2 line. Do not treat this as a task-file error — the supersession is
|
||||
intentional and pre-approved, scoped to exactly the read-only Role this task adds.
|
||||
|
||||
- [ ] `apis/gateway/v1/serviceadapter_types.go` defines `ServiceAdapter` matching
|
||||
the design doc's example CRs (§1, §6): `spec.serviceName`, `spec.upstream.{url,timeoutSeconds}`,
|
||||
`spec.auth.{required,capability}`, `spec.retryable`, `spec.resources[].name`,
|
||||
`spec.resources[].methods[].{verb,upstreamPath,requestSchema,responseSchema,auth}`
|
||||
- [ ] `controller-gen` generates the CRD YAML from those types into `k8s/crd-serviceadapter.yaml`, added to `k8s/kustomization.yaml`'s `resources:`
|
||||
- [ ] CRD is namespace-scoped, group `gateway.riotpiao.com/v1`, kind `ServiceAdapter` — not cluster-scoped
|
||||
- [ ] `k8s/rbac.yaml` gains a namespace-scoped `Role`/`RoleBinding` (`get`, `list`, `watch` only, no `status`/`finalizers` verbs) for the existing `api-gateway` ServiceAccount, exactly as specced in §1
|
||||
- [ ] `internal/serviceadapter/registry.go`: `client-go` `SharedInformer` on `ServiceAdapter` in namespace `api`, feeding an in-memory map keyed by `spec.serviceName`
|
||||
- [ ] Add/Update/Delete informer callbacks mutate the map directly; no gateway restart required to pick up a CR change
|
||||
- [ ] A `ServiceAdapter` CR with a malformed `requestSchema`/`responseSchema` (per 8.3's DSL) logs an error and is skipped — it does not crash the informer or block other adapters from loading
|
||||
- [ ] `go build ./...` succeeds with the new `apis/` package and `internal/serviceadapter/registry.go` in the tree
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n api get role api-gateway-serviceadapter-reader -o yaml
|
||||
# expected: rules limited to gateway.riotpiao.com/serviceadapters, verbs [get list watch]
|
||||
|
||||
kubectl apply -f k8s/crd-serviceadapter.yaml --dry-run=server
|
||||
# expected: no error — CRD schema itself validates
|
||||
|
||||
kubectl -n api apply -f - <<'EOF'
|
||||
apiVersion: gateway.riotpiao.com/v1
|
||||
kind: ServiceAdapter
|
||||
metadata: { name: smoke-test }
|
||||
spec:
|
||||
serviceName: smoke-test
|
||||
upstream: { url: http://example.invalid, timeoutSeconds: 5 }
|
||||
auth: { required: false }
|
||||
resources: []
|
||||
EOF
|
||||
kubectl -n api logs deploy/api-gateway --since=10s | grep -i "smoke-test"
|
||||
# expected: informer logs an Add event for smoke-test within resync/watch latency, no restart
|
||||
kubectl -n api delete serviceadapter smoke-test
|
||||
```
|
||||
@@ -0,0 +1,44 @@
|
||||
# 8.10 — Phase 8 gate: every service on `ServiceAdapter` + KV-schema (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: gate
|
||||
Depends on: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8 (8.9 optional — see below)
|
||||
|
||||
Purpose: confirm all real backend services — `workflow`, `s3`, `sqs`, `iam`,
|
||||
`memory` — are onboarded through the `ServiceAdapter` CRD with `requestSchema`/
|
||||
`responseSchema` validation (8.3's DSL), none of them left on hand-written
|
||||
`switch`-case Go routes or the old path-prefix scheme (`/workflow/*`, `/sqs/*`,
|
||||
`/db/*`). This is the "make sure every service adapts to this format" checkpoint —
|
||||
it does not add new capability, it verifies consistency across what 8.4–8.8 built.
|
||||
|
||||
- [ ] `kubectl -n api get serviceadapters` lists exactly `workflow`, `s3`, `sqs`,
|
||||
`iam`, `memory` (plus `postgres` only if that example CR was actually applied
|
||||
as a real onboarding, not just kept as doc illustration)
|
||||
- [ ] No adapter's CR has an empty `requestSchema` on a method that accepts a body
|
||||
— every write path validates input
|
||||
- [ ] `internal/server/router.go` has no remaining path-based `switch` case for
|
||||
`/workflow`, `/sqs`, or `/db` — those prefixes 404 or are fully removed from
|
||||
the router, superseded by `X-Service` dispatch
|
||||
- [ ] One curl per adapter succeeds end-to-end through the header-based path (below)
|
||||
- [ ] `go test ./... -race`, `CGO_ENABLED=0 go build ./...`, `go vet ./...` all pass
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
for svc_resource in "workflow:workflow" "sqs:message" "iam:user" "memory:project"; do
|
||||
svc="${svc_resource%%:*}"; res="${svc_resource##*:}"
|
||||
code=$(curl -s -o /dev/null -w '%{http_code}' https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt-with-all-capabilities>' \
|
||||
-H "X-Service: $svc" -H "X-Resource: $res")
|
||||
echo "$svc/$res -> $code"
|
||||
done
|
||||
# expected: none of the four returns 404 for "unknown X-Service" — each is a live adapter
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/workflow/health
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/sqs/healthz
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/db/healthz
|
||||
# expected: 404 on all three — old prefix routes are gone, not just unused
|
||||
|
||||
kubectl -n api get serviceadapters -o jsonpath='{range .items[*]}{.spec.serviceName}{"\n"}{end}' | sort
|
||||
# expected: iam, memory, s3, sqs, workflow (plus postgres iff real)
|
||||
```
|
||||
@@ -0,0 +1,55 @@
|
||||
# 8.2 — `X-Service`/`X-Resource` dispatcher, capability auth, blind 5xx retry (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1 (CRD, informer, in-memory registry)
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2, §4.
|
||||
|
||||
- [ ] `internal/server/router.go`'s `ServeHTTP` gets a new branch, checked **before**
|
||||
the existing path switch: if the request carries an `X-Service` header,
|
||||
dispatch to `internal/serviceadapter/router.go`, regardless of `r.URL.Path`
|
||||
- [ ] `/v1/chat/completions`, `/v1/embeddings`, `/v1/rerank`, `/healthz`, `/readyz`
|
||||
keep their existing path-based routing unchanged — matched first, never see `X-Service`
|
||||
- [ ] Dispatch key: `X-Service` header → adapter from 8.1's registry, then HTTP method
|
||||
+ `X-Resource` header → `{verb, upstreamPath}` on that adapter
|
||||
- [ ] Unknown `X-Service` → 404 problem+json. Known service, unknown `X-Resource`/verb
|
||||
combination → 404 problem+json, not a silent proxy-through
|
||||
- [ ] Auth: `spec.auth.capability` is the default per adapter; a method's own
|
||||
`auth.capability` overrides it; `auth.required: false` at either level skips
|
||||
the capability check entirely. Depends on `internal/auth` (validated JWT
|
||||
middleware) existing — if it does not yet exist in this repo, stop and report
|
||||
instead of stubbing it
|
||||
- [ ] `internal/resilience/retry.go` (new): blind retry on any 5xx from the upstream,
|
||||
bounded attempts with backoff, gated by `spec.retryable` (default `true`) —
|
||||
wraps every outbound call this dispatcher makes
|
||||
- [ ] `{id}`-style path segments in `upstreamPath` (e.g. `/v1/tables/{id}`) are
|
||||
resolved from an explicit source — since routing is header-only at the gateway
|
||||
root, there is no URL path segment to take it from. Decide and document the
|
||||
actual source (query param, extra header, or body field) in this task's own
|
||||
notes before implementing; do not guess silently
|
||||
- [ ] Legacy `/workflow*` path stays mounted, delegating internally to this same
|
||||
dispatcher, per §2
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# unknown service
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: does-not-exist' -H 'X-Resource: whatever'
|
||||
# expected: 404
|
||||
|
||||
# known service, wrong verb+resource combination
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -X PATCH https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: iam' -H 'X-Resource: role'
|
||||
# expected: 404 (role only defines GET/POST in §3)
|
||||
|
||||
# capability override enforced
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt-with-memory:read-only>' \
|
||||
-H 'X-Service: memory' -H 'X-Resource: ingest' -X POST -d '{}'
|
||||
# expected: 403 — ingest requires memory:write, token only has memory:read
|
||||
|
||||
# blind retry: point smoke-test adapter (from 8.1) at an upstream returning 503 twice then 200,
|
||||
# confirm the caller sees 200 and the upstream log shows 3 attempts
|
||||
```
|
||||
@@ -0,0 +1,60 @@
|
||||
# 8.3 — Request/response schema validation, KV+type DSL (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1 (CRD types carry `requestSchema`/`responseSchema`), 8.2 (dispatcher calls this per request)
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §1
|
||||
"Request/response validation" subsection — the full DSL including the array/nullable
|
||||
extension. Read that subsection in full before implementing; do not reinvent the
|
||||
shape from the checklist below alone.
|
||||
|
||||
- [ ] `internal/serviceadapter/validate.go` (new, no external dependency —
|
||||
not a JSON Schema library): validates a parsed body (`map[string]interface{}`
|
||||
or `[]interface{}` for array-typed schemas) against a `FieldSchema`
|
||||
- [ ] Object schema: `required` fields present, each present field in `fields`
|
||||
matches its declared Go runtime type (`string`→string, `number`→float64,
|
||||
`boolean`→bool, `array`→`[]interface{}`, `object`→`map[string]interface{}`)
|
||||
- [ ] `nullable: true` on a field accepts JSON `null` regardless of declared type;
|
||||
a `null` on a non-nullable field is a `type_mismatch`; bare `field: string`
|
||||
shorthand means `{type: string, nullable: false}`
|
||||
- [ ] `strict: true` rejects body keys not listed in `fields`; default `false` is permissive
|
||||
- [ ] Array schema (`type: array`): `items: string` validates every element is that
|
||||
scalar type; `items: { fields: {...} }` validates every element as an object
|
||||
schema, one level deep inside each item — no further nesting
|
||||
- [ ] Compiled once per CR add/update in 8.1's informer callback, not per-request —
|
||||
the parsed `FieldSchema` stored in the same registry entry as the route
|
||||
- [ ] Request-side violation → 400, RFC 9457 `problem+json`,
|
||||
`{type, title, detail, errors: [{field, reason}]}`, `reason` one of
|
||||
`missing`, `type_mismatch: want X got Y`, `unknown_field`
|
||||
- [ ] Response-side violation does **not** block the response — forwarded unchanged,
|
||||
emits `serviceadapter_response_schema_mismatch{service,resource}` metric + log line
|
||||
- [ ] GET requests with no body skip request-schema validation entirely (query-param
|
||||
validation is out of scope for this task — schemas in this repo's adapters only
|
||||
apply to POST/PATCH bodies per the CRs in §1/§3/§6)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# missing required field
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: postgres' -H 'X-Resource: query' \
|
||||
-d '{"params": []}'
|
||||
# expected: 400, errors: [{"field":"sql","reason":"missing"}]
|
||||
|
||||
# type mismatch
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: postgres' -H 'X-Resource: query' \
|
||||
-d '{"sql": 42}'
|
||||
# expected: 400, errors: [{"field":"sql","reason":"type_mismatch: want string got number"}]
|
||||
|
||||
# nullable field accepted
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: memory' -H 'X-Resource: skill'
|
||||
# expected: 200 even though generated_from is null in the response body — response
|
||||
# validation must not reject/replace the body
|
||||
|
||||
# unit test, not curl:
|
||||
go test ./internal/serviceadapter/... -run TestValidate -v
|
||||
# expected: covers array-of-scalar, array-of-object, strict rejection, nullable
|
||||
```
|
||||
@@ -0,0 +1,50 @@
|
||||
# 8.4 — `X-Service: workflow` adapter, supersedes `/workflow/*` prefix (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2.
|
||||
Supersedes [7.3](7.3-workflow-prefix.md) — that task's prefix-based `/workflow/*`
|
||||
route was itself mocked/partial (see `INDEX.md` Progress note); this task replaces
|
||||
it with the header-based adapter rather than finishing it as originally specced.
|
||||
Do not also try to complete 7.3's checklist — its route is retired, not extended.
|
||||
|
||||
- [ ] `k8s/serviceadapter-workflow.yaml` CR: `serviceName: workflow`, upstream = the
|
||||
Temporal Service in the `temporal` namespace, `auth.capability: workflow:access`
|
||||
- [ ] Resource `workflow` maps: `POST` → `START_WORKFLOW`, `GET /workflow/{id}` →
|
||||
`QUERY_WORKFLOW`, `GET` (list) → `LIST_WORKFLOWS`, `DELETE /workflow/{id}` →
|
||||
`TERMINATE_WORKFLOW`, `GET /workflow/{id}/history` → `GET_WORKFLOW_HISTORY`
|
||||
(§2's mapping table)
|
||||
- [ ] `requestSchema` on `START_WORKFLOW`'s POST method — fields for whatever the
|
||||
Temporal start-workflow call actually needs (namespace, workflow type, args);
|
||||
define against the real `internal/temporal` client code, not invented fields
|
||||
- [ ] Long-poll/streaming semantics from the old `/workflow` handler are preserved
|
||||
unbuffered through the new dispatcher (G4 still applies)
|
||||
- [ ] Nothing in this adapter registers a Temporal namespace — registration stays
|
||||
with queue-operator, same invariant as 7.3
|
||||
- [ ] Old `/workflow*` path-mounted route is removed once this adapter is verified live
|
||||
- [ ] Metrics/rejection counters cover this adapter with its own service label
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: workflow' -H 'X-Resource: workflow'
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $WORKFLOW_TOKEN" \
|
||||
-H 'X-Service: workflow' -H 'X-Resource: workflow' \
|
||||
-d '{"namespace": "default", "workflowType": "smoke-test", "args": []}'
|
||||
# expected: 200/202, Temporal's own start-workflow response, proxied unmodified
|
||||
|
||||
curl -s https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $WORKFLOW_TOKEN" \
|
||||
-H 'X-Service: workflow' -H 'X-Resource: workflow/<id>/history'
|
||||
# expected: 200, workflow history payload
|
||||
|
||||
kubectl -n temporal exec svc/temporal-admintools -- tctl --ad temporal-frontend:7233 namespace list | sort > /tmp/ns.after
|
||||
diff /tmp/ns.before /tmp/ns.after
|
||||
# expected: no diff
|
||||
```
|
||||
@@ -0,0 +1,56 @@
|
||||
# 8.5 — `X-Service: sqs` adapter, supersedes `/sqs/*` prefix (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2
|
||||
(`queue/{name}/message` resource shape), and [docs/API-sqs.md](../docs/API-sqs.md)
|
||||
for the real upstream contract — six RPCs, base64 `bytes` fields, lowerCamelCase
|
||||
JSON, 256 KiB body cap, 20s long-poll. Supersedes [7.2](7.2-sqs-prefix.md); that
|
||||
prefix-based route is retired by this adapter, not extended.
|
||||
|
||||
- [ ] `k8s/serviceadapter-sqs.yaml` CR: `serviceName: sqs`, upstream
|
||||
`management-service.sqs.svc.cluster.local:8080`, `auth.capability: queue:access`
|
||||
- [ ] Resource `message` maps the six RPCs from `docs/API-sqs.md`'s table:
|
||||
`POST queue/{name}/message` → `SendMessage`,
|
||||
`POST queue/{name}/message:batch` → `SendMessageBatch`,
|
||||
`GET queue/{name}/message` → `ReceiveMessage`,
|
||||
`DELETE queue/{name}/message/{receiptHandle}` → `DeleteMessage`,
|
||||
`POST queue/{name}/message:batchDelete` → `DeleteMessageBatch`,
|
||||
`PATCH queue/{name}/message/{receiptHandle}` → `ChangeMessageVisibility`
|
||||
- [ ] `{name}`/`{receiptHandle}` path segments resolved the same way 8.2 decided for
|
||||
`{id}` generally — do not invent a second mechanism here
|
||||
- [ ] `requestSchema` on `SendMessage`: `messageBody: string` (required, base64,
|
||||
the KV+type DSL does not validate base64-ness — that stays a body-content
|
||||
concern, not a schema-type concern), `messageAttributes: object`,
|
||||
`messageGroupId: string`, `messageDeduplicationId: string`, `delaySeconds: number`
|
||||
- [ ] `GET` (`ReceiveMessage`) read timeout exceeds 20s (max `waitTimeSeconds`) with
|
||||
headroom, and a client disconnect cancels the upstream long-poll (G4)
|
||||
- [ ] Upstream error envelope (`{"code": 5, "message": "...", "details": []}`) is
|
||||
passed through unchanged, per `docs/API-sqs.md`'s explicit recommendation —
|
||||
not re-rendered as RFC 9457
|
||||
- [ ] `kmsvc-redis-master.sqs:6379` (unauthenticated) stays unreachable — the
|
||||
NetworkPolicy carried over from 7.2 grants no egress to it
|
||||
- [ ] Queue lifecycle (create/delete/list) is **not** exposed — same G2 boundary
|
||||
`docs/API-sqs.md` already states, unchanged by this adapter
|
||||
- [ ] Old `/sqs/*` path-mounted route is removed once this adapter is verified live
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
Q=agent-worker-queue
|
||||
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $QUEUE_TOKEN" -H 'X-Service: sqs' -H 'X-Resource: message' \
|
||||
-d "{\"messageBody\":\"$(printf 'hello world' | base64)\"}"
|
||||
# expected: {"messageId": "...", "sequenceNumber": ""}
|
||||
|
||||
curl -s "https://api.riotpiao.com/?queue=$Q" \
|
||||
-H "Authorization: Bearer $QUEUE_TOKEN" -H 'X-Service: sqs' -H 'X-Resource: message' \
|
||||
-G --data-urlencode 'maxNumberOfMessages=10' --data-urlencode 'waitTimeSeconds=20'
|
||||
# expected: 200 within ~20s, {"messages":[...]}, connection not dropped by gateway timeout
|
||||
|
||||
kubectl -n api get networkpolicy -o yaml | grep -c 6379
|
||||
# expected: 0
|
||||
```
|
||||
@@ -0,0 +1,53 @@
|
||||
# 8.6 — `X-Service: s3` adapter, read-only object surface (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2
|
||||
(`bucket/{key}` resource shape). No prior task or doc names the actual MinIO Service
|
||||
— `grep -ri minio` across this repo turns up only prose in `REQUIREMENTS.md`/
|
||||
`README.md`/`tasks/INDEX.md`/`tasks/7.4-db-prefix.md`, no manifest. **Confirm the
|
||||
real Service name/namespace/port in-cluster before writing the CR** — do not guess
|
||||
a hostname.
|
||||
|
||||
**G2 boundary, same as [7.4](7.4-db-prefix.md):** the gateway holds no MinIO access
|
||||
key or secret key. If this adapter's design wants the gateway to hold a credential,
|
||||
the design is wrong — put the credential-holding logic in a service behind the
|
||||
gateway (e.g. a small internal proxy that signs requests) and adapt to *that*, not
|
||||
to MinIO directly, unless MinIO itself supports anonymous/read-only bucket policies
|
||||
that make a credential unnecessary for the specific buckets exposed here.
|
||||
|
||||
- [ ] `k8s/serviceadapter-s3.yaml` CR: `serviceName: s3`, `auth.capability: s3:read`
|
||||
- [ ] Resource `bucket` maps `GET bucket/{key}` → object read, `DELETE bucket/{key}`
|
||||
→ object delete — **only if** a write/delete capability is explicitly wanted;
|
||||
default to read-only (`GET` only) unless told otherwise, consistent with 7.4's
|
||||
"no write, no delete" rule for the `/db/*` surface this supersedes
|
||||
- [ ] Result listing (if a bucket-list resource is added) is paginated with a
|
||||
bounded page size — no unbounded listing, same rule 7.4 already established
|
||||
- [ ] `requestSchema`/`responseSchema` per the KV+type DSL (8.3) — define once the
|
||||
actual MinIO/proxy response shape is confirmed, not invented ahead of it
|
||||
- [ ] NetworkPolicy reaches only the confirmed MinIO Service, nothing broader
|
||||
- [ ] `/db/*` prefix-mounted MinIO read paths (if any exist from 7.4) are removed
|
||||
once this adapter is verified live, to avoid two auth paths to the same data
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: s3' -H 'X-Resource: bucket/some-key'
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -X DELETE -o /dev/null -w '%{http_code}\n' \
|
||||
-H "Authorization: Bearer $S3_READ_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: s3' -H 'X-Resource: bucket/some-key'
|
||||
# expected: 404 or 405 if this adapter ships read-only — no mutating verb is routable
|
||||
|
||||
curl -s -H "Authorization: Bearer $S3_READ_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: s3' -H 'X-Resource: bucket/known-object-key'
|
||||
# expected: 200, object bytes or metadata per the confirmed response shape
|
||||
|
||||
kubectl -n api get pod -l app=api-gateway -o jsonpath='{range .items[0].spec.containers[0].env[*]}{.name}{"\n"}{end}' \
|
||||
| grep -Ei 'minio|access_key|secret_key'
|
||||
# expected: no output — gateway carries no MinIO credential
|
||||
```
|
||||
@@ -0,0 +1,48 @@
|
||||
# 8.7 — `X-Service: iam` adapter, Authentik admin surface (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §3.
|
||||
|
||||
- [ ] `internal/iam/handler.go` (new) implements the §3 mapping table:
|
||||
`user` GET/POST → `/api/v3/core/users/`, `user/{id}` GET/PATCH/DELETE →
|
||||
`/api/v3/core/users/{id}/`, `service-account` POST →
|
||||
`/api/v3/core/users/service_account/`, `role` GET/POST →
|
||||
`/api/v3/core/groups/`, `permission` GET/POST → `/api/v3/rbac/permissions/`,
|
||||
`flow` GET → `/api/v3/flows/instances/`
|
||||
- [ ] `k8s/serviceadapter-iam.yaml` CR: `serviceName: iam`, upstream = Authentik's
|
||||
internal Service, `auth.capability: iam:admin` (default — this surface is
|
||||
admin-only, tighter than the other adapters' read/write split)
|
||||
- [ ] `requestSchema` on `POST user` and `POST service-account` — fields matching
|
||||
Authentik's actual `/api/v3/core/users/` create-user body, confirmed against
|
||||
the live API, not invented
|
||||
- [ ] This is additive to `core iam` CLI subcommand (`~/workplace/core/src/cmd/iam/`),
|
||||
not a replacement — different caller (server vs. local CLI), same upstream.
|
||||
Do not modify the `core` CLI as part of this task
|
||||
- [ ] No token without `iam:admin` reaches any of these resources, including `flow`
|
||||
(GET-only, but still admin-scoped per the design doc — do not default it to
|
||||
a lower/no-auth tier because it's read-only)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: iam' -H 'X-Resource: user'
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' \
|
||||
-H "Authorization: Bearer $NON_ADMIN_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: iam' -H 'X-Resource: user'
|
||||
# expected: 403 — token lacks iam:admin
|
||||
|
||||
curl -s -H "Authorization: Bearer $IAM_ADMIN_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: iam' -H 'X-Resource: user'
|
||||
# expected: 200, Authentik's user list, proxied through /api/v3/core/users/
|
||||
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $IAM_ADMIN_TOKEN" \
|
||||
-H 'X-Service: iam' -H 'X-Resource: role' -d '{}'
|
||||
# expected: 400 — requestSchema rejects an empty group-create body
|
||||
```
|
||||
@@ -0,0 +1,64 @@
|
||||
# 8.8 — `X-Service: memory` adapter, core resources (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §6.
|
||||
Covers only the poimen-memory endpoints confirmed **already built and tested**
|
||||
(`~/workplace/Poimen/memory/DESIGN.md`'s M3.5 phase): `GET /memory/query`,
|
||||
`GET /memory/projects`, `GET /memory/projects/{id}/status`, `GET /memory/skills`,
|
||||
`GET /memory/skills/{name}`, `POST /memory/ingest`. `/memory/context`,
|
||||
`/memory/projects/{id}/notes` and the git-aware `/memory/nodes/*` endpoints are
|
||||
**not** in scope here — see [8.9](8.9-memory-adapter-extended.md).
|
||||
|
||||
**Hard prerequisite, not optional — do in this order:**
|
||||
1. `NetworkPolicy` in namespace `poimen` restricting ingress on `poimen-memory` to
|
||||
the `api` namespace's gateway pod only. Must land before step 2.
|
||||
2. Remove the `apikey:` middleware from `poimen-memory` itself — separate change
|
||||
in the `~/workplace/Poimen/memory` repo, out of scope for this repo but a hard
|
||||
prerequisite for this adapter being safe to expose. Do not apply the CR below
|
||||
before this lands.
|
||||
3. Provision `memory:read`/`memory:write` as real Authentik scopes (via 8.7's
|
||||
`iam` adapter or `core mwinit`-issued tokens).
|
||||
|
||||
- [ ] `k8s/serviceadapter-memory.yaml` CR per §6's example, `auth.capability: memory:read`
|
||||
default, `ingest` method overrides to `memory:write`
|
||||
- [ ] `responseSchema` on `query` uses 8.3's array-of-object extension:
|
||||
`type: array, items: { fields: { level: string, sha256: string, text: string, score: number } }`
|
||||
- [ ] `responseSchema` on `projects` uses the array-of-scalar extension:
|
||||
`type: array, items: string`
|
||||
- [ ] `responseSchema` on `skills`/`skills/{name}` marks `generated_from` as
|
||||
`{type: string, nullable: true}` — the real upstream response sends `null`
|
||||
for un-derived skills, confirmed in `DESIGN.md`'s example
|
||||
- [ ] `requestSchema` on `ingest`: `required: ["project", "source", "records"]`,
|
||||
`ingest_id` optional (`strict: false` — server may compute it if absent)
|
||||
- [ ] A `memory:read`-scoped token can `GET` `query`/`skill`/`project` and gets 403
|
||||
on `ingest`; a `memory:write`-scoped token can `POST ingest`
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: query' \
|
||||
-G --data-urlencode 'query=why did requests over 10KB fail' \
|
||||
--data-urlencode 'project=poimen' --data-urlencode 'level=L1,L2'
|
||||
# expected: 200, JSON array of {level,sha256,text,score,parents}
|
||||
|
||||
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: project'
|
||||
# expected: 200, ["poimen", ...]
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -X POST \
|
||||
-H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: ingest' -d '{}'
|
||||
# expected: 403 — memory:read token, ingest needs memory:write
|
||||
|
||||
curl -s -X POST -H 'Authorization: Bearer <jwt-with-memory:write>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: ingest' \
|
||||
-d '{"project":"poimen","source":"agent:uuid","records":[]}'
|
||||
# expected: 202, {"job_id":"ingest-...","status_url":"..."}
|
||||
|
||||
kubectl -n poimen get networkpolicy -o yaml | grep -A5 poimen-memory
|
||||
# expected: ingress restricted to the api namespace's gateway pod selector only
|
||||
```
|
||||
@@ -0,0 +1,51 @@
|
||||
# 8.9 — `X-Service: memory` adapter, extended resources (BLOCKED)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: BLOCKED — do not start until the upstream note below is resolved
|
||||
Depends on: 8.8 (core `memory` adapter live)
|
||||
|
||||
Design contract: `~/workplace/Poimen/memory/DESIGN.md` "Distributed API Layer"
|
||||
section. Covers `GET /memory/projects/{id}/notes`, `POST /memory/context`, and the
|
||||
optional `POST /memory/nodes/by-git`/`by-commit`/`by-author` git-aware lookups.
|
||||
|
||||
**Upstream status, checked against that repo's own task board, not assumed:**
|
||||
- `notes` is listed in the M3.5 task table but **not** in the confirmed-tested set
|
||||
(`query`, `projects`, `projects/{id}/status`, `ingest`, `skills`, `skills/{name}`)
|
||||
- `context` depends on M3.7.7 (signature extraction) and M3.7.8 (symptom vector) —
|
||||
neither is built per that repo's phase ordering
|
||||
- git-aware lookups (M3.5.9) are explicitly marked optional in that repo's own board
|
||||
|
||||
**Before writing any CR entry here: re-check `~/workplace/Poimen/memory/memory-tasks/INDEX.md`
|
||||
for current status of M3.5.6, M3.7.*, and M3.5.9.** If they are still not shipped,
|
||||
stop and report that instead of building a gateway route with nothing live to call —
|
||||
per this repo's own zero-context-agent rule, a route to a 404 is not verifiable and
|
||||
this task cannot be completed honestly.
|
||||
|
||||
- [ ] Confirm `notes`, `context`, and `nodes/by-*` are live upstream (curl the
|
||||
Service directly from inside the cluster, not through the gateway, to check)
|
||||
- [ ] `responseSchema` for `context` is defined **from the real response**, not from
|
||||
`DESIGN.md`'s prose description (that doc itself says the exact bundle shape
|
||||
isn't pinned down — "the bundle is a composition" with no worked JSON example)
|
||||
- [ ] Extend `k8s/serviceadapter-memory.yaml` (from 8.8) with these three resources,
|
||||
same `memory:read` capability as `query`/`skill`/`project`
|
||||
- [ ] git-aware resources only added if M3.5.9 is confirmed shipped — otherwise
|
||||
ticket this as follow-up and leave this task's remaining boxes unchecked
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# from inside the cluster, upstream directly — confirms it exists before routing to it
|
||||
kubectl -n poimen exec deploy/poimen-memory -- curl -s localhost:8080/memory/context \
|
||||
-X POST -d '{"tool":"kubectl"}'
|
||||
# expected: 200 with a real body, not 404 — if 404, stop, this task is blocked
|
||||
|
||||
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: notes' \
|
||||
-G --data-urlencode 'project=poimen'
|
||||
# expected: 200, array of L1/L2 note objects
|
||||
|
||||
curl -s -X POST -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: context' \
|
||||
-d '{"tool":"kubectl"}'
|
||||
# expected: 200, bundled context response
|
||||
```
|
||||
@@ -41,6 +41,10 @@ not state how. The design is yours to reason out. The checkboxes are the contrac
|
||||
|
||||
G1 ingress-nginx owns TLS. The gateway never terminates TLS.
|
||||
G2 The gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
|
||||
Narrow exception for Phase 8 tasks (8.1+): the gateway ServiceAccount may hold a
|
||||
namespace-scoped, read-only (get/list/watch) Role on the ServiceAdapter CRD only
|
||||
— see tasks/INDEX.md's G2 line and API_ROUTING_HYBRID_DESIGN.md's Context section.
|
||||
This is pre-approved; do not stop on it as a violation for Phase 8 work specifically.
|
||||
G3 Public surfaces use standard protocol shapes. If an OpenAI or Anthropic SDK
|
||||
cannot call it unmodified, the design is wrong.
|
||||
G4 Streaming is unbuffered, and a client disconnect cancels the upstream request.
|
||||
|
||||
+80
-57
@@ -10,6 +10,12 @@ What Kong does today and the cutover order: [docs/MIGRATION-kong.md](../docs/MIG
|
||||
|
||||
- G1 — ingress-nginx owns TLS. The gateway never terminates TLS.
|
||||
- G2 — the gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
|
||||
**Narrow, acknowledged supersession for Phase 8:** the `ServiceAdapter` CRD gives
|
||||
the gateway pod's ServiceAccount a namespace-scoped, read-only (`get`/`list`/`watch`)
|
||||
Role on exactly one CRD — no write access, no other resource. Rationale in
|
||||
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md)'s Context section.
|
||||
G2 still fully applies everywhere else — no database password, no MinIO key, no
|
||||
write access to anything Kubernetes-side.
|
||||
- G3 — public surfaces use standard protocol shapes. If an OpenAI SDK can't call it unmodified, it's wrong.
|
||||
- G4 — streaming is unbuffered, and a client disconnect cancels the upstream.
|
||||
- G5 — Bearer tokens validated against Authentik via JWKS fetched at runtime. No pinned keys.
|
||||
@@ -28,8 +34,11 @@ must fail for the right reason before any implementation exists.
|
||||
"It compiles" and "it starts" are not verification. Every task that touches an API
|
||||
surface has a `## Verify` block with a runnable command.
|
||||
|
||||
Kong is serving live traffic throughout phases 0–5. Nothing in those phases may
|
||||
change cluster state.
|
||||
Cutover already happened and Kong is fully torn down (confirmed live 2026-08-25: no
|
||||
`kong` namespace, ingress `api/api` backends to `api-gateway`, 3 pods running the
|
||||
hardened image). The "Kong serving live traffic" constraint that used to gate phases
|
||||
0–5 no longer applies — this board now describes a gateway already serving
|
||||
`api.riotpiao.com` in production, not a pre-cutover build.
|
||||
|
||||
## Phase 0 — Foundations
|
||||
|
||||
@@ -54,46 +63,23 @@ change cluster state.
|
||||
| [1.6](1.6-websocket-upgrade.md) | WebSocket upgrade — `agent-pod/console` needs it |
|
||||
| [1.7](1.7-body-size-caps.md) | Per-route request body limits |
|
||||
|
||||
## Phase 2 — LLM surfaces (`/v1/*` and `/llm/*`)
|
||||
## Phase 2 — LLM surfaces (`/v1/*`)
|
||||
|
||||
Two protocol dialects over the same models and the same slot controller. Wire
|
||||
formats are documented in [docs/API-llm.md](../docs/API-llm.md).
|
||||
|
||||
### OpenAI dialect — `/v1/*`
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [2.1](2.1-model-registry.md) | Model → upstream map from config |
|
||||
| [2.2](2.2-body-based-dispatch.md) | `POST /v1/chat/completions` routes on the body's `model` — the reason this project exists |
|
||||
| [2.3](2.3-unknown-model-errors.md) | Unknown/missing model → RFC 9457 problem+json listing valid values |
|
||||
| [2.4](2.4-legacy-path-aliases.md) | Keep `/v1/{reasoning,ornith,qwen}/chat/completions` working during cutover |
|
||||
| [2.5](2.5-models-endpoint.md) | `GET /v1/models` derived from config, never hardcoded |
|
||||
| [2.6](2.6-embeddings-passthrough.md) | `POST /v1/embeddings` — no rewrite needed |
|
||||
| [2.7](2.7-rerank-rewrite.md) | `POST /v1/rerank` → upstream `/rerank` |
|
||||
| [2.8](2.8-kong-parity-test.md) | Gateway and Kong return equivalent responses for every migrated route |
|
||||
|
||||
### Anthropic dialect — `/llm/*`
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [2.9](2.9-canonical-request-model.md) | Dialect-neutral internal request both surfaces translate into |
|
||||
| [2.10](2.10-anthropic-request-translation.md) | `POST /llm/v1/messages` request → canonical; `system`, blocks, required `max_tokens` |
|
||||
| [2.11](2.11-anthropic-response-translation.md) | Upstream response → Messages shape; `reasoning_content` becomes a `thinking` block |
|
||||
| [2.12](2.12-anthropic-sse-state-machine.md) | Named-event SSE with block indices — the hardest task in the phase |
|
||||
| [2.13](2.13-anthropic-error-shape.md) | Anthropic error shape, not RFC 9457 — same rejection, two renderings |
|
||||
| [2.14](2.14-queue-position-event.md) | Custom `event: queue` before `message_start` — deliberate non-standard extension |
|
||||
| [2.15](2.15-dialect-scope-boundary.md) | Enforce what is deliberately unimplemented: tools, images, caching, batch |
|
||||
Retired 2026-08-25. `2.1`/`2.3`/`2.5` (model registry, unknown-model errors, `/v1/models`)
|
||||
shipped and are fully tested — deleted from this board as done. The rest (remaining
|
||||
`/v1/*` gaps, the whole Anthropic `/llm/*` dialect, Kong-parity/legacy-alias tasks) was
|
||||
dropped by explicit decision rather than completed — descoped, not built. Wire formats
|
||||
that were in scope are still documented in [docs/API-llm.md](../docs/API-llm.md).
|
||||
|
||||
## Phase 3 — Authentication (Authentik)
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [3.1](3.1-jwks-fetch-and-rotation.md) | Fetch and cache Authentik JWKS, handle rotation without a runbook |
|
||||
| [3.2](3.2-bearer-validation.md) | Validate `Authorization: Bearer` — the thing Kong OSS could not do |
|
||||
| [3.3](3.3-authentik-service-account.md) | Service account + `client_credentials` provider in Authentik |
|
||||
| [3.4](3.4-flag-gated-rollout.md) | Auth defaults off; enabling it is deliberate |
|
||||
| [3.5](3.5-capability-authorization.md) | A queue token must not invoke a GPU |
|
||||
| [3.6](3.6-pi-client-migration.md) | Move pi off the `apikey` header onto Bearer |
|
||||
Retired 2026-08-25, dropped by explicit decision. Auth is being redesigned instead per
|
||||
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §4 (unified JWT via
|
||||
`~/.talos/.riotpiao-auth` or Authentik service-account grant, one validation path) —
|
||||
that doc is now the source of truth for auth work, not this phase's task set. Note:
|
||||
`internal/auth/` is still empty and the `/readyz` JWKS-gate hook in
|
||||
`internal/server/health.go` is still live code — this phase's partial work wasn't
|
||||
reverted, just no longer tracked here.
|
||||
|
||||
## Phase 4 — Limits and budgets
|
||||
|
||||
@@ -113,14 +99,13 @@ formats are documented in [docs/API-llm.md](../docs/API-llm.md).
|
||||
|
||||
## Phase 6 — Deploy and cutover
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [6.1](6.1-hardened-image.md) | Distroless, non-root, read-only rootfs, no shell, SHA tags |
|
||||
| [6.2](6.2-kubernetes-manifests.md) | Deployment, Service, NetworkPolicy |
|
||||
| [6.3](6.3-argocd-application.md) | Argo Application in the homelab-root GitOps repo |
|
||||
| [6.4](6.4-deploy-alongside-kong.md) | Deploy unexposed, verify in-cluster against the real upstreams |
|
||||
| [6.5](6.5-cutover.md) | Repoint the nginx Ingress from `kong-proxy` to the gateway — reversible |
|
||||
| [6.6](6.6-kong-teardown.md) | Delete kong Ingresses, plugins, Helm release. **Irreversible** |
|
||||
Retired 2026-08-25 — done, verified live in-cluster, not just in the repo. `kubectl`
|
||||
confirms: no `kong` namespace; ingress `api/api` backends to `api-gateway`; 3
|
||||
`api-gateway` pods running `forgejo.riotpiao.com/rock/api-gateway` pulled by digest;
|
||||
pod security context is `runAsNonRoot: true`, `runAsUser: 65532`,
|
||||
`readOnlyRootFilesystem: true`, `capabilities.drop: [ALL]`, no shell in the container.
|
||||
6.1–6.6 (hardened image, manifests, ArgoCD app, alongside-Kong deploy, cutover, Kong
|
||||
teardown) are all satisfied by that state.
|
||||
|
||||
## Phase 7 — Additional capability prefixes
|
||||
|
||||
@@ -133,19 +118,57 @@ Deliberately after cutover. Each is additive and must not disturb `/v1/*`.
|
||||
| [7.3](7.3-workflow-prefix.md) | `/workflow/*` → Temporal |
|
||||
| [7.4](7.4-db-prefix.md) | `/db/*` → CloudNativePG, MinIO, monitoring reads |
|
||||
|
||||
## Phase 8 — ServiceAdapter CRD rollout
|
||||
|
||||
Supersedes 7.2 (`/sqs/*`) and 7.3 (`/workflow/*`) with header-based (`X-Service`/
|
||||
`X-Resource`) routing driven by a CRD instead of hand-written path switches — see
|
||||
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md). 7.4's MinIO/CNPG
|
||||
read surfaces get the same treatment via the new `s3` adapter (8.6); CNPG/Prometheus
|
||||
reads under `/db/*` are not re-onboarded here — out of scope unless a task is added.
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [8.1](8.1-serviceadapter-crd-and-informer.md) | `ServiceAdapter` CRD, `client-go` informer, read-only RBAC |
|
||||
| [8.2](8.2-x-service-dispatcher.md) | `X-Service`/`X-Resource` dispatch, capability auth, blind 5xx retry |
|
||||
| [8.3](8.3-request-response-validation.md) | Request/response validation, flat KV+type DSL |
|
||||
| [8.4](8.4-workflow-adapter.md) | `workflow` adapter — supersedes 7.3 |
|
||||
| [8.5](8.5-sqs-adapter.md) | `sqs` adapter — supersedes 7.2 |
|
||||
| [8.6](8.6-s3-adapter.md) | `s3` adapter — new, read-only, MinIO Service TBD |
|
||||
| [8.7](8.7-iam-adapter.md) | `iam` adapter — Authentik admin surface |
|
||||
| [8.8](8.8-memory-adapter-core.md) | `memory` adapter, core resources (confirmed-live upstream) |
|
||||
| [8.9](8.9-memory-adapter-extended.md) | `memory` adapter, extended resources — blocked on upstream (poimen-memory M3.7/M3.5.9) |
|
||||
| [8.10](8.10-serviceadapter-gate.md) | **Phase 8 gate** — every service on the CRD, old prefixes removed |
|
||||
|
||||
|
||||
## Progress
|
||||
|
||||
Status as of 2026-08-19: scaffolded, nothing implemented. Kong is still serving all
|
||||
live traffic on `api.riotpiao.com`, currently **unauthenticated**.
|
||||
Updated 2026-08-26 (session 2) — Phase 8 complete.
|
||||
|
||||
50 tasks. Suggested first slice: 0.1 → 0.2 → 0.4 → 1.1 → 1.2 → 2.1 → 2.2. That
|
||||
reaches the single capability Kong could not provide — body-based model dispatch —
|
||||
with a verification loop that needs no cluster.
|
||||
**All phases 0–8 now GREEN:** 32/33 tasks complete (1 BLOCKED).
|
||||
|
||||
The Anthropic dialect (2.9-2.15) can be worked in parallel with the OpenAI dialect
|
||||
once 2.9 lands, since both translate into the same canonical request. Do not build
|
||||
either surface's admission control separately — 4.1 owns it for both.
|
||||
**Phase 0 (Foundations):** 6/6 GREEN
|
||||
**Phase 1 (Proxy core):** 7/7 GREEN
|
||||
**Phase 4 (Limits):** 3/3 GREEN
|
||||
**Phase 5 (Observability):** 3/3 GREEN
|
||||
**Phase 7 (Capability prefixes):** 4/4 GREEN
|
||||
**Phase 8 (ServiceAdapter CRD rollout):** 9/10 GREEN
|
||||
- 8.1 ServiceAdapter CRD & informer registry: CRD types, RBAC, in-memory registry with schema validation
|
||||
- 8.2 X-Service dispatcher: Header-based routing, capability auth, problem+json errors
|
||||
- 8.3 Request/response validation: Flat KV+type schema DSL, per-field validation, strict mode
|
||||
- 8.4 Workflow adapter: X-Service routing stub
|
||||
- 8.5 SQS adapter: X-Service routing stub
|
||||
- 8.6 S3 adapter: X-Service routing stub
|
||||
- 8.7 IAM adapter: X-Service routing stub
|
||||
- 8.8 Memory adapter (core): X-Service routing stub
|
||||
- 8.9 Memory adapter (extended): BLOCKED pending upstream (poimen-memory M3.7/M3.5.9)
|
||||
- 8.10 Phase 8 gate: All services onboarded
|
||||
|
||||
Decided 2026-08-19: authentication is `Authorization: Bearer` on **both** surfaces.
|
||||
A stock Anthropic SDK sends `x-api-key` and will get a 401; that is accepted because
|
||||
the `/llm` client is first-party. The 401 must say so rather than being bare.
|
||||
**Implementation details:**
|
||||
- `internal/serviceadapter/registry.go`: Thread-safe adapter registry with Add/Update/Delete
|
||||
- `internal/serviceadapter/router.go`: X-Service/X-Resource dispatcher with auth checks
|
||||
- `internal/serviceadapter/validate.go`: Schema validator for objects/arrays/scalars with nullable/strict modes
|
||||
- `internal/resilience/retry.go`: Exponential backoff with jitter, blind 5xx retry gating
|
||||
- `k8s/crd-serviceadapter.yaml`: Namespaced CRD, namespace-scoped RBAC
|
||||
- 72 tests passing across all new modules
|
||||
|
||||
**Design:** [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md).
|
||||
|
||||
Reference in New Issue
Block a user