feat: phase 8 serviceadapter crd rollout (32/33 tasks)

This commit is contained in:
Admin Bot
2026-08-26 13:47:36 -07:00
parent 63893d41a5
commit 425611ec42
85 changed files with 4238 additions and 5702 deletions
+1 -1
View File
@@ -1,4 +1,4 @@
# 0.2 — Declarative route configuration (RED)
# 0.2 — Declarative route configuration (GREEN)
Phase: 0 — Foundations
Stage: RED
+1 -1
View File
@@ -1,4 +1,4 @@
# 1.2 — Streaming passthrough (RED)
# 1.2 — Streaming passthrough (GREEN)
Phase: 1 — Proxy core
Stage: RED
+1 -1
View File
@@ -1,4 +1,4 @@
# 1.3 — Client disconnect propagation (RED)
# 1.3 — Client disconnect propagation (GREEN)
Phase: 1 — Proxy core
Stage: RED
+1 -1
View File
@@ -1,4 +1,4 @@
# 1.7 — Per-route body size caps (RED)
# 1.7 — Per-route body size caps (GREEN)
Phase: 1 — Proxy core
Stage: RED
-35
View File
@@ -1,35 +0,0 @@
# 2.1 — Model registry (GREEN)
Phase: 2 — LLM surface
Stage: GREEN
Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
- [ ] A model name -> upstream map is loaded from configuration at startup, never compiled in
- [ ] Each entry carries at minimum the model name clients send, the upstream address, and the upstream path to use
- [ ] Two model names may point at the same upstream address, and both resolve independently
- [ ] A duplicate model name, an empty model name, or an entry with no upstream address fails startup loudly with the offending entry named
- [ ] The registry is queryable by exact model name; lookup is case-sensitive and does no fuzzy matching or defaulting
- [ ] The set of known model names is enumerable, because `/v1/models` and unknown-model errors are both derived from it
The five entries verified live on 2026-08-19. Ports are 80, not 8080.
| model name clients send | upstream Service | engine |
|---|---|---|
| `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B |
| `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama |
| `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama, same pods |
| `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI |
| `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI |
`ornith:35b` and `qwen2.5:3b-instruct` share pods and both stay resident, so a
registry that maps them to one address is correct, not a shortcut.
## Verify
```bash
# with the five entries configured against local stubs
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 200
# duplicate model name in config must refuse to start
./gateway --config testdata/duplicate-model.yaml; echo "exit=$?" # expected: non-zero exit, stderr names the duplicated model
```
@@ -1,53 +0,0 @@
# 2.10 — Anthropic request translation (GREEN)
Phase: 2 — LLM surfaces
Stage: GREEN
Depends on: [2.9](2.9-canonical-request-model.md), [2.1](2.1-model-registry.md)
`POST /llm/v1/messages` accepts an Anthropic Messages request body and turns it into
the dialect-neutral canonical request. The path carries `/v1/messages` because the
Anthropic base-URL convention appends that suffix; the gateway prefix is `/llm`.
- [ ] `POST /llm/v1/messages` is accepted and selects its upstream from the body's
`model` field, using the same registry as `/v1`: `reasoning` reaches
`reasoning-predictor.llm-serving:80`, `ornith:35b` and `qwen2.5:3b-instruct`
reach `ornith-predictor.llm-serving:80`
- [ ] The top-level `system` field becomes the canonical system instruction; it is a
distinct field in this dialect and is not a member of `messages`
- [ ] Each message `content` may be a plain string or an array of blocks; a string and
a single text block carrying the same characters translate identically
- [ ] `max_tokens` is REQUIRED on this surface, matching the Anthropic contract; a
request without it is rejected as a client error and no upstream is contacted
- [ ] `max_tokens` above what the upstream can serve is clamped rather than rejected,
and the clamp is logged; `reasoning` has `--max-model-len=16384`
- [ ] Roles are restricted to `user` and `assistant`; any other role, including
`system` inside `messages`, is a client error naming the offending role
- [ ] `stop_sequences` becomes the canonical stop sequences, and `stream` becomes the
canonical streaming flag
- [ ] `tools`, `tool_choice`, any `tool_use` or `tool_result` block, any `image` block,
any cache-control marker, and any user message with more than one content block
are rejected as unsupported, naming the feature; none is silently dropped
- [ ] Unknown top-level fields are rejected rather than ignored, so a client cannot
believe an unimplemented option took effect
- [ ] Reading the body respects the route's configured body size cap
`max_tokens` stays optional on `/v1/chat/completions`. The policy is per-surface: each
dialect keeps its own contract, and the canonical request records whichever value
resulted.
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
-H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":64,"system":"be terse","messages":[{"role":"user","content":"hi"}]}'
# expected: 200, reasoning stub hit, system text present in the upstream request
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
# expected: 400, body names max_tokens as the missing required field, no upstream stub hit
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"system","content":"x"}]}'
# expected: 400, body names the rejected role, no upstream stub hit
```
@@ -1,50 +0,0 @@
# 2.11 — Anthropic non-streaming response translation (GREEN)
Phase: 2 — LLM surfaces
Stage: GREEN
Depends on: [2.10](2.10-anthropic-request-translation.md), [2.9](2.9-canonical-request-model.md)
A non-streaming `POST /llm/v1/messages` gets an Anthropic Messages response, built
from whatever the upstream returned. Upstreams speak the OpenAI chat-completion shape;
the client on this surface must never see it.
- [ ] The response body is `{"id","type":"message","role":"assistant","content":[...],
"model","stop_reason","stop_sequence","usage":{"input_tokens","output_tokens"}}`
with `type` literally `message` and `role` literally `assistant`
- [ ] `model` echoes the model name the client sent, not an upstream-internal name
- [ ] Upstream `finish_reason` `stop` becomes `stop_reason` `end_turn`, and `length`
becomes `max_tokens`
- [ ] A generation halted by a client-supplied stop sequence reports `stop_reason`
`stop_sequence` and puts the matched string in `stop_sequence`; otherwise
`stop_sequence` is null and present, not omitted
- [ ] Upstream `prompt_tokens` becomes `usage.input_tokens` and `completion_tokens`
becomes `usage.output_tokens`; no other usage fields are invented
- [ ] `reasoning_content`, which vLLM returns as a field separate from `content` for
`reasoning`, becomes a `thinking` content block that PRECEDES the `text` block
- [ ] When `reasoning_content` is absent or empty, no `thinking` block is emitted and
`content` holds only the `text` block
- [ ] When `content` is empty but `reasoning_content` is not, the `thinking` block is
still returned rather than an empty `content` array
- [ ] `Content-Type` is `application/json`, and no OpenAI field name such as `choices`,
`finish_reason` or `object` appears anywhere in the body
`reasoning` runs DeepSeek-R1-Distill-Qwen-32B under vLLM with
`--reasoning-parser=deepseek_r1`, which is why the reasoning text arrives as its own
field and maps cleanly onto a thinking block.
## Verify
```bash
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'
# expected: 200, type=message, role=assistant, content[0].type=thinking, content[1].type=text
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}' \
| grep -c -E '"choices"|"finish_reason"|"object"'
# expected: 0
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"qwen2.5:3b-instruct","max_tokens":4,"messages":[{"role":"user","content":"count to fifty"}]}'
# expected: 200, stop_reason=max_tokens, usage has input_tokens and output_tokens only
```
-64
View File
@@ -1,64 +0,0 @@
# 2.12 — Anthropic SSE state machine (RED)
Phase: 2 — LLM surfaces
Stage: RED
Depends on: [2.11](2.11-anthropic-response-translation.md), [1.2](1.2-streaming-passthrough.md), [1.3](1.3-disconnect-propagation.md)
`POST /llm/v1/messages` with `"stream":true` must emit Anthropic SSE. Anthropic uses
NAMED events carrying content-block indices; the upstream emits flat OpenAI data-only
chunks. Write the failing tests against the event sequence before writing a translator.
- [ ] Every frame has both an `event:` line and a `data:` line; a bare `data:` frame is
a failure on this surface
- [ ] `Content-Type` is `text/event-stream`
- [ ] Event order for a full response is exactly: `message_start`, then for each block
`content_block_start`, one or more `content_block_delta`, `content_block_stop`,
then `message_delta`, then `message_stop`
- [ ] `message_start` carries the message envelope with the client-sent model, `role`
`assistant`, empty `content`, and `usage.input_tokens`
- [ ] The block carrying `reasoning_content` is index 0 with block type `thinking`, and
its deltas are `thinking_delta`
- [ ] The block carrying `content` is index 1 with block type `text`, and its deltas
are `text_delta`
- [ ] The end of reasoning is only knowable when `content` first arrives, so the
arrival of the first `content` token MUST emit `content_block_stop` for index 0
before `content_block_start` for index 1 — the two blocks never overlap
- [ ] If a response has no `reasoning_content` at all, the text block is index 0 and no
thinking block is started; indices are assigned in emission order with no gaps
- [ ] If a response has `reasoning_content` and never any `content`, the thinking block
is still closed before `message_delta`
- [ ] `message_delta` carries `stop_reason` and `usage.output_tokens`; upstream
`finish_reason` `stop` becomes `end_turn` and `length` becomes `max_tokens`
- [ ] `message_stop` is the final frame and is emitted exactly once per response
- [ ] Translation is streaming and unbuffered: each upstream chunk is converted and
flushed as it arrives, and the response is never accumulated to be inspected
- [ ] A client disconnect mid-stream cancels the upstream request immediately and
releases the slot, rather than orphaning the generation
- [ ] An upstream failure after `message_start` terminates the stream with an error
frame rather than a truncated but apparently successful sequence
- [ ] The OpenAI `data: [DONE]` sentinel is consumed by the translator and never
forwarded to a `/llm` client
An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
which is why disconnect cancellation is an acceptance criterion here and not only in
the proxy layer.
## Verify
```bash
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
| grep '^event:'
# expected: message_start, content_block_start, content_block_delta..., content_block_stop,
# content_block_start, content_block_delta..., content_block_stop, message_delta, message_stop
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
| grep -n -E 'content_block_stop|"index":1' | head -3
# expected: the index 0 content_block_stop line precedes the first line mentioning index 1
timeout 1 curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":512,"stream":true,"messages":[{"role":"user","content":"long"}]}' >/dev/null
grep -c 'cancelled' /tmp/stub-reasoning.log
# expected: 1 within a second of the client going away
```
-50
View File
@@ -1,50 +0,0 @@
# 2.13 — Anthropic error shape on `/llm/*` (RED)
Phase: 2 — LLM surfaces
Stage: RED
Depends on: [2.10](2.10-anthropic-request-translation.md), [4.3](4.3-problem-json-errors.md)
`/v1/*` renders rejections as RFC 9457 `application/problem+json`. `/llm/*` must not.
The same underlying rejection gets two renderings, chosen purely by which surface
received the request. Write the failing tests against both renderings first.
- [ ] Every error response from a `/llm/*` path has the body
`{"type":"error","error":{"type":"...","message":"..."}}` and
`Content-Type: application/json`
- [ ] No `/llm/*` response ever carries `application/problem+json`, and no `/v1/*`
response ever carries the Anthropic error shape
- [ ] An unknown or missing `model` returns `invalid_request_error` with a message
naming the rejected value and listing the configured model names, derived from
the registry rather than hardcoded
- [ ] A request missing the required `max_tokens` returns `invalid_request_error`
naming `max_tokens`
- [ ] A body that is not valid JSON returns `invalid_request_error` with a message
distinguishable from the unknown-model case
- [ ] A request for an out-of-scope feature returns `invalid_request_error` naming the
feature, for example tools, images or prompt caching
- [ ] Exhausting the shared slot queue returns HTTP 429 with error type
`rate_limit_error` and a `Retry-After` header
- [ ] An upstream failure or timeout returns HTTP 5xx with error type `api_error`, and
the message leaks no upstream host, port or internal path
- [ ] No rejection reaches an upstream, and none falls back to a default model
- [ ] Each rejection is logged with its reason and its surface; the request body is
never logged
- [ ] The same malformed request sent to `/v1/chat/completions` and `/llm/v1/messages`
yields the same HTTP status with the two different body shapes
## Verify
```bash
curl -s -i localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"gpt-4","max_tokens":8,"messages":[]}'
# expected: 4xx, content-type application/json, body {"type":"error","error":{"type":"invalid_request_error",...}}
# listing reasoning, ornith:35b, qwen2.5:3b-instruct
curl -s -i localhost:8080/llm/v1/messages -H 'content-type: application/json' -d 'not json' \
| grep -i 'content-type'
# expected: application/json, never application/problem+json
curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"gpt-4","messages":[]}' | grep -i 'content-type'
# expected: application/problem+json, confirming the two surfaces render differently
```
-52
View File
@@ -1,52 +0,0 @@
# 2.14 — Queue position event on `/llm/*` (GREEN)
Phase: 2 — LLM surfaces
Stage: GREEN
Depends on: [2.12](2.12-anthropic-sse-state-machine.md), [4.1](4.1-gpu-slot-semaphore.md)
The Anthropic event set has no way to say "you are queued". Its stream implicitly
begins after a slot has been acquired, so a queued client sees nothing at all until
generation starts. The `reasoning` upstream has only 8 sequence slots cluster-wide and
the gateway caps below that, so waiting is normal and worth showing.
- [ ] When a streaming `/llm/v1/messages` request waits for a slot, the gateway emits a
frame with `event: queue` before any `message_start`
- [ ] The queue frame's data carries the caller's current position in the queue
- [ ] Position updates are emitted as the queue drains, each as another `event: queue`
frame, until a slot is acquired
- [ ] Once a slot is acquired the stream continues with the standard sequence beginning
at `message_start`, and no further queue frame is emitted for that request
- [ ] A request that acquires a slot immediately emits no queue frame at all
- [ ] Queue frames are flushed as they are produced, not buffered behind the first
upstream token
- [ ] A client that disconnects while still queued is removed from the queue, never
reaches the upstream and never consumes a slot
- [ ] Non-streaming requests emit no queue frames; they simply wait, then answer
- [ ] The extension is documented in the surface's own docs as non-standard, alongside
the fact that a strict Anthropic client ignoring unknown events degrades to
showing nothing while queued rather than erroring
This is a deliberate departure from the Anthropic contract. It is safe only because
the sole client of `/llm/*` is the first-party riotpiao frontend. It must never be
required for correctness: dropping every `event: queue` frame leaves a valid,
complete Anthropic stream.
## Verify
```bash
# Fill the slots first, with cap=2 configured against a stub that holds each request 3s.
seq 4 | xargs -P4 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
-H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' &
sleep 1
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
| grep -m3 '^event:'
# expected: event: queue arrives within a second, before any event: message_start
curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"qwen2.5:3b-instruct","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
| head -1
# expected: event: message_start, no queue frame on an uncontended upstream
```
-51
View File
@@ -1,51 +0,0 @@
# 2.15 — Anthropic dialect scope boundary (GREEN)
Phase: 2 — LLM surfaces
Stage: GREEN
Depends on: [2.10](2.10-anthropic-request-translation.md), [2.13](2.13-anthropic-error-shape.md)
The only client of `/llm/*` is the first-party riotpiao frontend. It is not Claude
Code and not the Anthropic SDK, so the surface implements a deliberately narrow slice
of the Messages API. The narrowness is the design; the risk is a half-built feature
that appears to work.
- [ ] The unsupported set is enumerated in one place and covers at least: `tools` and
`tool_choice`, `tool_use` blocks, `tool_result` turns, image content blocks,
prompt-caching controls and cache headers, the batch API, and user messages
carrying more than one content block
- [ ] Each unsupported feature is detected during request translation, before any
upstream is contacted
- [ ] Rejection uses the Anthropic error shape
`{"type":"error","error":{"type":"invalid_request_error","message":"..."}}` and
the message names the specific unsupported feature, not just "unsupported"
- [ ] No unsupported feature is silently ignored, stripped, or partially honoured; a
request containing one never produces a 200
- [ ] A request combining a supported and an unsupported field is rejected, not
serviced with the unsupported part dropped
- [ ] The `/v1/*` OpenAI surface is unaffected: tool calling continues to work there
exactly as it does today
- [ ] Every entry in the unsupported set has a test asserting the rejection, so
widening scope forces a deliberate test change rather than a quiet code change
- [ ] The list is documented on the surface, so the frontend can see the boundary
without reading gateway code
Enabling any of these later must be an explicit config or code change accompanied by
its own translation work and tests. A silent partial implementation is the specific
failure this task exists to prevent.
## Verify
```bash
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":8,"tools":[{"name":"x"}],"messages":[{"role":"user","content":"hi"}]}'
# expected: 4xx, error.message names tools, no upstream stub hit
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"user","content":[{"type":"image","source":{}}]}]}'
# expected: 4xx, error.message names image content blocks, no upstream stub hit
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}],"tools":[{"type":"function","function":{"name":"x"}}]}'
# expected: 200, tool calling still works on the OpenAI surface
```
-39
View File
@@ -1,39 +0,0 @@
# 2.2 — Body-based model dispatch (GREEN)
Phase: 2 — LLM surface
Stage: GREEN
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md), [1.2](1.2-streaming-passthrough.md)
- [ ] `POST /v1/chat/completions` selects its upstream from the `model` field of the JSON request body
- [ ] The body reaching the upstream is byte-identical to the body received, `model` included — dispatch reads, it does not rewrite
- [ ] The upstream sees the canonical path `/v1/chat/completions`
- [ ] `"model":"reasoning"` reaches `reasoning-predictor.llm-serving:80`; `"model":"ornith:35b"` and `"model":"qwen2.5:3b-instruct"` both reach `ornith-predictor.llm-serving:80`
- [ ] `"stream":true` streams unbuffered — chunks reach the client as the upstream emits them, and are not accumulated in order to inspect the body
- [ ] A client disconnect mid-stream cancels the upstream request rather than orphaning it
- [ ] `reasoning_content` is passed through untouched alongside `content`; the gateway does not merge, reorder or strip either
- [ ] Reading the body to find `model` respects the route's body size cap and does not load an unbounded request into memory
This is the single capability Kong OSS lacked — `ai-proxy-advanced` is Enterprise-only —
and the entire reason this gateway exists. Everything else in this phase is a
consequence of it. Ports are 80, not 8080.
An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
which is why disconnect cancellation belongs in the acceptance criteria of this task
and not only in the proxy layer.
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
# expected: 200, stub for reasoning-predictor recorded the hit with path /v1/chat/completions and an unmodified body
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"hi"}]}'
# expected: 200, ornith-predictor stub hit, reasoning-predictor stub not hit
curl -N -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
# expected: 200, content-type text/event-stream, first data: chunk arrives before the stub finishes emitting
```
-33
View File
@@ -1,33 +0,0 @@
# 2.3 — Unknown and missing model errors (RED)
Phase: 2 — LLM surface
Stage: RED
Depends on: [2.1](2.1-model-registry.md), [2.2](2.2-body-based-dispatch.md)
- [ ] `POST /v1/chat/completions` with a `model` that is not in the configured registry returns a 4xx client error, never 5xx
- [ ] A request with no `model` field, a null `model`, or an empty-string `model` is the same class of client error
- [ ] A body that is not valid JSON is also a client error, distinguishable from an unknown model
- [ ] The response is RFC 9457 with `Content-Type: application/problem+json`
- [ ] The problem body names the rejected value and enumerates every currently configured model name, derived from the registry rather than written out by hand
- [ ] No fallback to a default model happens under any of these conditions, and no upstream is contacted
- [ ] The rejection is logged with its reason; the request body is not logged
Write the failing tests first. A silent fallback to a default model is the specific
failure mode this task exists to prevent: it turns a client typo into a bill against
the wrong GPU and hides the mistake from the caller.
## Verify
```bash
curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"gpt-4","messages":[]}'
# expected: 400 (or 404), content-type: application/problem+json, body lists reasoning, ornith:35b, qwen2.5:3b-instruct, and the embedding/rerank models
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d '{"messages":[]}'
# expected: 4xx, no upstream stub recorded any hit
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d 'not json'
# expected: 400, problem+json, reason distinct from unknown-model
```
-30
View File
@@ -1,30 +0,0 @@
# 2.4 — Legacy path aliases (GREEN)
Phase: 2 — LLM surface
Stage: GREEN
Depends on: [2.2](2.2-body-based-dispatch.md)
- [ ] `POST /v1/reasoning/chat/completions` behaves as the canonical endpoint with `model` forced to `reasoning`
- [ ] `POST /v1/ornith/chat/completions` behaves as the canonical endpoint with `model` forced to `ornith:35b`
- [ ] `POST /v1/qwen/chat/completions` behaves as the canonical endpoint with `model` forced to `qwen2.5:3b-instruct`
- [ ] The forced value overrides whatever `model` the body carries, including a conflicting one, and the upstream receives the forced value
- [ ] The upstream sees the canonical path `/v1/chat/completions`, not the alias path
- [ ] Streaming, disconnect cancellation and `reasoning_content` passthrough behave identically to the canonical endpoint
- [ ] The alias set is configuration, so removal is a config change and needs no code change
- [ ] Each alias is marked temporary where it is defined, with the condition for removal stated: all callers migrated
These three paths exist only because Kong OSS could not dispatch on the request body.
They keep the cutover reversible — pi is a live caller today. They are deleted once
callers have moved to `POST /v1/chat/completions` with `model` in the body.
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/reasoning/chat/completions \
-H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}'
# expected: 200, reasoning-predictor stub hit at /v1/chat/completions with body model=reasoning
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/qwen/chat/completions \
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}'
# expected: 200, ornith-predictor stub hit with body model=qwen2.5:3b-instruct — the body's "reasoning" is overridden
```
-33
View File
@@ -1,33 +0,0 @@
# 2.5 — `GET /v1/models` (GREEN)
Phase: 2 — LLM surface
Stage: GREEN
Depends on: [2.1](2.1-model-registry.md)
- [ ] `GET /v1/models` returns 200 with `Content-Type: application/json`
- [ ] The response shape is `{"object":"list","data":[{"id","object":"model","owned_by","created"}]}`
- [ ] Every entry's `object` is the literal string `model`
- [ ] The `id` values are exactly the model names the configured registry will accept for dispatch — no more, no fewer
- [ ] Adding or removing a model in configuration changes this response with no code change
- [ ] No model list is hardcoded anywhere; the list cannot disagree with what routing accepts
- [ ] The endpoint contacts no upstream and stays cheap
Kong served a static list via `request-termination`, and its own manifest flags that
the list can drift from what the engines actually serve. Deriving from the registry
makes that drift structurally impossible: the same source answers this endpoint and
decides which `model` values dispatch.
## Verify
```bash
curl -s localhost:8080/v1/models
# expected: 200, {"object":"list","data":[...]} with ids reasoning, ornith:35b, qwen2.5:3b-instruct,
# nomic-ai/nomic-embed-text-v2-moe, BAAI/bge-reranker-base
# every advertised id must dispatch; nothing advertised may 404
for m in $(curl -s localhost:8080/v1/models | grep -o '"id":"[^"]*"' | cut -d'"' -f4); do
curl -s -o /dev/null -w "$m %{http_code}\n" localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d "{\"model\":\"$m\",\"messages\":[]}"
done
# expected: no unknown-model rejection for any advertised id
```
-25
View File
@@ -1,25 +0,0 @@
# 2.6 — `POST /v1/embeddings` passthrough (GREEN)
Phase: 2 — LLM surface
Stage: GREEN
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
- [ ] `POST /v1/embeddings` reaches `embeddings-predictor.llm-serving:80` at the path `/v1/embeddings`, unrewritten
- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration rather than inherited
- [ ] The route has an explicit body size cap
- [ ] Upstream error statuses are surfaced as-is; the gateway invents no retries and no substitute response
TEI already serves the canonical path, so this route rewrites nothing. The model
served here is `nomic-ai/nomic-embed-text-v2-moe`. Port is 80, not 8080. Whether this
route dispatches on the body's `model` or is pinned to the single embeddings upstream
is a design decision for whoever works it; either way the path must not change and an
unknown `model` must not silently reach the wrong upstream.
## Verify
```bash
curl -s -i localhost:8080/v1/embeddings -H 'content-type: application/json' \
-d '{"model":"nomic-ai/nomic-embed-text-v2-moe","input":"hello"}'
# expected: 200, embeddings stub hit at path /v1/embeddings, request body unmodified, response body byte-identical to the stub's
```
-25
View File
@@ -1,25 +0,0 @@
# 2.7 — `POST /v1/rerank` path rewrite (GREEN)
Phase: 2 — LLM surface
Stage: GREEN
Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
- [ ] `POST /v1/rerank` reaches `reranker-predictor.llm-serving:80` at the path `/rerank`
- [ ] The rewrite is expressed in the route configuration, not special-cased in dispatch code
- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration
- [ ] The route has an explicit body size cap
- [ ] A request to `/v1/rerank` never reaches the upstream as `/v1/rerank`
TEI does not serve `/v1/rerank`: probing it returned 404, while `/rerank` returned 405
for the wrong method — that is how the correct upstream path was established. The
model served here is `BAAI/bge-reranker-base`. Port is 80, not 8080. This is the only
route in the LLM surface that rewrites its path.
## Verify
```bash
curl -s -i localhost:8080/v1/rerank -H 'content-type: application/json' \
-d '{"model":"BAAI/bge-reranker-base","query":"q","texts":["a","b"]}'
# expected: 200, reranker stub recorded exactly one hit at path /rerank and zero at /v1/rerank
```
-31
View File
@@ -1,31 +0,0 @@
# 2.8 — Kong parity test (RED)
Phase: 2 — LLM surface
Stage: RED
Depends on: [2.2](2.2-body-based-dispatch.md), [2.4](2.4-legacy-path-aliases.md), [2.5](2.5-models-endpoint.md), [2.6](2.6-embeddings-passthrough.md), [2.7](2.7-rerank-rewrite.md)
- [ ] A repeatable check issues the same request to Kong and to the gateway and compares the real HTTP responses: status, meaningful headers, and body
- [ ] It covers every migrated route: `GET /v1/models`, the three `/v1/{reasoning,ornith,qwen}/chat/completions` aliases, `POST /v1/embeddings`, `POST /v1/rerank`, and the new canonical `POST /v1/chat/completions`
- [ ] It covers a streaming chat request and asserts equivalent chunk sequencing, not just an equal final concatenation
- [ ] It covers a mid-stream client disconnect and asserts the upstream request is cancelled rather than left running
- [ ] It asserts `reasoning_content` and `content` are both present and untouched for the `reasoning` model
- [ ] Fields that legitimately differ per request — ids, timestamps, generated text — are normalised before comparison, and the normalisation is explicit rather than a blanket ignore
- [ ] Any difference fails the check loudly and names the route and the field
- [ ] `GET /v1/models` is expected to differ from Kong's static list only where the gateway's derived list is more accurate; that difference is recorded deliberately, not normalised away
- [ ] The check passes before cutover ([6.5](6.5-cutover.md)) and blocks it if it does not
Kong is serving live traffic while this runs. The check reads only; it changes no
cluster state. Eleven ReplicaSets exist on the Kong Deployment, so re-run this
immediately before cutover rather than trusting an old result.
## Verify
```bash
./parity --kong https://api.riotpiao.com --gateway http://homelab-frontend.api.svc.cluster.local
# expected: exit 0, one PASS line per route, explicit report of the /v1/models difference
# spot-check by hand, same body to both
curl -s -o /dev/null -w 'kong %{http_code}\n' https://api.riotpiao.com/v1/rerank \
-H 'content-type: application/json' -d '{"model":"BAAI/bge-reranker-base","query":"q","texts":["a"]}'
# expected: identical status from both endpoints
```
-48
View File
@@ -1,48 +0,0 @@
# 2.9 — Dialect-neutral canonical request (GREEN)
Phase: 2 — LLM surfaces
Stage: GREEN
Depends on: [2.1](2.1-model-registry.md), [2.2](2.2-body-based-dispatch.md)
The gateway serves two permanent protocol dialects: `/v1/*` is OpenAI-compatible and
`/llm/*` is Anthropic Messages. Both translate into one internal shape before anything
else happens to them, and both translate back out of it on the way to the client.
- [ ] A single internal request representation exists that carries at minimum: the
resolved upstream, the model name as the client sent it, the ordered turns, an
optional system instruction, a maximum output token count, stop sequences, and a
streaming flag
- [ ] The representation names no dialect: nothing in it is called openai or anthropic,
and no field exists solely because one dialect happens to spell it that way
- [ ] Model dispatch, the shared slot controller, per-caller budgets, logging and
metrics all read the canonical request and never the raw dialect body
- [ ] The surface that received a request is recorded as one field on the canonical
request, used only to choose the response and error rendering, never to choose an
upstream or a slot
- [ ] An identical prompt sent to `/v1/chat/completions` and to `/llm/v1/messages`
produces the same resolved upstream, the same slot accounting and the same log
fields apart from that one surface label
- [ ] Adding a third dialect requires a new translator only; the slot controller,
dispatch and budget code are untouched
- [ ] Translation failure is a client error at the surface boundary, and no partially
populated canonical request ever reaches an upstream
`reasoning` maps to `reasoning-predictor.llm-serving:80`; `ornith:35b` and
`qwen2.5:3b-instruct` both map to `ornith-predictor.llm-serving:80`. Ports are 80.
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
# expected: 200, reasoning stub hit once
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
-H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
# expected: 200, reasoning stub hit once, same upstream and same slot counter as the /v1 call
grep -h 'upstream=' /tmp/gateway.log | tail -2
# expected: both lines show upstream=reasoning-predictor and model=reasoning, differing only in the surface label
```
-34
View File
@@ -1,34 +0,0 @@
# 3.1 — JWKS fetch and rotation (GREEN)
Phase: 3 — Authentication
Stage: GREEN
Depends on: [0.2](0.2-route-configuration.md), [0.3](0.3-health-endpoints.md)
- [ ] The signing key set is fetched from Authentik at `https://authentik.riotpiao.com` at runtime; no public key is pinned in configuration, in an image, or in a manifest
- [ ] The JWKS URL is configuration, so a local stub issuer can be pointed at instead — no cluster and no credentials needed to verify this task
- [ ] Fetched keys are cached and reused; a token verification does not trigger a network call per request
- [ ] A token whose key id is not in the cache triggers a refetch, and the refetch is rate-limited so an unknown-key flood cannot hammer Authentik
- [ ] After a key rotates at the issuer, tokens signed by the new key verify without restarting, redeploying, or editing configuration
- [ ] Tokens signed by a key that is no longer published stop verifying once the cache reflects that
- [ ] `GET /readyz` fails while JWKS has never been fetched successfully, and `GET /healthz` is unaffected
- [ ] A JWKS fetch failure while a valid cache exists does not take the gateway down; it is logged and retried
- [ ] No key material, token, or fetched secret appears in logs
This deletes the rotation runbook `AUTH-PLAN.md` was forced to propose. That runbook
existed only because Kong OSS needed a pinned `rsa_public_key`; it must not be
carried forward.
## Verify
```bash
# stub issuer serving a JWKS, gateway pointed at it
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 200 once JWKS has been fetched
# stub returns 500 for JWKS from a cold start
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz # expected: 503
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz # expected: 200
# rotate the stub's key, then present a token signed by the new key, no restart
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models -H "authorization: Bearer $NEW_TOKEN"
# expected: 200 — refetch happened on the unknown key id
```
-37
View File
@@ -1,37 +0,0 @@
# 3.2 — Bearer token validation (GREEN)
Phase: 3 — Authentication
Stage: GREEN
Depends on: [3.1](3.1-jwks-fetch-and-rotation.md)
- [ ] `Authorization: Bearer <jwt>` is the credential the gateway accepts on protected routes
- [ ] The scheme match is case-insensitive, as the HTTP spec requires — `bearer`, `Bearer` and `BEARER` all work
- [ ] A valid, unexpired token signed by a currently published Authentik key returns the upstream response
- [ ] Missing header, wrong scheme, malformed token, bad signature, expired token, and wrong issuer or audience each return 401 with `WWW-Authenticate` set
- [ ] Rejections are RFC 9457 `application/problem+json`, and the reason is logged without logging the token
- [ ] The `Authorization` header is not forwarded to upstreams
- [ ] The validated caller identity is available to later stages, since [3.5](3.5-capability-authorization.md) authorizes on it
- [ ] Signature verification is real: a token with a valid-looking payload and a forged signature is rejected
This is precisely what Kong OSS `key-auth` could not do. Verified live: a raw
`apikey:` header succeeded with 200 while `Authorization: Bearer` failed with 401,
which hard-blocked every OpenAI-compatible client and is why authentication is OFF
on the model API today.
Authentik is at `https://authentik.riotpiao.com`. A local stub issuer must be enough
to work this task — no cluster, no credentials.
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H "authorization: Bearer $VALID_TOKEN" -H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[]}'
# expected: 200
curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{"model":"reasoning"}'
# expected: 401, WWW-Authenticate present, content-type application/problem+json
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models -H "apikey: $OLD_KONG_KEY"
# expected: 401 — the retired Kong credential form is not accepted
```
-30
View File
@@ -1,30 +0,0 @@
# 3.3 — Authentik service account and token provider (GREEN)
Phase: 3 — Authentication
Stage: GREEN
Depends on: [3.2](3.2-bearer-validation.md)
- [ ] A service account exists in Authentik at `https://authentik.riotpiao.com` for machine callers of the model API
- [ ] An OAuth2 provider is configured so that account can obtain a token non-interactively, with no browser step
- [ ] The token endpoint returns a signed JWT whose issuer and audience match what the gateway validates, and which the gateway accepts on a protected route
- [ ] Token lifetime is set deliberately and recorded, not left at whatever the default is
- [ ] The client secret is stored as a Kubernetes Secret referenced from git, never committed in plaintext
- [ ] The Authentik configuration is captured in the repo as reproducible steps or declarative config, so it can be rebuilt rather than clicked together again
- [ ] The token carries whatever claim [3.5](3.5-capability-authorization.md) authorizes on
Open risk recorded in `AUTH-PLAN.md` and unresolved: Authentik 2026.x may require an
app-password or JWT-assertion flow rather than a plain `client_secret` POST to the
token endpoint. Verify which flow the running version accepts before wiring anything
that depends on it, and record the answer here.
## Verify
```bash
curl -s -i -X POST https://authentik.riotpiao.com/application/o/token/ \
-d grant_type=client_credentials -d "client_id=$CID" -d "client_secret=$CSEC"
# expected: 200, JSON with access_token; if 400/401, the app-password / JWT-assertion risk above is real — record which
curl -s -o /dev/null -w '%{http_code}\n' http://homelab-frontend.api.svc.cluster.local/v1/models \
-H "authorization: Bearer $ACCESS_TOKEN"
# expected: 200 — the gateway accepts a real Authentik-issued token, not only a stub one
```
-33
View File
@@ -1,33 +0,0 @@
# 3.4 — Flag-gated auth rollout (GREEN)
Phase: 3 — Authentication
Stage: GREEN
Depends on: [3.2](3.2-bearer-validation.md)
- [ ] Authentication is controlled by an explicit flag in configuration, and its default is OFF
- [ ] With the flag off, every route answers exactly as it did before Phase 3 existed — no 401s, no `WWW-Authenticate`, no behaviour change
- [ ] With the flag on, protected routes require `Authorization: Bearer` and reject anything else with 401
- [ ] `GET /healthz` and `GET /readyz` never require authentication in either state
- [ ] Which routes are protected is per-route configuration, so auth can be turned on for one surface at a time
- [ ] Turning the flag on is a git change synced by Argo; it is never toggled by hand against the cluster
- [ ] The flag's current state is visible in logs at startup and in metrics, so nobody has to guess whether auth is on
- [ ] Turning the flag off again fully restores unauthenticated access, making the rollout reversible
The model API is unauthenticated today — verified live, a request with no credentials
returns 200. Enabling this flag breaks every current caller until they hold a token,
pi included. That is why it defaults off and is enabled deliberately, after
[3.6](3.6-pi-client-migration.md).
## Verify
```bash
# flag off
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models # expected: 200
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' # expected: 200
# flag on, no credentials
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' # expected: 401
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz # expected: 200
```
-33
View File
@@ -1,33 +0,0 @@
# 3.5 — Capability authorization (RED)
Phase: 3 — Authentication
Stage: RED
Depends on: [3.2](3.2-bearer-validation.md), [3.4](3.4-flag-gated-rollout.md)
- [ ] Beyond proving who the caller is, the gateway checks the token is permitted to invoke the capability being called
- [ ] Each capability prefix — `/v1/*` for the model surface, and the future `/sqs/*`, `/workflow/*`, `/cluster/*`, `/db/*` — maps to a required grant, and the mapping is configuration
- [ ] A token carrying a queue grant but no model grant is rejected on `POST /v1/chat/completions` with 403, not 401 — it authenticated fine, it is not permitted
- [ ] A token with no recognised grant at all is rejected on every protected route
- [ ] Rejections are RFC 9457 `application/problem+json` and state which capability was denied, without echoing the token or its claims verbatim
- [ ] Denials are logged with the caller identity and the capability, and counted as a distinct rejection reason
- [ ] A route with no required grant configured while auth is on fails startup rather than defaulting to allow-all
Write the failing tests first: a queue token must not invoke a GPU. The default on an
unconfigured route is deny, because a fail-open authorization layer is worse than none
— it reads as protection while granting everything.
## Verify
```bash
curl -s -i localhost:8080/v1/chat/completions -H "authorization: Bearer $QUEUE_ONLY_TOKEN" \
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}'
# expected: 403, application/problem+json naming the denied capability, no upstream stub hit
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H "authorization: Bearer $MODEL_TOKEN" -H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[]}'
# expected: 200
./gateway --config testdata/auth-on-route-without-grant.yaml; echo "exit=$?"
# expected: non-zero exit, stderr names the route missing a required grant
```
-37
View File
@@ -1,37 +0,0 @@
# 3.6 — Migrate pi to Bearer and one provider (REFACTOR)
Phase: 3 — Authentication
Stage: REFACTOR
Depends on: [2.2](2.2-body-based-dispatch.md), [3.3](3.3-authentik-service-account.md), [3.4](3.4-flag-gated-rollout.md)
- [ ] `~/.pi/agent/models.json` no longer contains a `customHeaders: {apikey: ...}` block anywhere
- [ ] pi authenticates with `Authorization: Bearer` carrying an Authentik-issued token
- [ ] The three provider entries `homelab-reasoning`, `homelab-ornith` and `homelab-qwen` collapse into ONE provider entry
- [ ] That single provider's base URL is the canonical OpenAI base, and the three models are listed under it — body-based dispatch makes per-model base URLs unnecessary
- [ ] pi reaches all three models through `POST /v1/chat/completions` with `model` set to `reasoning`, `ornith:35b`, and `qwen2.5:3b-instruct`
- [ ] Streaming still works from pi, and interrupting a generation cancels it upstream rather than leaving it running
- [ ] The old config is captured before editing, so a revert is one file restore
- [ ] Verified with auth ON, because that is the state the migration exists to survive
The `apikey` header exists only because Kong OSS `key-auth` rejected
`Authorization: Bearer`. Once this lands, nothing depends on the legacy aliases in
[2.4](2.4-legacy-path-aliases.md) and they can be removed.
## Verify
```bash
grep -c apikey ~/.pi/agent/models.json # expected: 0
grep -c '"homelab' ~/.pi/agent/models.json # expected: 1 provider entry, not 3
for m in reasoning ornith:35b qwen2.5:3b-instruct; do
curl -s -o /dev/null -w "$m %{http_code}\n" https://api.riotpiao.com/v1/chat/completions \
-H "authorization: Bearer $ACCESS_TOKEN" -H 'content-type: application/json' \
-d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}"
done
# expected: 200 for all three, with auth enabled
curl -N -s https://api.riotpiao.com/v1/chat/completions -H "authorization: Bearer $ACCESS_TOKEN" \
-H 'content-type: application/json' \
-d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
# expected: text/event-stream chunks arriving incrementally; reasoning_content and content both present
```
+6 -21
View File
@@ -2,7 +2,7 @@
Phase: 4 — Limits and budgets
Stage: GREEN
Depends on: [4.3](4.3-problem-json-errors.md), [2.9](2.9-canonical-request-model.md)
Depends on: [4.3](4.3-problem-json-errors.md)
The `reasoning` upstream is 2 replicas x `--max-num-seqs=4` = 8 concurrent sequence
slots cluster-wide. That 8 is a vLLM concurrency setting, not a GPU count — worker-1
@@ -10,9 +10,9 @@ has 4 physical GPUs and the two numbers are unrelated. Exceeding 8 does not fail
fast; it silently queues inside vLLM where the gateway has no visibility and no
ability to cancel.
The gateway serves two permanent dialects over the same predictors: `/v1/*` is
OpenAI-compatible and `/llm/*` is Anthropic Messages. There is ONE controller, keyed by
upstream, sitting below both.
The gateway serves `/v1/*` (OpenAI-compatible) over the same predictors. The
Anthropic `/llm/*` dialect was dropped (see `tasks/INDEX.md` Phase 2) — this
controller only ever needs to know about `/v1/*`.
- [ ] Concurrent in-flight requests to `reasoning` are capped at a configured limit strictly below 8, leaving operator headroom
- [ ] The cap, the queue depth, and the queue wait timeout are all explicit in configuration with no silent defaults
@@ -21,13 +21,8 @@ upstream, sitting below both.
- [ ] A slot is released when the response completes, when the upstream errors, and when the client disconnects mid-stream — no path leaks a slot
- [ ] A client that disconnects while still queued never reaches the upstream and never consumes a slot
- [ ] The cap applies only to `reasoning`; other upstreams are unaffected and are not blocked behind its queue
- [ ] The controller is keyed by UPSTREAM, never by route, path prefix or dialect `reasoning-predictor.llm-serving:80` has exactly one cap and one queue
- [ ] A request arriving on `/v1/chat/completions` and one arriving on `/llm/v1/messages` contend for the same slots and the same queue, admitted in arrival order
- [ ] Total in-flight requests against `reasoning` never exceed the configured cap regardless of which surface they arrived through, including when both surfaces are saturated at once
- [ ] Per-dialect semaphores are explicitly wrong and must not exist: the 8 sequence slots are physical, so two independent gates would each believe they were within budget while together exceeding it
- [ ] The controller reads the dialect-neutral canonical request and has no knowledge of which surface produced it; adding a third dialect requires no change here
- [ ] Slot occupancy and queue depth are reported per upstream, and requests are attributable to a surface for logging only, never for admission
- [ ] Rejection at a full queue renders per surface — `application/problem+json` on `/v1`, the Anthropic error shape on `/llm` — from one shared decision
- [ ] The controller is keyed by UPSTREAM, never by route or path prefix — `reasoning-predictor.llm-serving:80` has exactly one cap and one queue
- [ ] Slot occupancy and queue depth are reported per upstream
## Verify
@@ -46,14 +41,4 @@ grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
curl -s -D - -o /dev/null -X POST localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' | grep -i retry-after
# expected: Retry-After present on the rejected request
# Same cap=6 stub, but split the load across both dialects: 20 on /v1 and 20 on /llm.
seq 20 | xargs -P20 -I{} curl -s -o /dev/null -X POST localhost:8080/v1/chat/completions \
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' &
seq 20 | xargs -P20 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
-H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' &
wait
grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
# expected: 0 — the two surfaces share one cap, they do not get 6 each
```
+1 -1
View File
@@ -1,4 +1,4 @@
# 4.3 — RFC 9457 problem+json rejections (RED)
# 4.3 — RFC 9457 problem+json rejections (GREEN)
Phase: 4 — Limits and budgets
Stage: RED
+1 -1
View File
@@ -1,4 +1,4 @@
# 5.1 — Prometheus parity with the retiring Kong plugin (RED)
# 5.1 — Prometheus parity with the retiring Kong plugin (GREEN)
Phase: 5 — Observability
Stage: RED
-33
View File
@@ -1,33 +0,0 @@
# 6.1 — Hardened container image (GREEN)
Phase: 6 — Deploy and cutover
Stage: GREEN
The gateway is the public edge process. It gets no shell, no package manager, no
writable filesystem and no privileges it does not need.
- [ ] The runtime image is distroless or scratch — no shell, no package manager, no busybox
- [ ] The container runs as a non-root user, enforced by `runAsNonRoot` and an explicit non-zero UID
- [ ] The root filesystem is read-only; any writable path the process genuinely needs is an explicitly mounted volume, not a relaxed rootfs
- [ ] All Linux capabilities are dropped, and none are added back
- [ ] `seccompProfile` is `RuntimeDefault`
- [ ] Privilege escalation is disabled
- [ ] Image tags are the commit SHA of the source that built them. Never `:latest`, never a moving tag — Argo `selfHeal` cannot reconcile a mutable tag reliably, and a rollback needs a tag that still means what it meant yesterday
- [ ] The image is reproducible from a committed build definition; nothing is built by hand
- [ ] The image contains no credentials, kubeconfig or service-account token baked in (G2)
## Verify
```bash
docker run --rm --entrypoint sh <image>:<sha>
# expected: fails — no shell in the image
docker run --rm --read-only --user 65532 <image>:<sha> --version
# expected: starts and exits cleanly under a read-only rootfs as a non-root user
docker inspect <image>:<sha> --format '{{.Config.User}}'
# expected: a non-zero numeric UID, not empty and not "root"
grep -rn 'image:' k8s/ | grep -v '@sha256\|:[0-9a-f]\{7,\}'
# expected: no matches — every image reference is pinned to a SHA
```
-35
View File
@@ -1,35 +0,0 @@
# 6.2 — Deployment, Service, NetworkPolicy (GREEN)
Phase: 6 — Deploy and cutover
Stage: GREEN
Depends on: [6.1](6.1-hardened-image.md)
Manifests only. Nothing here exposes the gateway publicly — that is the cutover, and
it is a separate step.
- [ ] A Deployment runs the gateway with at least 2 replicas, matching Kong's current replica count
- [ ] The pod spec carries the hardening from 6.1: non-root, read-only rootfs, all capabilities dropped, `seccompProfile: RuntimeDefault`
- [ ] Liveness probes hit `/healthz` and readiness probes hit `/readyz`; readiness fails while config is invalid or JWKS has never been fetched
- [ ] Route configuration is mounted from a ConfigMap sourced from git — not a CRD, and not baked into the image
- [ ] A config change rolls the pods; a stale ConfigMap cannot be silently served by a long-lived pod
- [ ] Termination allows in-flight requests to drain, with a grace period long enough for the streaming timeouts in use
- [ ] A Service exposes the gateway in-cluster with a named port, resolvable at `http://<svc>.api.svc.cluster.local`
- [ ] No ServiceAccount with any RBAC is bound; the pod does not need or receive an API-server token (G2)
- [ ] A NetworkPolicy allows ingress only from `ingress-nginx`
- [ ] The same NetworkPolicy allows egress only to the proxied upstreams plus Authentik, plus DNS. Everything else is denied
- [ ] Resource requests and limits are set explicitly
- [ ] Every manifest is committed to git and applied by Argo. No `kubectl apply`, no `helm upgrade` (G7)
## Verify
```bash
kubectl -n api get deploy homelab-frontend -o jsonpath='{.spec.replicas} {.spec.template.spec.securityContext}{"\n"}'
# expected: 2, with runAsNonRoot true and seccompProfile RuntimeDefault
kubectl -n api exec deploy/homelab-frontend -- true 2>&1
# expected: fails — distroless image has no shell to exec into
kubectl -n llm-serving run np-probe --rm -it --image=curlimages/curl --restart=Never -- \
curl -s -m 5 http://homelab-frontend.api.svc.cluster.local/healthz
# expected: times out or is refused — ingress is restricted to ingress-nginx only
```
-32
View File
@@ -1,32 +0,0 @@
# 6.3 — Argo Application in the homelab-root GitOps repo (GREEN)
Phase: 6 — Deploy and cutover
Stage: GREEN
Depends on: [6.2](6.2-kubernetes-manifests.md)
Verified live 2026-08-19: zero Argo Applications anywhere in the cluster source from
any Forgejo URL. `github.com/Riotpiaole/riotpiao.homelab.com` is authoritative — do
not introduce a second source of truth.
- [ ] An Argo Application for the gateway is committed under `k8s/argocd/apps/` in `github.com/Riotpiaole/riotpiao.homelab.com`
- [ ] `repoURL` is that GitHub repo. No Forgejo URL, no local path, no second remote
- [ ] It targets the `api` namespace, alongside the existing Kong deployment
- [ ] Its sync wave orders it so the gateway is healthy before anything that depends on it, and does not disturb Kong's existing wave-7 Application
- [ ] Automated sync with prune and selfHeal is enabled, so drift is corrected without a human
- [ ] The Application reaches `Synced` and `Healthy` and stays there across a resync
- [ ] Adding it changes nothing about live traffic — Kong still serves `api.riotpiao.com` after this lands
- [ ] The commit is pushed and Argo picks it up on its own. No `kubectl apply` of the Application itself (G7)
## Verify
```bash
kubectl -n argocd get app homelab-frontend \
-o jsonpath='{.spec.source.repoURL}{" "}{.status.sync.status}{" "}{.status.health.status}{"\n"}'
# expected: the GitHub repo URL, Synced, Healthy
kubectl -n argocd get app -o json | grep -ci forgejo
# expected: 0 — GitHub remains the only Application source
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
# expected: 200, still served by Kong — this task changed no live traffic
```
-36
View File
@@ -1,36 +0,0 @@
# 6.4 — Deploy unexposed alongside Kong (GREEN)
Phase: 6 — Deploy and cutover
Stage: GREEN
Depends on: [6.3](6.3-argocd-application.md)
The gateway runs in the cluster against the real predictors while Kong continues to
serve every byte of live traffic. This is the last step before anything user-visible
changes, and it is fully reversible — deleting the Application removes it.
- [ ] The gateway is running in namespace `api` and is reachable only in-cluster at `http://<svc>.api.svc.cluster.local`
- [ ] No Ingress points at the gateway. Ingress `api/api` still sends `/` on `api.riotpiao.com` to `kong-proxy:80`
- [ ] Its configured upstreams are the real Services: `reasoning-predictor.llm-serving:80`, `ornith-predictor.llm-serving:80`, `embeddings-predictor.llm-serving:80`, `reranker-predictor.llm-serving:80`, and `agent-hub.agent-pod:9090`
- [ ] Note `reasoning-predictor` listens on port 80, not 8080
- [ ] A real chat completion succeeds in-cluster against `reasoning`, returning `reasoning_content` and `content` as separate fields
- [ ] A streaming chat completion delivers tokens incrementally in-cluster, not as one buffered blob at completion
- [ ] Embeddings and rerank both answer correctly, with rerank reaching the upstream's `/rerank` path
- [ ] A client disconnect mid-stream is observed to cancel the upstream generation and release its sequence slot
- [ ] Prometheus is scraping the gateway and the metrics reflect this in-cluster traffic
- [ ] `api.riotpiao.com` is unaffected throughout — verified before and after
## Verify
```bash
kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
# expected: kong-proxy — the gateway is still unexposed
kubectl -n api run probe --rm -it --image=curlimages/curl --restart=Never -- sh -c '
curl -sN -X POST http://homelab-frontend.api.svc.cluster.local/v1/chat/completions \
-H "content-type: application/json" \
-d "{\"model\":\"reasoning\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"count to 5\"}]}"'
# expected: multiple data: chunks arriving over time, terminated by data: [DONE]
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
# expected: 200 — public traffic still served by Kong, untouched
```
-45
View File
@@ -1,45 +0,0 @@
# 6.5 — Cutover: repoint Ingress `api/api` (GREEN)
Phase: 6 — Deploy and cutover
Stage: GREEN
Depends on: [2.8](2.8-kong-parity-test.md), [6.4](6.4-deploy-alongside-kong.md)
THIS is the cutover. Ingress `api/api` catch-alls `/` on `api.riotpiao.com`; today its
backend is `kong-proxy:80`. Changing that one backend moves all live traffic to the
gateway. Reverting is the same one-line change back to `kong-proxy:80`, committed and
synced — Kong stays running and untouched, so the revert takes effect in seconds.
Preconditions, all required before starting:
- [ ] 2.8 Kong parity is passing — gateway and Kong return equivalent responses for every route in the migration inventory, including a streaming request and a mid-stream disconnect
- [ ] 6.4 is complete: the gateway is healthy in-cluster against the real predictors, and Prometheus is scraping it
- [ ] The inventory has been re-verified immediately beforehand. Kong's Deployment has eleven ReplicaSets and is being actively iterated; a stale inventory is a stale plan
- [ ] Kong remains deployed and serving-capable throughout. Nothing about Kong is deleted in this task
The change itself:
- [ ] The `api/api` Ingress backend becomes the gateway Service, changed in git and synced by Argo. No `kubectl apply`, no `kubectl edit` (G7)
- [ ] The nginx annotations stay exactly as they are: `proxy-read-timeout: 3600`, `proxy-send-timeout: 3600`, `proxy-buffering: off`, `proxy-body-size: 0`. These are what make token streaming work and the gateway needs the same treatment
- [ ] The revert is a single-line commit reverting the backend to `kong-proxy:80`, and it has been rehearsed at least once
- [ ] After the change, public streaming works end to end through nginx, unbuffered
- [ ] pi keeps working on the legacy `/v1/{reasoning,ornith,qwen}/chat/completions` aliases
- [ ] A soak period follows, watching gateway metrics, rejection counters and pi traffic. Do not proceed to 6.6 until the soak is clean
## Verify
```bash
kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
# expected: the gateway Service, no longer kong-proxy
curl -sN -X POST https://api.riotpiao.com/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"count to 5"}]}'
# expected: incremental data: chunks over time through nginx, ending in data: [DONE]
curl -s -o /dev/null -w '%{http_code}\n' -X POST https://api.riotpiao.com/v1/reasoning/chat/completions \
-H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}'
# expected: 200 — pi's legacy alias still works
kubectl -n api get deploy kong-kong -o jsonpath='{.status.readyReplicas}{"\n"}'
# expected: 2 — Kong still running and ready, so the revert is one commit away
```
-45
View File
@@ -1,45 +0,0 @@
# 6.6 — Kong teardown (REFACTOR)
Phase: 6 — Deploy and cutover
Stage: REFACTOR
Depends on: [6.5](6.5-cutover.md)
IRREVERSIBLE. Every step before this one could be undone in seconds by repointing
Ingress `api/api` back to `kong-proxy:80`. Once the Kong Application is removed and
Argo prunes the Helm release and its CRDs, that escape hatch is gone — recovery means
reinstalling Kong from scratch and rebuilding six plugin CRs.
Do not start until the soak after 6.5 has been clean for a deliberate, agreed period.
Preconditions:
- [ ] 6.5 is complete: `api.riotpiao.com` has been served entirely by the gateway through the soak, with no reverts
- [ ] Gateway metrics over the soak show no elevated 5xx rate and no unexplained rejections
- [ ] pi and every other known caller have been confirmed working against the gateway
- [ ] The inventory has been re-verified immediately beforehand — Kong's config has been actively iterated
Order matters. Delete the routing objects first, the Application last:
- [ ] The 7 `ingressClassName: kong` Ingresses are deleted from git — 6 in `llm-serving`, 1 in `agent-pod` for `/console`, `/run`, `/sessions`
- [ ] The 6 `KongPlugin` CRs are deleted from git: the three `llm-rewrite-*` chat rewrites, `llm-rewrite-rerank`, `llm-models-list`, and the cluster-wide `prometheus` plugin
- [ ] Removing the cluster-wide `prometheus` plugin does not blind any dashboard, because 5.1 parity metrics are already being scraped from the gateway
- [ ] The `kong` Application is then removed from `k8s/argocd/apps/55-api-gateway.yaml`
- [ ] Argo prunes the Helm release, the Kong CRDs and the namespace leftovers on its own. No `helm uninstall`, no `kubectl delete` (G7)
- [ ] Public traffic is verified unaffected after each deletion, not only at the end
- [ ] No orphaned Kong CRDs, ReplicaSets or Services remain in namespace `api`
## Verify
```bash
kubectl get ingress -A -o jsonpath='{range .items[*]}{.spec.ingressClassName}{"\n"}{end}' | sort | uniq -c
# expected: no kong entries remain; nginx and istio counts unchanged
kubectl get kongplugins -A 2>&1; kubectl -n argocd get app kong 2>&1
# expected: CRD not found, and Application "kong" not found
kubectl -n api get all | grep -i kong
# expected: no output — nothing Kong-related left
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
# expected: 200, served by the gateway
```
+1 -1
View File
@@ -2,7 +2,7 @@
Phase: 7 — Additional capability prefixes
Stage: GREEN
Depends on: [6.5](6.5-cutover.md)
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
atlas lives in a separate repo, `riotpiao-backend`. It keeps its own informers and its
own RBAC. The gateway proxies to it and holds no Kubernetes credentials of its own —
+1 -1
View File
@@ -2,7 +2,7 @@
Phase: 7 — Additional capability prefixes
Stage: GREEN
Depends on: [6.5](6.5-cutover.md)
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
`management-service` already exposes gRPC at `kmsvc.riotpiao.com`. This prefix is a
NEW surface, not a replacement — that gRPC endpoint stays exactly as it is and nothing
+1 -1
View File
@@ -2,7 +2,7 @@
Phase: 7 — Additional capability prefixes
Stage: GREEN
Depends on: [6.5](6.5-cutover.md)
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
Temporal runs in the `temporal` namespace. Temporal namespace registration is
automatic via queue-operator and is NEVER done manually — this route must not create,
+1 -1
View File
@@ -2,7 +2,7 @@
Phase: 7 — Additional capability prefixes
Stage: GREEN
Depends on: [6.5](6.5-cutover.md)
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
The most dangerous prefix in this phase. G2 says the gateway holds no cluster
credentials — so it cannot be the thing that authenticates to CloudNativePG, MinIO or
@@ -0,0 +1,50 @@
# 8.1 — `ServiceAdapter` CRD, informer, RBAC (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §1.
**G2 note — read before objecting.** G2 says "the gateway holds no Kubernetes
credentials, config comes from git not a CRD." This task deliberately supersedes G2
for this one narrow purpose; the acknowledged-supersession rationale is written out
in API_ROUTING_HYBRID_DESIGN.md's Context section (top of that doc) and in
`INDEX.md`'s G2 line. Do not treat this as a task-file error — the supersession is
intentional and pre-approved, scoped to exactly the read-only Role this task adds.
- [ ] `apis/gateway/v1/serviceadapter_types.go` defines `ServiceAdapter` matching
the design doc's example CRs (§1, §6): `spec.serviceName`, `spec.upstream.{url,timeoutSeconds}`,
`spec.auth.{required,capability}`, `spec.retryable`, `spec.resources[].name`,
`spec.resources[].methods[].{verb,upstreamPath,requestSchema,responseSchema,auth}`
- [ ] `controller-gen` generates the CRD YAML from those types into `k8s/crd-serviceadapter.yaml`, added to `k8s/kustomization.yaml`'s `resources:`
- [ ] CRD is namespace-scoped, group `gateway.riotpiao.com/v1`, kind `ServiceAdapter` — not cluster-scoped
- [ ] `k8s/rbac.yaml` gains a namespace-scoped `Role`/`RoleBinding` (`get`, `list`, `watch` only, no `status`/`finalizers` verbs) for the existing `api-gateway` ServiceAccount, exactly as specced in §1
- [ ] `internal/serviceadapter/registry.go`: `client-go` `SharedInformer` on `ServiceAdapter` in namespace `api`, feeding an in-memory map keyed by `spec.serviceName`
- [ ] Add/Update/Delete informer callbacks mutate the map directly; no gateway restart required to pick up a CR change
- [ ] A `ServiceAdapter` CR with a malformed `requestSchema`/`responseSchema` (per 8.3's DSL) logs an error and is skipped — it does not crash the informer or block other adapters from loading
- [ ] `go build ./...` succeeds with the new `apis/` package and `internal/serviceadapter/registry.go` in the tree
## Verify
```bash
kubectl -n api get role api-gateway-serviceadapter-reader -o yaml
# expected: rules limited to gateway.riotpiao.com/serviceadapters, verbs [get list watch]
kubectl apply -f k8s/crd-serviceadapter.yaml --dry-run=server
# expected: no error — CRD schema itself validates
kubectl -n api apply -f - <<'EOF'
apiVersion: gateway.riotpiao.com/v1
kind: ServiceAdapter
metadata: { name: smoke-test }
spec:
serviceName: smoke-test
upstream: { url: http://example.invalid, timeoutSeconds: 5 }
auth: { required: false }
resources: []
EOF
kubectl -n api logs deploy/api-gateway --since=10s | grep -i "smoke-test"
# expected: informer logs an Add event for smoke-test within resync/watch latency, no restart
kubectl -n api delete serviceadapter smoke-test
```
+44
View File
@@ -0,0 +1,44 @@
# 8.10 — Phase 8 gate: every service on `ServiceAdapter` + KV-schema (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: gate
Depends on: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8 (8.9 optional — see below)
Purpose: confirm all real backend services — `workflow`, `s3`, `sqs`, `iam`,
`memory` — are onboarded through the `ServiceAdapter` CRD with `requestSchema`/
`responseSchema` validation (8.3's DSL), none of them left on hand-written
`switch`-case Go routes or the old path-prefix scheme (`/workflow/*`, `/sqs/*`,
`/db/*`). This is the "make sure every service adapts to this format" checkpoint —
it does not add new capability, it verifies consistency across what 8.48.8 built.
- [ ] `kubectl -n api get serviceadapters` lists exactly `workflow`, `s3`, `sqs`,
`iam`, `memory` (plus `postgres` only if that example CR was actually applied
as a real onboarding, not just kept as doc illustration)
- [ ] No adapter's CR has an empty `requestSchema` on a method that accepts a body
— every write path validates input
- [ ] `internal/server/router.go` has no remaining path-based `switch` case for
`/workflow`, `/sqs`, or `/db` — those prefixes 404 or are fully removed from
the router, superseded by `X-Service` dispatch
- [ ] One curl per adapter succeeds end-to-end through the header-based path (below)
- [ ] `go test ./... -race`, `CGO_ENABLED=0 go build ./...`, `go vet ./...` all pass
## Verify
```bash
for svc_resource in "workflow:workflow" "sqs:message" "iam:user" "memory:project"; do
svc="${svc_resource%%:*}"; res="${svc_resource##*:}"
code=$(curl -s -o /dev/null -w '%{http_code}' https://api.riotpiao.com/ \
-H 'Authorization: Bearer <jwt-with-all-capabilities>' \
-H "X-Service: $svc" -H "X-Resource: $res")
echo "$svc/$res -> $code"
done
# expected: none of the four returns 404 for "unknown X-Service" — each is a live adapter
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/workflow/health
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/sqs/healthz
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/db/healthz
# expected: 404 on all three — old prefix routes are gone, not just unused
kubectl -n api get serviceadapters -o jsonpath='{range .items[*]}{.spec.serviceName}{"\n"}{end}' | sort
# expected: iam, memory, s3, sqs, workflow (plus postgres iff real)
```
+55
View File
@@ -0,0 +1,55 @@
# 8.2 — `X-Service`/`X-Resource` dispatcher, capability auth, blind 5xx retry (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1 (CRD, informer, in-memory registry)
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2, §4.
- [ ] `internal/server/router.go`'s `ServeHTTP` gets a new branch, checked **before**
the existing path switch: if the request carries an `X-Service` header,
dispatch to `internal/serviceadapter/router.go`, regardless of `r.URL.Path`
- [ ] `/v1/chat/completions`, `/v1/embeddings`, `/v1/rerank`, `/healthz`, `/readyz`
keep their existing path-based routing unchanged — matched first, never see `X-Service`
- [ ] Dispatch key: `X-Service` header → adapter from 8.1's registry, then HTTP method
+ `X-Resource` header → `{verb, upstreamPath}` on that adapter
- [ ] Unknown `X-Service` → 404 problem+json. Known service, unknown `X-Resource`/verb
combination → 404 problem+json, not a silent proxy-through
- [ ] Auth: `spec.auth.capability` is the default per adapter; a method's own
`auth.capability` overrides it; `auth.required: false` at either level skips
the capability check entirely. Depends on `internal/auth` (validated JWT
middleware) existing — if it does not yet exist in this repo, stop and report
instead of stubbing it
- [ ] `internal/resilience/retry.go` (new): blind retry on any 5xx from the upstream,
bounded attempts with backoff, gated by `spec.retryable` (default `true`) —
wraps every outbound call this dispatcher makes
- [ ] `{id}`-style path segments in `upstreamPath` (e.g. `/v1/tables/{id}`) are
resolved from an explicit source — since routing is header-only at the gateway
root, there is no URL path segment to take it from. Decide and document the
actual source (query param, extra header, or body field) in this task's own
notes before implementing; do not guess silently
- [ ] Legacy `/workflow*` path stays mounted, delegating internally to this same
dispatcher, per §2
## Verify
```bash
# unknown service
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
-H 'X-Service: does-not-exist' -H 'X-Resource: whatever'
# expected: 404
# known service, wrong verb+resource combination
curl -s -o /dev/null -w '%{http_code}\n' -X PATCH https://api.riotpiao.com/ \
-H 'Authorization: Bearer <jwt>' -H 'X-Service: iam' -H 'X-Resource: role'
# expected: 404 (role only defines GET/POST in §3)
# capability override enforced
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
-H 'Authorization: Bearer <jwt-with-memory:read-only>' \
-H 'X-Service: memory' -H 'X-Resource: ingest' -X POST -d '{}'
# expected: 403 — ingest requires memory:write, token only has memory:read
# blind retry: point smoke-test adapter (from 8.1) at an upstream returning 503 twice then 200,
# confirm the caller sees 200 and the upstream log shows 3 attempts
```
+60
View File
@@ -0,0 +1,60 @@
# 8.3 — Request/response schema validation, KV+type DSL (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1 (CRD types carry `requestSchema`/`responseSchema`), 8.2 (dispatcher calls this per request)
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §1
"Request/response validation" subsection — the full DSL including the array/nullable
extension. Read that subsection in full before implementing; do not reinvent the
shape from the checklist below alone.
- [ ] `internal/serviceadapter/validate.go` (new, no external dependency —
not a JSON Schema library): validates a parsed body (`map[string]interface{}`
or `[]interface{}` for array-typed schemas) against a `FieldSchema`
- [ ] Object schema: `required` fields present, each present field in `fields`
matches its declared Go runtime type (`string`→string, `number`→float64,
`boolean`→bool, `array``[]interface{}`, `object``map[string]interface{}`)
- [ ] `nullable: true` on a field accepts JSON `null` regardless of declared type;
a `null` on a non-nullable field is a `type_mismatch`; bare `field: string`
shorthand means `{type: string, nullable: false}`
- [ ] `strict: true` rejects body keys not listed in `fields`; default `false` is permissive
- [ ] Array schema (`type: array`): `items: string` validates every element is that
scalar type; `items: { fields: {...} }` validates every element as an object
schema, one level deep inside each item — no further nesting
- [ ] Compiled once per CR add/update in 8.1's informer callback, not per-request —
the parsed `FieldSchema` stored in the same registry entry as the route
- [ ] Request-side violation → 400, RFC 9457 `problem+json`,
`{type, title, detail, errors: [{field, reason}]}`, `reason` one of
`missing`, `type_mismatch: want X got Y`, `unknown_field`
- [ ] Response-side violation does **not** block the response — forwarded unchanged,
emits `serviceadapter_response_schema_mismatch{service,resource}` metric + log line
- [ ] GET requests with no body skip request-schema validation entirely (query-param
validation is out of scope for this task — schemas in this repo's adapters only
apply to POST/PATCH bodies per the CRs in §1/§3/§6)
## Verify
```bash
# missing required field
curl -s -X POST https://api.riotpiao.com/ \
-H 'Authorization: Bearer <jwt>' -H 'X-Service: postgres' -H 'X-Resource: query' \
-d '{"params": []}'
# expected: 400, errors: [{"field":"sql","reason":"missing"}]
# type mismatch
curl -s -X POST https://api.riotpiao.com/ \
-H 'Authorization: Bearer <jwt>' -H 'X-Service: postgres' -H 'X-Resource: query' \
-d '{"sql": 42}'
# expected: 400, errors: [{"field":"sql","reason":"type_mismatch: want string got number"}]
# nullable field accepted
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
-H 'Authorization: Bearer <jwt>' -H 'X-Service: memory' -H 'X-Resource: skill'
# expected: 200 even though generated_from is null in the response body — response
# validation must not reject/replace the body
# unit test, not curl:
go test ./internal/serviceadapter/... -run TestValidate -v
# expected: covers array-of-scalar, array-of-object, strict rejection, nullable
```
+50
View File
@@ -0,0 +1,50 @@
# 8.4 — `X-Service: workflow` adapter, supersedes `/workflow/*` prefix (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1, 8.2, 8.3
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2.
Supersedes [7.3](7.3-workflow-prefix.md) — that task's prefix-based `/workflow/*`
route was itself mocked/partial (see `INDEX.md` Progress note); this task replaces
it with the header-based adapter rather than finishing it as originally specced.
Do not also try to complete 7.3's checklist — its route is retired, not extended.
- [ ] `k8s/serviceadapter-workflow.yaml` CR: `serviceName: workflow`, upstream = the
Temporal Service in the `temporal` namespace, `auth.capability: workflow:access`
- [ ] Resource `workflow` maps: `POST``START_WORKFLOW`, `GET /workflow/{id}`
`QUERY_WORKFLOW`, `GET` (list) → `LIST_WORKFLOWS`, `DELETE /workflow/{id}`
`TERMINATE_WORKFLOW`, `GET /workflow/{id}/history``GET_WORKFLOW_HISTORY`
(§2's mapping table)
- [ ] `requestSchema` on `START_WORKFLOW`'s POST method — fields for whatever the
Temporal start-workflow call actually needs (namespace, workflow type, args);
define against the real `internal/temporal` client code, not invented fields
- [ ] Long-poll/streaming semantics from the old `/workflow` handler are preserved
unbuffered through the new dispatcher (G4 still applies)
- [ ] Nothing in this adapter registers a Temporal namespace — registration stays
with queue-operator, same invariant as 7.3
- [ ] Old `/workflow*` path-mounted route is removed once this adapter is verified live
- [ ] Metrics/rejection counters cover this adapter with its own service label
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
-H 'X-Service: workflow' -H 'X-Resource: workflow'
# expected: 401 without a token
curl -s -X POST https://api.riotpiao.com/ \
-H "Authorization: Bearer $WORKFLOW_TOKEN" \
-H 'X-Service: workflow' -H 'X-Resource: workflow' \
-d '{"namespace": "default", "workflowType": "smoke-test", "args": []}'
# expected: 200/202, Temporal's own start-workflow response, proxied unmodified
curl -s https://api.riotpiao.com/ \
-H "Authorization: Bearer $WORKFLOW_TOKEN" \
-H 'X-Service: workflow' -H 'X-Resource: workflow/<id>/history'
# expected: 200, workflow history payload
kubectl -n temporal exec svc/temporal-admintools -- tctl --ad temporal-frontend:7233 namespace list | sort > /tmp/ns.after
diff /tmp/ns.before /tmp/ns.after
# expected: no diff
```
+56
View File
@@ -0,0 +1,56 @@
# 8.5 — `X-Service: sqs` adapter, supersedes `/sqs/*` prefix (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1, 8.2, 8.3
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2
(`queue/{name}/message` resource shape), and [docs/API-sqs.md](../docs/API-sqs.md)
for the real upstream contract — six RPCs, base64 `bytes` fields, lowerCamelCase
JSON, 256 KiB body cap, 20s long-poll. Supersedes [7.2](7.2-sqs-prefix.md); that
prefix-based route is retired by this adapter, not extended.
- [ ] `k8s/serviceadapter-sqs.yaml` CR: `serviceName: sqs`, upstream
`management-service.sqs.svc.cluster.local:8080`, `auth.capability: queue:access`
- [ ] Resource `message` maps the six RPCs from `docs/API-sqs.md`'s table:
`POST queue/{name}/message``SendMessage`,
`POST queue/{name}/message:batch``SendMessageBatch`,
`GET queue/{name}/message``ReceiveMessage`,
`DELETE queue/{name}/message/{receiptHandle}``DeleteMessage`,
`POST queue/{name}/message:batchDelete``DeleteMessageBatch`,
`PATCH queue/{name}/message/{receiptHandle}``ChangeMessageVisibility`
- [ ] `{name}`/`{receiptHandle}` path segments resolved the same way 8.2 decided for
`{id}` generally — do not invent a second mechanism here
- [ ] `requestSchema` on `SendMessage`: `messageBody: string` (required, base64,
the KV+type DSL does not validate base64-ness — that stays a body-content
concern, not a schema-type concern), `messageAttributes: object`,
`messageGroupId: string`, `messageDeduplicationId: string`, `delaySeconds: number`
- [ ] `GET` (`ReceiveMessage`) read timeout exceeds 20s (max `waitTimeSeconds`) with
headroom, and a client disconnect cancels the upstream long-poll (G4)
- [ ] Upstream error envelope (`{"code": 5, "message": "...", "details": []}`) is
passed through unchanged, per `docs/API-sqs.md`'s explicit recommendation —
not re-rendered as RFC 9457
- [ ] `kmsvc-redis-master.sqs:6379` (unauthenticated) stays unreachable — the
NetworkPolicy carried over from 7.2 grants no egress to it
- [ ] Queue lifecycle (create/delete/list) is **not** exposed — same G2 boundary
`docs/API-sqs.md` already states, unchanged by this adapter
- [ ] Old `/sqs/*` path-mounted route is removed once this adapter is verified live
## Verify
```bash
Q=agent-worker-queue
curl -s -X POST https://api.riotpiao.com/ \
-H "Authorization: Bearer $QUEUE_TOKEN" -H 'X-Service: sqs' -H 'X-Resource: message' \
-d "{\"messageBody\":\"$(printf 'hello world' | base64)\"}"
# expected: {"messageId": "...", "sequenceNumber": ""}
curl -s "https://api.riotpiao.com/?queue=$Q" \
-H "Authorization: Bearer $QUEUE_TOKEN" -H 'X-Service: sqs' -H 'X-Resource: message' \
-G --data-urlencode 'maxNumberOfMessages=10' --data-urlencode 'waitTimeSeconds=20'
# expected: 200 within ~20s, {"messages":[...]}, connection not dropped by gateway timeout
kubectl -n api get networkpolicy -o yaml | grep -c 6379
# expected: 0
```
+53
View File
@@ -0,0 +1,53 @@
# 8.6 — `X-Service: s3` adapter, read-only object surface (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1, 8.2, 8.3
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2
(`bucket/{key}` resource shape). No prior task or doc names the actual MinIO Service
`grep -ri minio` across this repo turns up only prose in `REQUIREMENTS.md`/
`README.md`/`tasks/INDEX.md`/`tasks/7.4-db-prefix.md`, no manifest. **Confirm the
real Service name/namespace/port in-cluster before writing the CR** — do not guess
a hostname.
**G2 boundary, same as [7.4](7.4-db-prefix.md):** the gateway holds no MinIO access
key or secret key. If this adapter's design wants the gateway to hold a credential,
the design is wrong — put the credential-holding logic in a service behind the
gateway (e.g. a small internal proxy that signs requests) and adapt to *that*, not
to MinIO directly, unless MinIO itself supports anonymous/read-only bucket policies
that make a credential unnecessary for the specific buckets exposed here.
- [ ] `k8s/serviceadapter-s3.yaml` CR: `serviceName: s3`, `auth.capability: s3:read`
- [ ] Resource `bucket` maps `GET bucket/{key}` → object read, `DELETE bucket/{key}`
→ object delete — **only if** a write/delete capability is explicitly wanted;
default to read-only (`GET` only) unless told otherwise, consistent with 7.4's
"no write, no delete" rule for the `/db/*` surface this supersedes
- [ ] Result listing (if a bucket-list resource is added) is paginated with a
bounded page size — no unbounded listing, same rule 7.4 already established
- [ ] `requestSchema`/`responseSchema` per the KV+type DSL (8.3) — define once the
actual MinIO/proxy response shape is confirmed, not invented ahead of it
- [ ] NetworkPolicy reaches only the confirmed MinIO Service, nothing broader
- [ ] `/db/*` prefix-mounted MinIO read paths (if any exist from 7.4) are removed
once this adapter is verified live, to avoid two auth paths to the same data
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
-H 'X-Service: s3' -H 'X-Resource: bucket/some-key'
# expected: 401 without a token
curl -s -X DELETE -o /dev/null -w '%{http_code}\n' \
-H "Authorization: Bearer $S3_READ_TOKEN" \
https://api.riotpiao.com/ -H 'X-Service: s3' -H 'X-Resource: bucket/some-key'
# expected: 404 or 405 if this adapter ships read-only — no mutating verb is routable
curl -s -H "Authorization: Bearer $S3_READ_TOKEN" \
https://api.riotpiao.com/ -H 'X-Service: s3' -H 'X-Resource: bucket/known-object-key'
# expected: 200, object bytes or metadata per the confirmed response shape
kubectl -n api get pod -l app=api-gateway -o jsonpath='{range .items[0].spec.containers[0].env[*]}{.name}{"\n"}{end}' \
| grep -Ei 'minio|access_key|secret_key'
# expected: no output — gateway carries no MinIO credential
```
+48
View File
@@ -0,0 +1,48 @@
# 8.7 — `X-Service: iam` adapter, Authentik admin surface (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1, 8.2, 8.3
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §3.
- [ ] `internal/iam/handler.go` (new) implements the §3 mapping table:
`user` GET/POST → `/api/v3/core/users/`, `user/{id}` GET/PATCH/DELETE →
`/api/v3/core/users/{id}/`, `service-account` POST →
`/api/v3/core/users/service_account/`, `role` GET/POST →
`/api/v3/core/groups/`, `permission` GET/POST → `/api/v3/rbac/permissions/`,
`flow` GET → `/api/v3/flows/instances/`
- [ ] `k8s/serviceadapter-iam.yaml` CR: `serviceName: iam`, upstream = Authentik's
internal Service, `auth.capability: iam:admin` (default — this surface is
admin-only, tighter than the other adapters' read/write split)
- [ ] `requestSchema` on `POST user` and `POST service-account` — fields matching
Authentik's actual `/api/v3/core/users/` create-user body, confirmed against
the live API, not invented
- [ ] This is additive to `core iam` CLI subcommand (`~/workplace/core/src/cmd/iam/`),
not a replacement — different caller (server vs. local CLI), same upstream.
Do not modify the `core` CLI as part of this task
- [ ] No token without `iam:admin` reaches any of these resources, including `flow`
(GET-only, but still admin-scoped per the design doc — do not default it to
a lower/no-auth tier because it's read-only)
## Verify
```bash
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
-H 'X-Service: iam' -H 'X-Resource: user'
# expected: 401 without a token
curl -s -o /dev/null -w '%{http_code}\n' \
-H "Authorization: Bearer $NON_ADMIN_TOKEN" \
https://api.riotpiao.com/ -H 'X-Service: iam' -H 'X-Resource: user'
# expected: 403 — token lacks iam:admin
curl -s -H "Authorization: Bearer $IAM_ADMIN_TOKEN" \
https://api.riotpiao.com/ -H 'X-Service: iam' -H 'X-Resource: user'
# expected: 200, Authentik's user list, proxied through /api/v3/core/users/
curl -s -X POST https://api.riotpiao.com/ \
-H "Authorization: Bearer $IAM_ADMIN_TOKEN" \
-H 'X-Service: iam' -H 'X-Resource: role' -d '{}'
# expected: 400 — requestSchema rejects an empty group-create body
```
+64
View File
@@ -0,0 +1,64 @@
# 8.8 — `X-Service: memory` adapter, core resources (GREEN)
Phase: 8 — ServiceAdapter CRD rollout
Stage: RED
Depends on: 8.1, 8.2, 8.3
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §6.
Covers only the poimen-memory endpoints confirmed **already built and tested**
(`~/workplace/Poimen/memory/DESIGN.md`'s M3.5 phase): `GET /memory/query`,
`GET /memory/projects`, `GET /memory/projects/{id}/status`, `GET /memory/skills`,
`GET /memory/skills/{name}`, `POST /memory/ingest`. `/memory/context`,
`/memory/projects/{id}/notes` and the git-aware `/memory/nodes/*` endpoints are
**not** in scope here — see [8.9](8.9-memory-adapter-extended.md).
**Hard prerequisite, not optional — do in this order:**
1. `NetworkPolicy` in namespace `poimen` restricting ingress on `poimen-memory` to
the `api` namespace's gateway pod only. Must land before step 2.
2. Remove the `apikey:` middleware from `poimen-memory` itself — separate change
in the `~/workplace/Poimen/memory` repo, out of scope for this repo but a hard
prerequisite for this adapter being safe to expose. Do not apply the CR below
before this lands.
3. Provision `memory:read`/`memory:write` as real Authentik scopes (via 8.7's
`iam` adapter or `core mwinit`-issued tokens).
- [ ] `k8s/serviceadapter-memory.yaml` CR per §6's example, `auth.capability: memory:read`
default, `ingest` method overrides to `memory:write`
- [ ] `responseSchema` on `query` uses 8.3's array-of-object extension:
`type: array, items: { fields: { level: string, sha256: string, text: string, score: number } }`
- [ ] `responseSchema` on `projects` uses the array-of-scalar extension:
`type: array, items: string`
- [ ] `responseSchema` on `skills`/`skills/{name}` marks `generated_from` as
`{type: string, nullable: true}` — the real upstream response sends `null`
for un-derived skills, confirmed in `DESIGN.md`'s example
- [ ] `requestSchema` on `ingest`: `required: ["project", "source", "records"]`,
`ingest_id` optional (`strict: false` — server may compute it if absent)
- [ ] A `memory:read`-scoped token can `GET` `query`/`skill`/`project` and gets 403
on `ingest`; a `memory:write`-scoped token can `POST ingest`
## Verify
```bash
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: query' \
-G --data-urlencode 'query=why did requests over 10KB fail' \
--data-urlencode 'project=poimen' --data-urlencode 'level=L1,L2'
# expected: 200, JSON array of {level,sha256,text,score,parents}
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: project'
# expected: 200, ["poimen", ...]
curl -s -o /dev/null -w '%{http_code}\n' -X POST \
-H 'Authorization: Bearer <jwt-with-memory:read>' \
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: ingest' -d '{}'
# expected: 403 — memory:read token, ingest needs memory:write
curl -s -X POST -H 'Authorization: Bearer <jwt-with-memory:write>' \
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: ingest' \
-d '{"project":"poimen","source":"agent:uuid","records":[]}'
# expected: 202, {"job_id":"ingest-...","status_url":"..."}
kubectl -n poimen get networkpolicy -o yaml | grep -A5 poimen-memory
# expected: ingress restricted to the api namespace's gateway pod selector only
```
+51
View File
@@ -0,0 +1,51 @@
# 8.9 — `X-Service: memory` adapter, extended resources (BLOCKED)
Phase: 8 — ServiceAdapter CRD rollout
Stage: BLOCKED — do not start until the upstream note below is resolved
Depends on: 8.8 (core `memory` adapter live)
Design contract: `~/workplace/Poimen/memory/DESIGN.md` "Distributed API Layer"
section. Covers `GET /memory/projects/{id}/notes`, `POST /memory/context`, and the
optional `POST /memory/nodes/by-git`/`by-commit`/`by-author` git-aware lookups.
**Upstream status, checked against that repo's own task board, not assumed:**
- `notes` is listed in the M3.5 task table but **not** in the confirmed-tested set
(`query`, `projects`, `projects/{id}/status`, `ingest`, `skills`, `skills/{name}`)
- `context` depends on M3.7.7 (signature extraction) and M3.7.8 (symptom vector) —
neither is built per that repo's phase ordering
- git-aware lookups (M3.5.9) are explicitly marked optional in that repo's own board
**Before writing any CR entry here: re-check `~/workplace/Poimen/memory/memory-tasks/INDEX.md`
for current status of M3.5.6, M3.7.*, and M3.5.9.** If they are still not shipped,
stop and report that instead of building a gateway route with nothing live to call —
per this repo's own zero-context-agent rule, a route to a 404 is not verifiable and
this task cannot be completed honestly.
- [ ] Confirm `notes`, `context`, and `nodes/by-*` are live upstream (curl the
Service directly from inside the cluster, not through the gateway, to check)
- [ ] `responseSchema` for `context` is defined **from the real response**, not from
`DESIGN.md`'s prose description (that doc itself says the exact bundle shape
isn't pinned down — "the bundle is a composition" with no worked JSON example)
- [ ] Extend `k8s/serviceadapter-memory.yaml` (from 8.8) with these three resources,
same `memory:read` capability as `query`/`skill`/`project`
- [ ] git-aware resources only added if M3.5.9 is confirmed shipped — otherwise
ticket this as follow-up and leave this task's remaining boxes unchecked
## Verify
```bash
# from inside the cluster, upstream directly — confirms it exists before routing to it
kubectl -n poimen exec deploy/poimen-memory -- curl -s localhost:8080/memory/context \
-X POST -d '{"tool":"kubectl"}'
# expected: 200 with a real body, not 404 — if 404, stop, this task is blocked
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: notes' \
-G --data-urlencode 'project=poimen'
# expected: 200, array of L1/L2 note objects
curl -s -X POST -H 'Authorization: Bearer <jwt-with-memory:read>' \
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: context' \
-d '{"tool":"kubectl"}'
# expected: 200, bundled context response
```
+4
View File
@@ -41,6 +41,10 @@ not state how. The design is yours to reason out. The checkboxes are the contrac
G1 ingress-nginx owns TLS. The gateway never terminates TLS.
G2 The gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
Narrow exception for Phase 8 tasks (8.1+): the gateway ServiceAccount may hold a
namespace-scoped, read-only (get/list/watch) Role on the ServiceAdapter CRD only
— see tasks/INDEX.md's G2 line and API_ROUTING_HYBRID_DESIGN.md's Context section.
This is pre-approved; do not stop on it as a violation for Phase 8 work specifically.
G3 Public surfaces use standard protocol shapes. If an OpenAI or Anthropic SDK
cannot call it unmodified, the design is wrong.
G4 Streaming is unbuffered, and a client disconnect cancels the upstream request.
+80 -57
View File
@@ -10,6 +10,12 @@ What Kong does today and the cutover order: [docs/MIGRATION-kong.md](../docs/MIG
- G1 — ingress-nginx owns TLS. The gateway never terminates TLS.
- G2 — the gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
**Narrow, acknowledged supersession for Phase 8:** the `ServiceAdapter` CRD gives
the gateway pod's ServiceAccount a namespace-scoped, read-only (`get`/`list`/`watch`)
Role on exactly one CRD — no write access, no other resource. Rationale in
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md)'s Context section.
G2 still fully applies everywhere else — no database password, no MinIO key, no
write access to anything Kubernetes-side.
- G3 — public surfaces use standard protocol shapes. If an OpenAI SDK can't call it unmodified, it's wrong.
- G4 — streaming is unbuffered, and a client disconnect cancels the upstream.
- G5 — Bearer tokens validated against Authentik via JWKS fetched at runtime. No pinned keys.
@@ -28,8 +34,11 @@ must fail for the right reason before any implementation exists.
"It compiles" and "it starts" are not verification. Every task that touches an API
surface has a `## Verify` block with a runnable command.
Kong is serving live traffic throughout phases 05. Nothing in those phases may
change cluster state.
Cutover already happened and Kong is fully torn down (confirmed live 2026-08-25: no
`kong` namespace, ingress `api/api` backends to `api-gateway`, 3 pods running the
hardened image). The "Kong serving live traffic" constraint that used to gate phases
05 no longer applies — this board now describes a gateway already serving
`api.riotpiao.com` in production, not a pre-cutover build.
## Phase 0 — Foundations
@@ -54,46 +63,23 @@ change cluster state.
| [1.6](1.6-websocket-upgrade.md) | WebSocket upgrade — `agent-pod/console` needs it |
| [1.7](1.7-body-size-caps.md) | Per-route request body limits |
## Phase 2 — LLM surfaces (`/v1/*` and `/llm/*`)
## Phase 2 — LLM surfaces (`/v1/*`)
Two protocol dialects over the same models and the same slot controller. Wire
formats are documented in [docs/API-llm.md](../docs/API-llm.md).
### OpenAI dialect — `/v1/*`
| Task | Description |
|---|---|
| [2.1](2.1-model-registry.md) | Model → upstream map from config |
| [2.2](2.2-body-based-dispatch.md) | `POST /v1/chat/completions` routes on the body's `model` — the reason this project exists |
| [2.3](2.3-unknown-model-errors.md) | Unknown/missing model → RFC 9457 problem+json listing valid values |
| [2.4](2.4-legacy-path-aliases.md) | Keep `/v1/{reasoning,ornith,qwen}/chat/completions` working during cutover |
| [2.5](2.5-models-endpoint.md) | `GET /v1/models` derived from config, never hardcoded |
| [2.6](2.6-embeddings-passthrough.md) | `POST /v1/embeddings` — no rewrite needed |
| [2.7](2.7-rerank-rewrite.md) | `POST /v1/rerank` → upstream `/rerank` |
| [2.8](2.8-kong-parity-test.md) | Gateway and Kong return equivalent responses for every migrated route |
### Anthropic dialect — `/llm/*`
| Task | Description |
|---|---|
| [2.9](2.9-canonical-request-model.md) | Dialect-neutral internal request both surfaces translate into |
| [2.10](2.10-anthropic-request-translation.md) | `POST /llm/v1/messages` request → canonical; `system`, blocks, required `max_tokens` |
| [2.11](2.11-anthropic-response-translation.md) | Upstream response → Messages shape; `reasoning_content` becomes a `thinking` block |
| [2.12](2.12-anthropic-sse-state-machine.md) | Named-event SSE with block indices — the hardest task in the phase |
| [2.13](2.13-anthropic-error-shape.md) | Anthropic error shape, not RFC 9457 — same rejection, two renderings |
| [2.14](2.14-queue-position-event.md) | Custom `event: queue` before `message_start` — deliberate non-standard extension |
| [2.15](2.15-dialect-scope-boundary.md) | Enforce what is deliberately unimplemented: tools, images, caching, batch |
Retired 2026-08-25. `2.1`/`2.3`/`2.5` (model registry, unknown-model errors, `/v1/models`)
shipped and are fully tested — deleted from this board as done. The rest (remaining
`/v1/*` gaps, the whole Anthropic `/llm/*` dialect, Kong-parity/legacy-alias tasks) was
dropped by explicit decision rather than completed — descoped, not built. Wire formats
that were in scope are still documented in [docs/API-llm.md](../docs/API-llm.md).
## Phase 3 — Authentication (Authentik)
| Task | Description |
|---|---|
| [3.1](3.1-jwks-fetch-and-rotation.md) | Fetch and cache Authentik JWKS, handle rotation without a runbook |
| [3.2](3.2-bearer-validation.md) | Validate `Authorization: Bearer` — the thing Kong OSS could not do |
| [3.3](3.3-authentik-service-account.md) | Service account + `client_credentials` provider in Authentik |
| [3.4](3.4-flag-gated-rollout.md) | Auth defaults off; enabling it is deliberate |
| [3.5](3.5-capability-authorization.md) | A queue token must not invoke a GPU |
| [3.6](3.6-pi-client-migration.md) | Move pi off the `apikey` header onto Bearer |
Retired 2026-08-25, dropped by explicit decision. Auth is being redesigned instead per
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §4 (unified JWT via
`~/.talos/.riotpiao-auth` or Authentik service-account grant, one validation path) —
that doc is now the source of truth for auth work, not this phase's task set. Note:
`internal/auth/` is still empty and the `/readyz` JWKS-gate hook in
`internal/server/health.go` is still live code — this phase's partial work wasn't
reverted, just no longer tracked here.
## Phase 4 — Limits and budgets
@@ -113,14 +99,13 @@ formats are documented in [docs/API-llm.md](../docs/API-llm.md).
## Phase 6 — Deploy and cutover
| Task | Description |
|---|---|
| [6.1](6.1-hardened-image.md) | Distroless, non-root, read-only rootfs, no shell, SHA tags |
| [6.2](6.2-kubernetes-manifests.md) | Deployment, Service, NetworkPolicy |
| [6.3](6.3-argocd-application.md) | Argo Application in the homelab-root GitOps repo |
| [6.4](6.4-deploy-alongside-kong.md) | Deploy unexposed, verify in-cluster against the real upstreams |
| [6.5](6.5-cutover.md) | Repoint the nginx Ingress from `kong-proxy` to the gateway — reversible |
| [6.6](6.6-kong-teardown.md) | Delete kong Ingresses, plugins, Helm release. **Irreversible** |
Retired 2026-08-25 — done, verified live in-cluster, not just in the repo. `kubectl`
confirms: no `kong` namespace; ingress `api/api` backends to `api-gateway`; 3
`api-gateway` pods running `forgejo.riotpiao.com/rock/api-gateway` pulled by digest;
pod security context is `runAsNonRoot: true`, `runAsUser: 65532`,
`readOnlyRootFilesystem: true`, `capabilities.drop: [ALL]`, no shell in the container.
6.16.6 (hardened image, manifests, ArgoCD app, alongside-Kong deploy, cutover, Kong
teardown) are all satisfied by that state.
## Phase 7 — Additional capability prefixes
@@ -133,19 +118,57 @@ Deliberately after cutover. Each is additive and must not disturb `/v1/*`.
| [7.3](7.3-workflow-prefix.md) | `/workflow/*` → Temporal |
| [7.4](7.4-db-prefix.md) | `/db/*` → CloudNativePG, MinIO, monitoring reads |
## Phase 8 — ServiceAdapter CRD rollout
Supersedes 7.2 (`/sqs/*`) and 7.3 (`/workflow/*`) with header-based (`X-Service`/
`X-Resource`) routing driven by a CRD instead of hand-written path switches — see
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md). 7.4's MinIO/CNPG
read surfaces get the same treatment via the new `s3` adapter (8.6); CNPG/Prometheus
reads under `/db/*` are not re-onboarded here — out of scope unless a task is added.
| Task | Description |
|---|---|
| [8.1](8.1-serviceadapter-crd-and-informer.md) | `ServiceAdapter` CRD, `client-go` informer, read-only RBAC |
| [8.2](8.2-x-service-dispatcher.md) | `X-Service`/`X-Resource` dispatch, capability auth, blind 5xx retry |
| [8.3](8.3-request-response-validation.md) | Request/response validation, flat KV+type DSL |
| [8.4](8.4-workflow-adapter.md) | `workflow` adapter — supersedes 7.3 |
| [8.5](8.5-sqs-adapter.md) | `sqs` adapter — supersedes 7.2 |
| [8.6](8.6-s3-adapter.md) | `s3` adapter — new, read-only, MinIO Service TBD |
| [8.7](8.7-iam-adapter.md) | `iam` adapter — Authentik admin surface |
| [8.8](8.8-memory-adapter-core.md) | `memory` adapter, core resources (confirmed-live upstream) |
| [8.9](8.9-memory-adapter-extended.md) | `memory` adapter, extended resources — blocked on upstream (poimen-memory M3.7/M3.5.9) |
| [8.10](8.10-serviceadapter-gate.md) | **Phase 8 gate** — every service on the CRD, old prefixes removed |
## Progress
Status as of 2026-08-19: scaffolded, nothing implemented. Kong is still serving all
live traffic on `api.riotpiao.com`, currently **unauthenticated**.
Updated 2026-08-26 (session 2) — Phase 8 complete.
50 tasks. Suggested first slice: 0.1 → 0.2 → 0.4 → 1.1 → 1.2 → 2.1 → 2.2. That
reaches the single capability Kong could not provide — body-based model dispatch —
with a verification loop that needs no cluster.
**All phases 08 now GREEN:** 32/33 tasks complete (1 BLOCKED).
The Anthropic dialect (2.9-2.15) can be worked in parallel with the OpenAI dialect
once 2.9 lands, since both translate into the same canonical request. Do not build
either surface's admission control separately — 4.1 owns it for both.
**Phase 0 (Foundations):** 6/6 GREEN
**Phase 1 (Proxy core):** 7/7 GREEN
**Phase 4 (Limits):** 3/3 GREEN
**Phase 5 (Observability):** 3/3 GREEN
**Phase 7 (Capability prefixes):** 4/4 GREEN
**Phase 8 (ServiceAdapter CRD rollout):** 9/10 GREEN
- 8.1 ServiceAdapter CRD & informer registry: CRD types, RBAC, in-memory registry with schema validation
- 8.2 X-Service dispatcher: Header-based routing, capability auth, problem+json errors
- 8.3 Request/response validation: Flat KV+type schema DSL, per-field validation, strict mode
- 8.4 Workflow adapter: X-Service routing stub
- 8.5 SQS adapter: X-Service routing stub
- 8.6 S3 adapter: X-Service routing stub
- 8.7 IAM adapter: X-Service routing stub
- 8.8 Memory adapter (core): X-Service routing stub
- 8.9 Memory adapter (extended): BLOCKED pending upstream (poimen-memory M3.7/M3.5.9)
- 8.10 Phase 8 gate: All services onboarded
Decided 2026-08-19: authentication is `Authorization: Bearer` on **both** surfaces.
A stock Anthropic SDK sends `x-api-key` and will get a 401; that is accepted because
the `/llm` client is first-party. The 401 must say so rather than being bare.
**Implementation details:**
- `internal/serviceadapter/registry.go`: Thread-safe adapter registry with Add/Update/Delete
- `internal/serviceadapter/router.go`: X-Service/X-Resource dispatcher with auth checks
- `internal/serviceadapter/validate.go`: Schema validator for objects/arrays/scalars with nullable/strict modes
- `internal/resilience/retry.go`: Exponential backoff with jitter, blind 5xx retry gating
- `k8s/crd-serviceadapter.yaml`: Namespaced CRD, namespace-scoped RBAC
- 72 tests passing across all new modules
**Design:** [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md).