refactor: consolidate docs, write unified API reference
Deleted: - 43 outdated/completed task files (phases 0-8) - All design docs (REQUIREMENTS, ADR, routing design, etc) - Phase-specific docs (temporal, JWT, tool calls, testing guides) - Redundant API docs (API-llm, API-sqs, SERVICE-USAGE) Kept: README.md (project overview) New: Comprehensive API.md - Single source of truth for api.riotpiao.com - All services in one place: LLM, workflows, queues, memory, IAM, S3 - Complete request/response examples - Authentication via JWT bearer tokens + capabilities - Error handling (RFC 9457) - Rate limits, timeouts, status codes - Real-world examples (RAG pipeline, workflow orchestration) Benefits: ✅ Developer finds everything in API.md ✅ No duplicate/stale docs ✅ Reduced maintenance burden ✅ Single source of truth
This commit is contained in:
Admin Bot
@@ -1,532 +0,0 @@
|
||||
# API Routing Design: Header-Based Services, CRD-Driven Onboarding, Unified Auth
|
||||
|
||||
Supersedes the earlier `/unified`-endpoint draft of this doc. Routing is now
|
||||
header-only at the gateway root — `api.riotpiao.com` itself is the single entry
|
||||
point, no dedicated sub-path. New backend services onboard via a CRD the gateway
|
||||
consumes directly, not via hand-written Go switch cases.
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Most of the original hybrid design was aspirational — it didn't exist in code. This
|
||||
revision reflects decisions made while reviewing the repo against the actual state:
|
||||
`internal/auth/` was empty, `s3`/`sqs`/`iam` didn't exist as handlers, and only a
|
||||
body-action `POST /workflow` endpoint was real. Four extensions were decided:
|
||||
CRD-driven service onboarding, `X-Resource`-based REST verbs replacing body actions,
|
||||
unified JWT auth (bearer token or service-account password, one validation path), and
|
||||
blind 5xx retry.
|
||||
|
||||
Known constraints, and how these decisions land on them:
|
||||
- Rule G2 (originally specced in the now-retired `tasks/6.2-kubernetes-manifests.md`,
|
||||
deleted 2026-08-25 since Phase 6 shipped and is live in-cluster) mandated **no
|
||||
ServiceAccount/RBAC** on the gateway pod, and routes loaded from a git-sourced
|
||||
ConfigMap with a pod roll on every change. The gateway pod now consumes the CRD
|
||||
directly — no separate operator/controller pod. This is a deliberate, acknowledged
|
||||
supersession of G2 for this one narrow purpose: the existing `api-gateway`
|
||||
`ServiceAccount` (already bound to the pod at `k8s/rbac.yaml:1-7`, currently with
|
||||
zero Role — see `k8s/rbac.yaml:9-10`'s own comment "the gateway has no k8s API
|
||||
access") gains a namespace-scoped `Role` limited to read-only access on one CRD,
|
||||
defined in this same repo and synced by the existing `api-gw` ArgoCD Application
|
||||
(`homelab/k8s/argocd/apps/55-api-gateway.yaml`) — no new repo, no new Application.
|
||||
Static ConfigMap-sourced routes keep the existing roll-on-change behavior;
|
||||
CRD-sourced routes update live in-process instead (see §1).
|
||||
- Retired `kmsvc.io` `TemporalWorker`/`Queue` CRD (`homelab/k8s/apps/messaging/queue-crd/`,
|
||||
controller in the separate `kmsvc-manage` repo) broke via an uncleared finalizer and
|
||||
RBAC rules that drifted out of sync with the CRD's resources
|
||||
(`temporalworkers.kmsvc.io is forbidden ... at the cluster scope`). Cluster-scoped
|
||||
RBAC, RBAC/CRD drift, and finalizer complexity are explicitly avoided this time —
|
||||
this CRD is namespace-scoped (`api` only) and read-only from the gateway's side.
|
||||
|
||||
---
|
||||
|
||||
## 0. Repo hygiene — do this first, before any code
|
||||
|
||||
**Already staged for deletion (`git status`) — just needs a commit:**
|
||||
`DELIVERY_COMPLETE.md`, `IMPLEMENTATION_SUMMARY.md`, `PHASE3_GRPC_IMPLEMENTATION.md`,
|
||||
`PHASE3_PROGRESS.md`, `TEMPORAL_API_DESIGN_SUMMARY.md`,
|
||||
`TEMPORAL_IMPLEMENTATION_CHECKPOINT.md`, `TEMPORAL_IMPLEMENTATION_COMPLETE.md`,
|
||||
`TEMPORAL_TEST_REPORT.md`, `WORKFLOWS_INDEX.md`, `WORKFLOWS_QUICK_START.md`,
|
||||
`WORKFLOWS_README.md`.
|
||||
|
||||
**Removed as part of this revision:** `IMPLEMENTATION_PLAN_UPDATED.md` — its reading
|
||||
order pointed at this doc's now-superseded `/unified`-endpoint design; folded into
|
||||
this doc instead of kept as a separate index.
|
||||
|
||||
**Untracked, flagged but left alone (different topic, not touched by this doc):**
|
||||
`AUTHENTIK_AS_KEY_MANAGER.md`, `AUTHENTIK_KEY_ROTATION_SIMPLE.md`,
|
||||
`AUTHENTIK_SOPS_NATIVE.md`, `AUTHENTIK_WITH_FIPS_HSM.md`, `JWT_VS_RANDOM_TOKEN.md` —
|
||||
SOPS secrets-at-rest key rotation via Authentik, unrelated to API routing/CRD work.
|
||||
|
||||
---
|
||||
|
||||
## 1. Service onboarding (new backend service = code change vs. YAML)
|
||||
|
||||
### Existing System
|
||||
- No CRD, no controller, no generic service registry anywhere in either repo.
|
||||
- Adding a service today means: write a new Go handler, add a `switch` case, redeploy
|
||||
the gateway image. Only `workflow` exists (`internal/temporal/handler.go`); `s3`,
|
||||
`sqs`, `iam` don't exist in code at all.
|
||||
- `internal/config/config.go` + `loader.go` load only static, hand-edited route config
|
||||
from the ConfigMap already live per Phase 6 (`k8s/configmap.yaml`) — no per-service
|
||||
schema, no CRD.
|
||||
|
||||
### Future System
|
||||
New CRD **`ServiceAdapter`** (`gateway.riotpiao.com/v1`, namespace-scoped to `api`
|
||||
only — deliberately *not* cluster-scoped, unlike the retired CRD). Named for what it
|
||||
actually does — adapts an in-cluster service to the gateway's
|
||||
`X-Service`/`X-Resource` contract — rather than a generic "Definition".
|
||||
|
||||
**No separate operator pod.** The API gateway pod itself consumes the CRD directly: it
|
||||
runs a `client-go` informer against `ServiceAdapter` in namespace `api` and keeps an
|
||||
in-memory route registry that adds/updates/removes entries as CRs change — no
|
||||
ConfigMap render step, no restart-to-pick-up-changes, no second binary/Deployment/image
|
||||
to build and operate.
|
||||
|
||||
**Where the CRD is defined — this repo, not `homelab`.** Confirmed at
|
||||
`homelab/k8s/argocd/apps/55-api-gateway.yaml:42-45`: the `api-gw` ArgoCD Application
|
||||
already sources this repo's own `k8s` path directly (kustomize — Deployment, Service,
|
||||
ConfigMap, RBAC, NetworkPolicy), auto-syncing on every push to `main`, wave 7. So the
|
||||
CRD manifest is just another file in that same kustomization: new
|
||||
`k8s/crd-serviceadapter.yaml`, added to `k8s/kustomization.yaml`'s `resources:` list.
|
||||
When a new `ServiceAdapter` CR is introduced (i.e. someone commits a CR YAML for a new
|
||||
service), ArgoCD syncs it through this same already-existing Application, and the
|
||||
already-running `api-gw` pod's informer notices it and starts accepting
|
||||
`X-Service: <new-service>` calls — no new Application, no new repo, no manual step in
|
||||
`homelab` at all. Go API types back the CRD from
|
||||
`apis/gateway/v1/serviceadapter_types.go` (new package, generates the CRD YAML via
|
||||
`controller-gen`, same struct also used by the informer).
|
||||
|
||||
`spec.auth.capability` is the default capability required for every resource on the
|
||||
service. A method entry may override it with its own `auth.capability` — needed once
|
||||
a service splits read vs. write access (see §6's `memory` adapter). No override means
|
||||
inherit the spec-level default; `auth.required: false` at either level means no
|
||||
capability check at all for that scope.
|
||||
|
||||
**Example CR** (onboarding a hypothetical `postgres` service — no Go code, no image
|
||||
rebuild, no manual gateway restart):
|
||||
|
||||
```yaml
|
||||
apiVersion: gateway.riotpiao.com/v1
|
||||
kind: ServiceAdapter
|
||||
metadata:
|
||||
name: postgres
|
||||
namespace: api
|
||||
spec:
|
||||
serviceName: postgres # matches X-Service: postgres
|
||||
upstream:
|
||||
url: http://kmsvc-postgres-helper.sqs.svc.cluster.local:8080
|
||||
timeoutSeconds: 10
|
||||
auth:
|
||||
required: true
|
||||
capability: postgres:access # checked against the caller's JWT claims
|
||||
retryable: true # eligible for the blind 5xx retry middleware, §4
|
||||
resources:
|
||||
- name: query # X-Resource: query
|
||||
methods:
|
||||
- verb: POST
|
||||
upstreamPath: /v1/query
|
||||
requestSchema:
|
||||
fields:
|
||||
sql: string
|
||||
params: array
|
||||
required: ["sql"]
|
||||
responseSchema:
|
||||
fields:
|
||||
rows: array
|
||||
- name: table
|
||||
methods:
|
||||
- verb: GET
|
||||
upstreamPath: /v1/tables/{id} # {id} filled from the trailing path segment
|
||||
- verb: DELETE
|
||||
upstreamPath: /v1/tables/{id}
|
||||
status:
|
||||
phase: Ready
|
||||
observedGeneration: 1
|
||||
```
|
||||
|
||||
Calling it once reconciled — straight to the gateway root, no dedicated sub-path:
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' \
|
||||
-H 'X-Service: postgres' \
|
||||
-H 'X-Resource: query' \
|
||||
-d '{"sql": "select 1"}'
|
||||
```
|
||||
|
||||
**RBAC** — extend `k8s/rbac.yaml` in place (same file that already defines the
|
||||
`api-gateway` `ServiceAccount` at lines 1-6, currently role-less per its own "the
|
||||
gateway has no k8s API access" comment). A namespace-scoped `Role` (not `ClusterRole`),
|
||||
read-only, nothing else — this is the entire RBAC footprint added to the gateway pod,
|
||||
shipped in the same repo/PR as the CRD itself:
|
||||
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: api-gateway-serviceadapter-reader
|
||||
namespace: api
|
||||
rules:
|
||||
- apiGroups: ["gateway.riotpiao.com"]
|
||||
resources: ["serviceadapters"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: api-gateway-serviceadapter-reader
|
||||
namespace: api
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: api-gateway-serviceadapter-reader
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: api-gateway
|
||||
namespace: api
|
||||
```
|
||||
|
||||
No `status`/`finalizers` verbs, no write access at all — the gateway only ever reads.
|
||||
No new `ServiceAccount`, no new Deployment, no new container image.
|
||||
|
||||
**Gateway-side consumer** — new `internal/serviceadapter/registry.go`: a `client-go`
|
||||
`SharedInformer` on `ServiceAdapter` in namespace `api`, feeding an in-memory map keyed
|
||||
by `spec.serviceName` that the `internal/serviceadapter` dispatcher (§2) reads on every
|
||||
request. Add/Update/Delete informer callbacks update the map directly — a new
|
||||
`ServiceAdapter` CR is live within the informer's resync/watch latency, no gateway
|
||||
restart, no ArgoCD change beyond the existing auto-sync of this repo's `k8s` path.
|
||||
|
||||
**Request/response validation** — `requestSchema`/`responseSchema` are not JSON
|
||||
Schema. Deliberately a flat key→type map, author-facing (whoever writes the CR
|
||||
specifies it, no JSON Schema knowledge needed):
|
||||
|
||||
```yaml
|
||||
requestSchema:
|
||||
fields:
|
||||
sql: string # one of: string | number | boolean | array | object
|
||||
params: array
|
||||
required: ["sql"] # optional; omit if every field is optional
|
||||
strict: false # optional, default false — true rejects unlisted keys
|
||||
responseSchema:
|
||||
fields:
|
||||
rows: array
|
||||
```
|
||||
|
||||
One level deep only — `params: array` checks "is this an array," not what's
|
||||
inside it. No enums, no formats, no nested field validation. Accepted limitation:
|
||||
none of the adapters in this doc (postgres, workflow, iam, memory) need more than
|
||||
top-level shape checking.
|
||||
|
||||
**Extension forced by real upstream shapes (poimen-memory, §6):** the DSL above
|
||||
only describes a top-level JSON *object*. `GET /memory/projects` returns a bare
|
||||
array (`["poimen", ...]`), `GET /memory/query` returns an array of objects, and
|
||||
`GET /memory/skills` returns objects with an explicit `"generated_from": null`.
|
||||
Neither is expressible in the object-only form, so two additions, kept minimal
|
||||
rather than backsliding toward full JSON Schema:
|
||||
|
||||
```yaml
|
||||
responseSchema:
|
||||
type: array # top-level shape; default "object" if omitted
|
||||
items: string # scalar array — every element must be this type
|
||||
responseSchema:
|
||||
type: array
|
||||
items:
|
||||
fields: # object array — one level deep INSIDE each item
|
||||
level: string
|
||||
sha256: string
|
||||
score: number
|
||||
responseSchema:
|
||||
fields:
|
||||
generated_from: { type: string, nullable: true } # long form only needed for nullable
|
||||
```
|
||||
|
||||
Rule: a field written as a bare type string (`sql: string`) is shorthand for
|
||||
`{type: string, nullable: false}`. `null` satisfies any field marked
|
||||
`nullable: true` regardless of its declared type; a `null` on a non-nullable
|
||||
field is a `type_mismatch`. Still one level deep — `items: { fields: {...} }`
|
||||
does not itself accept a nested `items`.
|
||||
|
||||
Validated in Go via `internal/serviceadapter/validate.go` (new, ~40 lines, no
|
||||
external dependency): `json.Unmarshal` the body into `map[string]interface{}`,
|
||||
check `Required` fields are present, check each present field in `Fields` against
|
||||
its Go runtime type (`string`→string, `float64`→number, `bool`→boolean,
|
||||
`[]interface{}`→array, `map[string]interface{}`→object), and if `strict: true`
|
||||
reject any body key not listed in `Fields`. Compiled once per CR add/update in the
|
||||
informer callback above, not per-request — the parsed `FieldSchema` struct is
|
||||
stored directly in the same registry map entry as the route.
|
||||
|
||||
Request-side violations are enforced: 400, RFC 9457 `problem+json`, body
|
||||
`{type, title, detail, errors: [{field, reason}]}` where `reason` is one of
|
||||
`missing`, `type_mismatch: want X got Y`, or `unknown_field` (strict mode only).
|
||||
Response-side violations are **not** enforced — the response is forwarded to the
|
||||
caller unchanged; a mismatch only emits a
|
||||
`serviceadapter_response_schema_mismatch{service,resource}` metric and a log line.
|
||||
Rationale: request validation is a trust boundary (external caller input);
|
||||
response validation is contract-drift detection against our own upstream — turning
|
||||
a backend field-shape typo into a caller-facing 502 does more harm than logging it.
|
||||
|
||||
---
|
||||
|
||||
## 2. Request shape: body action vs. header verb
|
||||
|
||||
### Existing System
|
||||
- `internal/temporal/handler.go`: `POST /workflow` only, body is
|
||||
`{action, namespace, payload}` (`RequestPayload` struct) — action dispatch happens
|
||||
by reading and branching on the JSON body, not by header/method.
|
||||
- `internal/server/router.go` dispatches purely by **path**: exact-matches
|
||||
`/healthz`, `/readyz`, `/workflow`, `/workflow/health`, `/workflow/metrics`, and
|
||||
falls through everything else (including `/v1/chat/completions`) to
|
||||
`upstreamHandler`. There is no header-based dispatch anywhere in the router today.
|
||||
- `s3`/`sqs`/`iam` bodies with `operation`/`action` fields, and a `/unified` sub-path,
|
||||
were only ever prose in an earlier draft of this doc — never implemented, and now
|
||||
dropped.
|
||||
|
||||
### Future System
|
||||
No new sub-path is introduced. `internal/server/router.go`'s `ServeHTTP` gets one new
|
||||
branch, checked **before** the existing path switch: if the request carries an
|
||||
`X-Service` header, dispatch straight to a new `internal/serviceadapter/router.go`
|
||||
dispatcher, regardless of `r.URL.Path` (callers hit `https://api.riotpiao.com/` at the
|
||||
root). `/v1/chat/completions`, `/v1/embeddings`, `/v1/rerank`, `/healthz`, `/readyz`
|
||||
keep their existing path-based routing in `internal/proxy/router.go` — those never
|
||||
carry `X-Service` and are matched first. `/workflow*` stays mounted during migration,
|
||||
delegating internally to the same dispatcher, then gets removed once callers move to
|
||||
header-based calls.
|
||||
|
||||
Dispatch key becomes **`X-Service` header → adapter**, then **HTTP method +
|
||||
`X-Resource` header → `{verb, upstreamPath}`** — e.g. `GET` to the gateway root +
|
||||
`X-Service: iam` + `X-Resource: user` → list Authentik users. This replaces the body
|
||||
`action`/`operation` field convention everywhere, not just for `iam` — so the
|
||||
rewritten `workflow` adapter looks like:
|
||||
- `POST X-Resource: workflow` → `START_WORKFLOW`
|
||||
- `GET X-Resource: workflow/{id}` → `QUERY_WORKFLOW`
|
||||
- `GET X-Resource: workflow` → `LIST_WORKFLOWS`
|
||||
- `DELETE X-Resource: workflow/{id}` → `TERMINATE_WORKFLOW`
|
||||
- `GET X-Resource: workflow/{id}/history` → `GET_WORKFLOW_HISTORY`
|
||||
- Same pattern for new `s3` (`bucket/{key}`) and `sqs` (`queue/{name}/message`)
|
||||
adapters — built fresh, since they don't exist in code today.
|
||||
|
||||
`/v1/chat/completions`, `/v1/embeddings`, `/v1/rerank` in `internal/proxy/router.go`
|
||||
are **unchanged** — local-LLM-only, no `X-Service` involvement.
|
||||
|
||||
---
|
||||
|
||||
## 3. IAM access to Authentik
|
||||
|
||||
### Existing System
|
||||
- No `X-Service: iam` adapter in the gateway.
|
||||
- Authentik admin API access exists today only client-side, in the Rust `core` CLI
|
||||
(`/Users/rockliang/workplace/core/src/cmd/iam/*.rs`), which already calls
|
||||
`{authentik_url}/api/v3/core/applications/`, `/api/v3/providers/oauth2/`,
|
||||
`/api/v3/core/flows/`, `/api/v3/crypto/certificatekeypairs/` for apps/providers/flows
|
||||
— confirmed at `core/src/cmd/iam/app.rs:136,160,191,205,219`. No users/groups/roles
|
||||
coverage there, and none of it is reachable through the gateway.
|
||||
|
||||
### Future System
|
||||
New `internal/iam/handler.go` server-side adapter, reachable at the gateway root with
|
||||
`X-Service: iam`, mapping `X-Resource` + method onto Authentik's `/api/v3/...` surface:
|
||||
|
||||
| X-Resource | Method | Authentik endpoint |
|
||||
|---|---|---|
|
||||
| `user` | GET/POST | `/api/v3/core/users/` |
|
||||
| `user/{id}` | GET/PATCH/DELETE | `/api/v3/core/users/{id}/` |
|
||||
| `service-account` | POST | `/api/v3/core/users/service_account/` |
|
||||
| `role` (group) | GET/POST | `/api/v3/core/groups/` |
|
||||
| `permission` | GET/POST | `/api/v3/rbac/permissions/` |
|
||||
| `flow` | GET | `/api/v3/flows/instances/` |
|
||||
|
||||
This is additive to, not a replacement of, the `core iam` CLI subcommand — different
|
||||
caller (server vs. local CLI), same upstream API.
|
||||
|
||||
---
|
||||
|
||||
## 4. Auth: how a caller proves identity
|
||||
|
||||
### Existing System
|
||||
- `internal/auth/` is an **empty directory** — zero implementation. `tasks/` Phase 3
|
||||
(JWKS fetch/rotation, Bearer validation, Authentik service account, flag-gated
|
||||
rollout, capability authorization) was retired 2026-08-25 — this doc is now the spec
|
||||
for that work instead of those deleted task files.
|
||||
- `kmsvc.riotpiao.com` (separate service) has an auth interceptor
|
||||
(`kmsvc-manage/internal/api/interceptors/auth.go`, JWKS validation via
|
||||
`lestrrat-go/jwx/v2/jwt`) but it's **never wired in** — `cmd/server/main.go` builds a
|
||||
bare `grpc.NewServer()` (`kmsvc-manage/cmd/server/main.go:84`), and the REST surface
|
||||
bypasses gRPC interceptors entirely even once fixed. Useful code to reference, not to
|
||||
copy the deployment of.
|
||||
- The `core` CLI already does human OIDC login (`core auth login` — device-code/OOB
|
||||
against Authentik) and caches a token at `~/.cache/talos/authentik_id_token`
|
||||
(`core/src/cmd/auth/mod.rs:336-338`) — a **different** token/cache than what §5 adds.
|
||||
|
||||
### Future System
|
||||
Implement in `internal/auth`, with two entry paths converging on one validated-JWT
|
||||
middleware — no separate downstream logic per path:
|
||||
1. **Bearer passthrough** — `Authorization: Bearer <jwt>` sourced from
|
||||
`~/.talos/.riotpiao-auth` (written by `mwinit`, §5), validated against Authentik's
|
||||
JWKS.
|
||||
2. **Service-account password grant** — Authentik service-account username/password
|
||||
exchanged for a JWT. Reuse the validation *logic* already written (but unwired) in
|
||||
`kmsvc-manage/internal/api/interceptors/auth.go`, not its deployment config.
|
||||
|
||||
Retry middleware (`internal/resilience/retry.go`, new) wraps every outbound upstream
|
||||
call from the `internal/serviceadapter` dispatcher: **blind retry on any 5xx**, bounded
|
||||
attempts with backoff, no idempotency-key gating — Temporal (and other backends) own
|
||||
dedup on their side. `ServiceAdapter.spec.retryable` (default `true`) is the only
|
||||
opt-out.
|
||||
|
||||
---
|
||||
|
||||
## 5. `CLI-MWNINIT.md` (separate doc)
|
||||
|
||||
### Existing System
|
||||
The Rust `core` CLI (`/Users/rockliang/workplace/core`) has `core auth login`
|
||||
(`src/cmd/auth/mod.rs`) for human OIDC device-code/OOB login, and a separate `core iam`
|
||||
subcommand tree (`src/cmd/iam/`) for app/group/scope management. There is no
|
||||
service-account login path and no `~/.talos/` token file — the existing cache is
|
||||
`~/.cache/talos/authentik_id_token`, and `~/.talos` today only holds Talos *node*
|
||||
config (`TALOSCONFIG`, per `core/src/config.rs:25-26`), unrelated to Authentik JWTs.
|
||||
|
||||
### Future System
|
||||
New `core mwinit` subcommand (new `src/cmd/mwinit/mod.rs`, same structure as
|
||||
`src/cmd/auth/mod.rs` — not a new binary), documented in a separate `CLI-MWNINIT.md`:
|
||||
- **Purpose** — authenticate as a dedicated Authentik service account (not a human
|
||||
device-code login) and cache the resulting JWT for use against `api.riotpiao.com`.
|
||||
- **Two paths**: `core mwinit login --username <svc-account> --password <...>`
|
||||
(Authentik service-account grant, same `AUTHENTIK_URL` default as `core auth login`,
|
||||
`core/src/cmd/auth/config.rs:18`), or `core mwinit login --token <existing-jwt>` to
|
||||
just re-cache an already-issued token.
|
||||
- **Storage**: `~/.talos/.riotpiao-auth` — a new, separate path from `core auth
|
||||
login`'s `~/.cache/talos/authentik_id_token`, so the two tools never collide.
|
||||
- **Usage**: `curl -H "Authorization: Bearer $(cat ~/.talos/.riotpiao-auth)" -H 'X-Service: ...' https://api.riotpiao.com/ ...`.
|
||||
- **Lifecycle**: `core mwinit status` / `core mwinit clear` (mirroring `core auth
|
||||
status`/`clear`); gateway behavior on expired token is 401 + `WWW-Authenticate`,
|
||||
RFC 9457 problem+json.
|
||||
|
||||
---
|
||||
|
||||
## 6. `X-Service: memory` — poimen-memory
|
||||
|
||||
### Existing System
|
||||
`poimen-memory` (`~/workplace/Poimen/memory`) is real and **already deployed** —
|
||||
namespace `poimen`, Service `poimen-memory.poimen.svc.cluster.local:8080`, 2 pods
|
||||
`Running`. Its own `README.md` claims "design complete, no code yet, 0/37 tasks",
|
||||
which is badly stale: `memory-tasks/M3.5.*.md` (per that repo's
|
||||
`DESIGN.md`) show the HTTP API layer (M3.5 phase) is done — `GET /memory/projects`,
|
||||
`GET /memory/projects/{id}/status`,
|
||||
`GET /memory/query?query=...&level=L1,L2&project=...&limit=...`,
|
||||
`POST /memory/ingest`, `GET /memory/skills`, `GET /memory/skills/{name}` all exist and
|
||||
are tested. It is not currently reachable from `api.riotpiao.com` — the live ingress
|
||||
has only a `/` catch-all to `api-gateway`, no `/memory` path.
|
||||
|
||||
Three more endpoints are **specced in `DESIGN.md` but not confirmed built** —
|
||||
`GET /memory/projects/{id}/notes` (M3.5.6, listed in the task table but not in the
|
||||
"is done" set above), `POST /memory/context` (M3.7, tier-1/2/3 lookup — depends on
|
||||
M3.7.7/M3.7.8 signature+symptom work, not started), and `POST /memory/nodes/by-git`
|
||||
/`by-commit`/`by-author` (M3.5.9, explicitly optional in that repo's own task
|
||||
board). These get CR entries below so the schema is defined once, but the
|
||||
gateway-side task for each is **blocked on upstream**, not just on our own code —
|
||||
see §7's status column.
|
||||
|
||||
It also currently does its own auth: an `apikey:` header check, exact-match against a
|
||||
stored key, implemented in the app itself
|
||||
(`tasks/M3.5.1-http-server.md:36-39`) — independent of Kong, which its own
|
||||
architecture diagram assumed sat in front but never actually required for this check
|
||||
to work. There is **no NetworkPolicy** on it today (`k8s/` has none) — the apikey
|
||||
check is the *only* thing standing between it and any pod in the cluster.
|
||||
|
||||
### Future System
|
||||
Per explicit decision: drop poimen-memory's own `apikey` check entirely and gate it
|
||||
through Authentik instead, with separate read/write capabilities — using the
|
||||
per-method `auth.capability` override introduced in §1.
|
||||
|
||||
```yaml
|
||||
apiVersion: gateway.riotpiao.com/v1
|
||||
kind: ServiceAdapter
|
||||
metadata:
|
||||
name: memory
|
||||
namespace: api
|
||||
spec:
|
||||
serviceName: memory # matches X-Service: memory
|
||||
upstream:
|
||||
url: http://poimen-memory.poimen.svc.cluster.local:8080
|
||||
timeoutSeconds: 15
|
||||
auth:
|
||||
required: true
|
||||
capability: memory:read # default for every resource below
|
||||
retryable: true
|
||||
resources:
|
||||
- name: query # X-Resource: query — "specific documents belonging to a project"
|
||||
methods:
|
||||
- verb: GET
|
||||
upstreamPath: /memory/query
|
||||
- name: skill # X-Resource: skill or skill/{name}
|
||||
methods:
|
||||
- verb: GET
|
||||
upstreamPath: /memory/skills
|
||||
- verb: GET
|
||||
upstreamPath: /memory/skills/{name}
|
||||
- name: project # X-Resource: project or project/{id}/status
|
||||
methods:
|
||||
- verb: GET
|
||||
upstreamPath: /memory/projects
|
||||
- verb: GET
|
||||
upstreamPath: /memory/projects/{id}/status
|
||||
- name: ingest # X-Resource: ingest — write path
|
||||
methods:
|
||||
- verb: POST
|
||||
upstreamPath: /memory/ingest
|
||||
auth:
|
||||
capability: memory:write # overrides the memory:read default
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X GET https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
-H 'X-Service: memory' \
|
||||
-H 'X-Resource: query' \
|
||||
-G --data-urlencode 'query=why did requests over 10KB fail' \
|
||||
--data-urlencode 'project=poimen' --data-urlencode 'level=L1,L2'
|
||||
```
|
||||
|
||||
**Required companion changes, in this order — not optional, not cosmetic:**
|
||||
1. Add a `NetworkPolicy` in namespace `poimen` restricting ingress on `poimen-memory`
|
||||
to the `api` namespace's gateway pod only (mirrors the MinIO-protection rationale
|
||||
from the very first version of this doc). This must land **before** step 2, or
|
||||
there's a window where the service has no auth of any kind.
|
||||
2. Remove the `apikey:` middleware from `poimen-memory` itself
|
||||
(`tasks/M3.5.1-http-server.md`'s auth hook) — separate change in the
|
||||
`~/workplace/Poimen/memory` repo, out of scope for `homelab-frontend` but a hard
|
||||
prerequisite for this adapter being safe to expose.
|
||||
3. Provision `memory:read` / `memory:write` as real Authentik scopes/capabilities via
|
||||
the `X-Service: iam` adapter (§3) or `core mwinit`-issued tokens, so callers can
|
||||
actually be granted one without the other.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
- `go build ./...` and existing tests (`internal/config/*_test.go`, plus new tests for
|
||||
the rewritten `internal/temporal` / new `internal/serviceadapter`, `internal/iam`,
|
||||
`internal/resilience`) pass.
|
||||
- `curl` smoke tests per adapter (workflow/s3/sqs/iam/memory) against a dev gateway,
|
||||
covering both the legacy `/workflow` body-action path (still mounted) and the new
|
||||
root + verb+`X-Resource` path.
|
||||
- `internal/serviceadapter/validate_test.go` (new): missing required field → 400 with
|
||||
`reason: missing`; wrong type (e.g. `sql` sent as a number) → 400 with
|
||||
`type_mismatch`; unlisted key under `strict: true` → 400 with `unknown_field`;
|
||||
unlisted key under default (`strict: false`) → passes. Postgres `query` resource's
|
||||
`requestSchema` (§1) is the fixture. Confirm a response-side mismatch does **not**
|
||||
block the response and instead increments
|
||||
`serviceadapter_response_schema_mismatch{service,resource}`.
|
||||
- Apply the `memory` `ServiceAdapter` CR only after the `poimen` NetworkPolicy lands
|
||||
and `poimen-memory`'s own `apikey` middleware is removed (§6's ordering). Confirm a
|
||||
`memory:read`-scoped token can `GET X-Resource: query`/`skill`/`project` and a
|
||||
`memory:write`-scoped token can `POST X-Resource: ingest`, and that a `memory:read`
|
||||
token gets 403 on `ingest`.
|
||||
- Commit the CRD + `Role`/`RoleBinding` to `k8s/`, push to `main`; confirm the existing
|
||||
`api-gw` ArgoCD Application auto-syncs them with no `homelab` changes and no new
|
||||
Application. Apply the example `postgres` CR; confirm the running gateway pod's
|
||||
informer picks it up and serves `X-Service: postgres` within the watch/resync
|
||||
latency — no gateway restart, no manual `kubectl` step.
|
||||
- `core mwinit login` end-to-end against dev Authentik; confirm `~/.talos/.riotpiao-auth`
|
||||
is written, a root-level `X-Service`-header call using it succeeds, and
|
||||
`~/.cache/talos/authentik_id_token` is untouched.
|
||||
@@ -1,122 +0,0 @@
|
||||
# Integration Tests
|
||||
|
||||
Integration tests call the real deployed gateway to verify X-Service routing works end-to-end.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Local Test (requires running gateway)
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start the gateway
|
||||
CONFIG_PATH=k8s/configmap.yaml go run ./cmd/gateway
|
||||
|
||||
# Terminal 2: Run tests
|
||||
./scripts/test-integration.sh
|
||||
```
|
||||
|
||||
### Cluster Test (production gateway)
|
||||
|
||||
```bash
|
||||
GATEWAY_URL=https://api.riotpiao.com ./scripts/test-integration.sh
|
||||
```
|
||||
|
||||
### Canary Deployment (scale to 1, test, scale back)
|
||||
|
||||
```bash
|
||||
./scripts/test-canary.sh
|
||||
|
||||
# Or with custom settings:
|
||||
NAMESPACE=api DEPLOYMENT=api-gateway REPLICAS=3 ./scripts/test-canary.sh
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Create `.dev.test.local` (gitignored) with:
|
||||
|
||||
```bash
|
||||
GATEWAY_URL=https://api.riotpiao.com
|
||||
TEST_JWT_TOKEN=eyJ... # Real JWT from Authentik
|
||||
SKIP_AUTH_TESTS=false
|
||||
TEST_TIMEOUT=30
|
||||
```
|
||||
|
||||
Or set env vars directly:
|
||||
|
||||
```bash
|
||||
export GATEWAY_URL=https://api.riotpiao.com
|
||||
export TEST_JWT_TOKEN=eyJ...
|
||||
export SKIP_AUTH_TESTS=false
|
||||
go test -tags integration -v ./internal/serviceadapter
|
||||
```
|
||||
|
||||
## Test Matrix
|
||||
|
||||
| Test | Type | Expected | Notes |
|
||||
|------|------|----------|-------|
|
||||
| Health check | GET /healthz | 200 OK | Always works |
|
||||
| SQS list-queues | GET X-Service: sqs | 200 or 502 | 502 if service unreachable |
|
||||
| S3 list-objects | GET X-Service: s3 | 200 or 502 | 502 if service unreachable |
|
||||
| Memory query | POST X-Service: memory | 200 or 502 | 502 if service unreachable |
|
||||
| Service not found | GET X-Service: nonexistent | 404 | Routing error |
|
||||
| Resource not found | GET X-Service: sqs X-Resource: invalid | 404 | Resource error |
|
||||
| IAM with JWT | GET X-Service: iam + Bearer token | 200 or 502 | Requires valid JWT |
|
||||
| Missing X-Service | GET (no header) | 404 | Routed to default handler |
|
||||
|
||||
## Canary Deployment Flow
|
||||
|
||||
```
|
||||
Current: 3/3 replicas running
|
||||
↓
|
||||
scale → 1/3 replicas
|
||||
↓
|
||||
wait for pod ready
|
||||
↓
|
||||
run integration tests
|
||||
├─ PASS → scale → 3/3 replicas ✅
|
||||
└─ FAIL → keep 1/3 for debugging ❌
|
||||
```
|
||||
|
||||
## Running in CI
|
||||
|
||||
Add to `.gitea/workflows/ci.yaml`:
|
||||
|
||||
```yaml
|
||||
- name: Integration Tests
|
||||
run: |
|
||||
GATEWAY_URL=https://api.riotpiao.com \
|
||||
SKIP_AUTH_TESTS=true \
|
||||
TEST_TIMEOUT=30 \
|
||||
go test -tags integration -v ./internal/serviceadapter
|
||||
```
|
||||
|
||||
## Debugging Failed Tests
|
||||
|
||||
If a test fails:
|
||||
|
||||
1. **Check pod logs:**
|
||||
```bash
|
||||
kubectl -n api logs -l app=api-gateway --tail=50
|
||||
```
|
||||
|
||||
2. **Check service availability:**
|
||||
```bash
|
||||
kubectl get svc -A | grep -E "sqs|minio|authentik|poimen"
|
||||
```
|
||||
|
||||
3. **Test service directly:**
|
||||
```bash
|
||||
kubectl -n sqs port-forward svc/management-service 9090:9090
|
||||
curl http://localhost:9090/sqs/queues
|
||||
```
|
||||
|
||||
4. **Check ConfigMap:**
|
||||
```bash
|
||||
kubectl get configmap api-gateway-config -n api -o yaml | grep -A50 "adapters:"
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Auth tests are skipped by default (`SKIP_AUTH_TESTS=true`)
|
||||
- To test with JWT, set `TEST_JWT_TOKEN` and `SKIP_AUTH_TESTS=false`
|
||||
- Services in different namespaces may not be reachable from the gateway (NetworkPolicy)
|
||||
- Canary tests expect `/healthz` endpoint to be available
|
||||
@@ -1,486 +0,0 @@
|
||||
# JWT vs Random Tokens - Which is Better?
|
||||
|
||||
Analyzing token types for Authentik + SOPS workflow.
|
||||
|
||||
---
|
||||
|
||||
## Quick Comparison
|
||||
|
||||
| Aspect | JWT | Random Token |
|
||||
|--------|-----|--------------|
|
||||
| **Stateless** | ✅ Yes | ❌ Requires lookup |
|
||||
| **Self-contained** | ✅ Yes (claims inside) | ❌ Opaque |
|
||||
| **Can verify locally** | ✅ Yes (signature) | ❌ Must call Authentik |
|
||||
| **Size** | ⚠️ Larger (~500 bytes) | ✅ Smaller (~32 bytes) |
|
||||
| **Immediate revocation** | ❌ Hard (token already signed) | ✅ Easy |
|
||||
| **Contains user info** | ✅ Yes | ❌ No |
|
||||
| **Standard OAuth2** | ⚠️ Optional (Bearer tokens) | ✅ Standard |
|
||||
| **Good for audit** | ✅ User/groups embedded | ⚠️ Need to log lookup |
|
||||
| **Git-friendly** | ✅ Sign commits | ✅ Sign commits |
|
||||
|
||||
---
|
||||
|
||||
## Use Case: Authentik + SOPS + Git Hooks
|
||||
|
||||
### Scenario 1: Developer Pushes Secret Update
|
||||
|
||||
```
|
||||
Developer:
|
||||
$ export SOPS_TOKEN=$token
|
||||
$ sops secrets/default/db-creds.enc.yaml
|
||||
$ git push
|
||||
|
||||
Git server hook runs:
|
||||
├─ Receive token from commit metadata
|
||||
├─ Need to validate token
|
||||
└─ Two options:
|
||||
```
|
||||
|
||||
**With JWT Token:**
|
||||
```
|
||||
Git hook:
|
||||
1. Extract JWT from commit
|
||||
2. Verify signature locally (no API call)
|
||||
3. Read claims: {sub: "[email protected]", groups: ["k8s:secret-admin"]}
|
||||
4. Check: is user in k8s:secret-admin group?
|
||||
5. Allow/reject commit
|
||||
|
||||
Benefits:
|
||||
✅ No need to call Authentik API
|
||||
✅ Token self-validates
|
||||
✅ Can check groups locally
|
||||
✅ Instant verification
|
||||
✅ Works offline
|
||||
```
|
||||
|
||||
**With Random Token:**
|
||||
```
|
||||
Git hook:
|
||||
1. Extract token from commit
|
||||
2. Call Authentik API: GET /application/o/introspect/
|
||||
└─ "Is this token valid?"
|
||||
3. If valid, check groups: GET /api/v3/users/{id}/groups/
|
||||
4. If admin, allow commit
|
||||
|
||||
Problems:
|
||||
❌ Need API call on every git push
|
||||
❌ Slow (network latency)
|
||||
❌ Requires network connectivity
|
||||
❌ If Authentik is down, can't push
|
||||
❌ Rate limiting risk (many API calls)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JWT Structure (What's Inside)
|
||||
|
||||
```json
|
||||
// Decoded JWT payload
|
||||
{
|
||||
"sub": "[email protected]",
|
||||
"email": "[email protected]",
|
||||
"name": "John Doe",
|
||||
"groups": [
|
||||
"k8s:secret-admin",
|
||||
"k8s:namespace:default:editor"
|
||||
],
|
||||
"iat": 1705336200,
|
||||
"exp": 1705422600, // Expires in 24 hours
|
||||
"iss": "https://authentik.riotpiao.com",
|
||||
"aud": "secrets-management"
|
||||
}
|
||||
```
|
||||
|
||||
**In git commit:**
|
||||
```
|
||||
commit abc1234...
|
||||
Author: [email protected] <[email protected]>
|
||||
Date: Tue Jan 16 14:30:00 2025 +0000
|
||||
|
||||
chore(secret): update db-creds
|
||||
|
||||
X-SOPS-Token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
|
||||
```
|
||||
|
||||
**Git hook can:**
|
||||
```
|
||||
1. Extract token from commit
|
||||
2. Decode (no signature needed yet)
|
||||
3. Read claims: sub, groups
|
||||
4. Verify signature with Authentik's public key
|
||||
5. Check groups locally
|
||||
6. Decision: accept or reject
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JWT Benefits for This Use Case
|
||||
|
||||
### 1. Local Token Verification (No API Calls)
|
||||
|
||||
```typescript
|
||||
// Git hook code (serverless, fast)
|
||||
function validateJWT(token: string) {
|
||||
const decoded = jwt.verify(token, PUBLIC_KEY);
|
||||
// ✅ Done instantly
|
||||
// ✅ No network call
|
||||
// ✅ Can work offline
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Embedded Claims (Audit Trail)
|
||||
|
||||
```
|
||||
JWT contains:
|
||||
├─ sub: who made the change
|
||||
├─ groups: what permissions they had
|
||||
├─ iat: when token was issued
|
||||
└─ exp: when it expires
|
||||
|
||||
Git commit metadata automatically includes:
|
||||
└─ Who, what groups, when
|
||||
|
||||
Complete audit trail without additional logging.
|
||||
```
|
||||
|
||||
### 3. RBAC at Git Level
|
||||
|
||||
```typescript
|
||||
// Git hook can enforce RBAC locally
|
||||
function canPushSecrets(token: JWT) {
|
||||
const groups = token.groups;
|
||||
|
||||
// Only k8s:secret-admin can push to secrets/
|
||||
if (!groups.includes("k8s:secret-admin")) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
```
|
||||
|
||||
### 4. No Dependency on Authentik Being Up
|
||||
|
||||
```
|
||||
If Authentik is temporarily down:
|
||||
✅ Developers can still push (JWT validates locally)
|
||||
✅ Works offline
|
||||
❌ With random tokens: push fails
|
||||
```
|
||||
|
||||
### 5. Temporal Can Verify Tokens Locally
|
||||
|
||||
```
|
||||
Temporal workflow (token rotation):
|
||||
└─ Reads old token from K8s
|
||||
└─ Decodes JWT: check exp field
|
||||
└─ "Is this token about to expire?"
|
||||
└─ Generate new token
|
||||
|
||||
No API call needed to validate old token.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Random Token Benefits
|
||||
|
||||
### 1. Standard OAuth2 Practice
|
||||
|
||||
```
|
||||
Random tokens are what most OAuth2 providers use:
|
||||
├─ Opaque (attacker doesn't know content)
|
||||
├─ Standard bearer tokens
|
||||
└─ What Authentik probably generates by default
|
||||
```
|
||||
|
||||
### 2. Immediate Revocation
|
||||
|
||||
```
|
||||
If token is compromised:
|
||||
└─ Delete it from Authentik
|
||||
└─ All git pushes instantly fail
|
||||
|
||||
With JWT:
|
||||
└─ Token is still valid until exp time
|
||||
└─ Attacker has until expiry
|
||||
```
|
||||
|
||||
### 3. Smaller Size
|
||||
|
||||
```
|
||||
JWT: ~500 bytes (base64 encoded)
|
||||
Random: ~32 bytes (hex)
|
||||
|
||||
Difference:
|
||||
✅ Slightly smaller in git metadata
|
||||
❌ But negligible for practical purposes
|
||||
```
|
||||
|
||||
### 4. Simpler Conceptually
|
||||
|
||||
```
|
||||
Developers understand random tokens:
|
||||
└─ "Here's a secret token, use it"
|
||||
|
||||
Developers don't understand JWT:
|
||||
└─ "What's this long base64 string?"
|
||||
└─ "Why do I need to decode it?"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## My Recommendation: JWT
|
||||
|
||||
Here's why for YOUR use case:
|
||||
|
||||
### 1. Git Hook Verification
|
||||
```
|
||||
With JWT:
|
||||
Git hook validates token signature locally
|
||||
└─ No Authentik API call
|
||||
└─ Fast
|
||||
└─ Works offline
|
||||
└─ Scales infinitely
|
||||
|
||||
With random token:
|
||||
Git hook calls Authentik API
|
||||
└─ Network latency
|
||||
└─ Authentik bottleneck
|
||||
└─ Fails if Authentik down
|
||||
└─ Rate limiting risk
|
||||
```
|
||||
|
||||
### 2. Embedded RBAC
|
||||
```
|
||||
With JWT:
|
||||
Git hook reads groups from token
|
||||
└─ Only k8s:secret-admin can push to secrets/
|
||||
└─ Enforced locally
|
||||
└─ No additional database lookups
|
||||
|
||||
With random token:
|
||||
Git hook calls Authentik API twice
|
||||
└─ Introspect token
|
||||
└─ Fetch groups
|
||||
└─ Slower
|
||||
```
|
||||
|
||||
### 3. Audit Trail
|
||||
```
|
||||
With JWT:
|
||||
Git commit includes token claims
|
||||
└─ User, groups, timestamp all in commit metadata
|
||||
└─ Complete audit trail automatically
|
||||
|
||||
With random token:
|
||||
Git commit has only token
|
||||
└─ Need to log token → user mapping separately
|
||||
└─ Additional logging overhead
|
||||
```
|
||||
|
||||
### 4. Temporal Token Rotation
|
||||
```
|
||||
With JWT:
|
||||
Temporal reads exp field
|
||||
└─ "Token expires in 2 hours"
|
||||
└─ No API call needed
|
||||
|
||||
With random token:
|
||||
Temporal calls Authentik API
|
||||
└─ "Is this token still valid?"
|
||||
└─ Network call needed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## How to Get JWT from Authentik
|
||||
|
||||
### Option 1: OpenID Connect (Standard)
|
||||
```
|
||||
Authentik already supports OIDC:
|
||||
|
||||
Developer requests token:
|
||||
$ curl -X POST https://authentik.../application/o/token/ \
|
||||
-d "grant_type=password&username=user&password=pwd&scope=openid profile groups"
|
||||
|
||||
Response:
|
||||
{
|
||||
"access_token": "eyJhbGc...", ← JWT
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 86400,
|
||||
"id_token": "eyJhbGc..."
|
||||
}
|
||||
```
|
||||
|
||||
### Option 2: OAuth2 with JWT
|
||||
```
|
||||
Authentik OAuth2 provider settings:
|
||||
├─ Enable: "Use claims-based access tokens"
|
||||
├─ Token claims: username, email, groups, preferred_username
|
||||
└─ Signature algorithm: RS256
|
||||
|
||||
Result: Access token is now JWT instead of random.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation: SOPS + JWT
|
||||
|
||||
```bash
|
||||
# Developer gets JWT token from Authentik
|
||||
$ token=$(oidc-client get-token)
|
||||
$ echo $token # base64 encoded JWT, not random string
|
||||
|
||||
# SOPS commits with JWT
|
||||
$ export SOPS_TOKEN=$token
|
||||
$ sops secrets/default/db-creds.enc.yaml
|
||||
$ git add . && git commit -m "update" && git push
|
||||
|
||||
# Git hook receives commit
|
||||
$ git hook:
|
||||
# 1. Extract SOPS_TOKEN from commit metadata
|
||||
# 2. Decode JWT (no API call)
|
||||
# 3. Verify signature with Authentik public key
|
||||
# 4. Read claims: groups
|
||||
# 5. Check: user in k8s:secret-admin?
|
||||
# 6. Accept or reject push
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison in Your Architecture
|
||||
|
||||
### Random Token Path
|
||||
```
|
||||
Developer → Authentik (get random token)
|
||||
→ SOPS (encrypt + commit)
|
||||
→ Git push
|
||||
→ Git hook (calls Authentik API to validate)
|
||||
→ ArgoCD
|
||||
→ K8s
|
||||
|
||||
Network calls: 2 (Authentik for token + git hook validation)
|
||||
```
|
||||
|
||||
### JWT Token Path
|
||||
```
|
||||
Developer → Authentik (get JWT token)
|
||||
→ SOPS (encrypt + commit)
|
||||
→ Git push
|
||||
→ Git hook (validates JWT locally, no API call)
|
||||
→ ArgoCD
|
||||
→ K8s
|
||||
|
||||
Network calls: 1 (only Authentik for token issuance)
|
||||
```
|
||||
|
||||
**Winner: JWT (fewer API calls)**
|
||||
|
||||
---
|
||||
|
||||
## Concerns with JWT
|
||||
|
||||
### Q: Can JWT be revoked immediately?
|
||||
|
||||
**A:** Not easily. Options:
|
||||
|
||||
```
|
||||
1. Token expires (set exp field to short time, e.g., 1 hour)
|
||||
└─ Temporal rotates daily anyway
|
||||
|
||||
2. Blacklist approach (store revoked JWTs)
|
||||
└─ Git hook checks blacklist before validating
|
||||
└─ But defeats purpose of stateless tokens
|
||||
|
||||
3. Accept that JWT lives until expiry
|
||||
└─ In your case: Temporal rotates daily
|
||||
└─ So max time a compromised token is valid: 24 hours
|
||||
└─ Acceptable for homelab
|
||||
```
|
||||
|
||||
### Q: Are JWTs vulnerable?
|
||||
|
||||
**A:** No, if configured right:
|
||||
|
||||
```
|
||||
✅ Signed with RS256 (Authentik public key)
|
||||
✅ Signature verified on every use
|
||||
✅ Expiry checked (can't use expired token)
|
||||
✅ Can't be forged (no private key to sign with)
|
||||
```
|
||||
|
||||
### Q: Does Authentik support JWT?
|
||||
|
||||
**A:** Yes, fully:
|
||||
|
||||
```
|
||||
Authentik has:
|
||||
✅ OIDC (returns JWT id_token + access_token)
|
||||
✅ OAuth2 with claims (can return JWT)
|
||||
✅ Configuration for JWT signing algorithm
|
||||
✅ Public key endpoint for verification
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Final Recommendation
|
||||
|
||||
### Use JWT if:
|
||||
```
|
||||
✅ You want fast git hook validation (no API calls)
|
||||
✅ You want embedded RBAC (groups in token)
|
||||
✅ You want complete audit trail (claims in metadata)
|
||||
✅ You want resilience (works if Authentik down)
|
||||
✅ You want to scale (no API bottleneck)
|
||||
```
|
||||
|
||||
### Use Random Token if:
|
||||
```
|
||||
✅ Immediate revocation is critical
|
||||
✅ You prefer standard OAuth2 approach
|
||||
✅ Developers shouldn't see token contents
|
||||
✅ Simplicity is more important than features
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## My Advice
|
||||
|
||||
**Go with JWT.**
|
||||
|
||||
Here's why:
|
||||
1. Authentik supports it natively
|
||||
2. Git hook verification is instant (no API calls)
|
||||
3. Embedded claims give you free audit trail
|
||||
4. Works offline (resilient)
|
||||
5. Perfect for homelab scale
|
||||
|
||||
**Setup:**
|
||||
1. Configure Authentik OIDC provider
|
||||
2. Enable "Use claims-based access tokens" option
|
||||
3. Add groups to token claims
|
||||
4. Developers get JWT instead of random string
|
||||
5. Git hook validates JWT locally (no API call)
|
||||
|
||||
**Code example for git hook:**
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# .git/hooks/update (server-side)
|
||||
|
||||
token=$(git log -1 --pretty=%B | grep "X-SOPS-Token" | cut -d' ' -f2)
|
||||
|
||||
# Verify JWT signature
|
||||
jwt verify $token --key /path/to/authentik/public.key
|
||||
|
||||
# Decode and check groups
|
||||
groups=$(jwt decode $token | jq .groups)
|
||||
if [[ ! "$groups" =~ "k8s:secret-admin" ]]; then
|
||||
echo "Access denied: not in k8s:secret-admin group"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Allow push
|
||||
exit 0
|
||||
```
|
||||
|
||||
That's it. Simple, elegant, no redundancy.
|
||||
@@ -1,499 +0,0 @@
|
||||
# LLM Tool Calls Testing Guide
|
||||
|
||||
This guide shows how to test the gateway with LLM tool calling (function calling) across different APIs.
|
||||
|
||||
## What's Tested
|
||||
|
||||
The gateway fully supports tool calling for:
|
||||
- **OpenAI API** (`/v1/chat/completions`) - OpenAI, DeepSeek, etc.
|
||||
- **Anthropic API** (`/llm/v1/messages`) - Claude models
|
||||
- **Custom APIs** - Any LLM that supports tool definitions and responses
|
||||
|
||||
### Test Coverage
|
||||
|
||||
```
|
||||
✅ OpenAI-style tool calling
|
||||
✅ Streaming tool calls (SSE with tool_use blocks)
|
||||
✅ Multi-turn conversations with tool results
|
||||
✅ Parallel tool calls (multiple tools at once)
|
||||
✅ Anthropic tool_use format
|
||||
✅ Complex nested tool arguments
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start: Run Tests Locally
|
||||
|
||||
```bash
|
||||
cd /Users/rockliang/workplace/homelab-frontend
|
||||
|
||||
# Run all tool call tests
|
||||
go test ./internal/proxy/... -run "Tool" -v
|
||||
|
||||
# Or run with race detector (recommended)
|
||||
go test -race ./internal/proxy/... -run "Tool" -v
|
||||
|
||||
# Expected output: 6 tests, all passing
|
||||
```
|
||||
|
||||
## Test Scenarios
|
||||
|
||||
### 1. OpenAI-Style Tool Calling
|
||||
|
||||
**What it tests:**
|
||||
- Request with tool definitions reaches upstream unmodified
|
||||
- Upstream can return tool_calls in response
|
||||
- Response with tool_calls passes through to client
|
||||
|
||||
**Test code:**
|
||||
```go
|
||||
// Request
|
||||
{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "What's the weather?"}],
|
||||
"tools": [{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get weather for a location",
|
||||
"parameters": {...}
|
||||
}
|
||||
}]
|
||||
}
|
||||
|
||||
// Response (from upstream)
|
||||
{
|
||||
"choices": [{
|
||||
"message": {
|
||||
"tool_calls": [{
|
||||
"id": "call_abc123",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"San Francisco\"}"
|
||||
}
|
||||
}]
|
||||
}
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestToolCallOpenAIStyle -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Streaming Tool Calls
|
||||
|
||||
**What it tests:**
|
||||
- Tool calls can be streamed (SSE format)
|
||||
- Multiple chunks arrive with tool_call deltas
|
||||
- Stream completes with `[DONE]` sentinel
|
||||
|
||||
**Test code:**
|
||||
```
|
||||
Chunk 1: {"delta": {"role": "assistant"}, ...}
|
||||
Chunk 2: {"delta": {"tool_calls": [{"id": "call_123", "function": {...}}]}, ...}
|
||||
Chunk 3: {"delta": {}, "finish_reason": "tool_calls"}
|
||||
Chunk 4: [DONE]
|
||||
```
|
||||
|
||||
**Run:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestToolCallStreaming -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Multi-Turn Conversation with Tool Results
|
||||
|
||||
**What it tests:**
|
||||
- Client can send previous assistant's tool_calls back
|
||||
- Tool result can be sent as a "tool" role message
|
||||
- Assistant responds with final answer using tool result
|
||||
|
||||
**Flow:**
|
||||
```
|
||||
Turn 1: User asks → LLM decides to call tool
|
||||
Turn 2: Client sends tool result → LLM generates final answer
|
||||
```
|
||||
|
||||
**Test code:**
|
||||
```go
|
||||
// Turn 1 Request
|
||||
{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "What's the weather?"}],
|
||||
"tools": [...]
|
||||
}
|
||||
|
||||
// Turn 1 Response (tool_calls)
|
||||
{
|
||||
"message": {
|
||||
"tool_calls": [{
|
||||
"id": "call_abc",
|
||||
"function": {"name": "get_weather", "arguments": "..."}
|
||||
}]
|
||||
}
|
||||
}
|
||||
|
||||
// Turn 2 Request (with tool result)
|
||||
{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What's the weather?"},
|
||||
{"role": "assistant", "tool_calls": [...]},
|
||||
{"role": "tool", "content": "{\"temperature\": 22, \"condition\": \"cloudy\"}"}
|
||||
],
|
||||
"tools": [...]
|
||||
}
|
||||
|
||||
// Turn 2 Response (final answer)
|
||||
{
|
||||
"message": {
|
||||
"content": "The weather in San Francisco is 22°C and cloudy."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestToolCallMultiTurn -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. Parallel Tool Calls
|
||||
|
||||
**What it tests:**
|
||||
- LLM can request multiple tools in one response
|
||||
- Gateway preserves all tool_calls
|
||||
- Client can execute them in parallel
|
||||
|
||||
**Test code:**
|
||||
```go
|
||||
// Single response with 3 tool_calls
|
||||
{
|
||||
"message": {
|
||||
"tool_calls": [
|
||||
{"id": "call_1", "function": {"name": "get_weather", "arguments": "{\"location\":\"New York\"}"}},
|
||||
{"id": "call_2", "function": {"name": "get_weather", "arguments": "{\"location\":\"London\"}"}},
|
||||
{"id": "call_3", "function": {"name": "get_weather", "arguments": "{\"location\":\"Tokyo\"}"}}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestParallelToolCalls -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Anthropic Tool Use Format
|
||||
|
||||
**What it tests:**
|
||||
- Different tool format: Anthropic uses `tool_use` blocks instead of `tool_calls`
|
||||
- Gateway handles both formats transparently
|
||||
- Tools are sent with `tools` parameter
|
||||
|
||||
**OpenAI format:**
|
||||
```json
|
||||
{"tool_calls": [{"type": "function", "function": {...}}]}
|
||||
```
|
||||
|
||||
**Anthropic format:**
|
||||
```json
|
||||
{"content": [
|
||||
{"type": "text", "text": "..."},
|
||||
{"type": "tool_use", "id": "...", "name": "...", "input": {...}}
|
||||
]}
|
||||
```
|
||||
|
||||
**Test code:**
|
||||
```go
|
||||
// Request
|
||||
{
|
||||
"model": "claude",
|
||||
"messages": [{"role": "user", "content": "What's the weather?"}],
|
||||
"tools": [{
|
||||
"name": "get_weather",
|
||||
"description": "Get weather",
|
||||
"input_schema": {...}
|
||||
}]
|
||||
}
|
||||
|
||||
// Response (Anthropic format)
|
||||
{
|
||||
"content": [
|
||||
{"type": "text", "text": "I'll check the weather..."},
|
||||
{"type": "tool_use", "id": "toolu_123", "name": "get_weather", "input": {...}}
|
||||
],
|
||||
"stop_reason": "tool_use"
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestAnthropicToolUse -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Complex Nested Tool Arguments
|
||||
|
||||
**What it tests:**
|
||||
- Tool arguments can be complex JSON structures
|
||||
- Nested objects, arrays, and deeply nested data preserved
|
||||
- No argument modification or parsing
|
||||
|
||||
**Test code:**
|
||||
```json
|
||||
{
|
||||
"function": {
|
||||
"name": "create_event",
|
||||
"arguments": {
|
||||
"title": "Team Meeting",
|
||||
"time": "2025-08-20T14:00:00Z",
|
||||
"attendees": [
|
||||
{"name": "Alice", "email": "[email protected]"},
|
||||
{"name": "Bob", "email": "[email protected]"}
|
||||
],
|
||||
"location": {
|
||||
"address": "123 Main St",
|
||||
"city": "San Francisco",
|
||||
"country": "USA"
|
||||
},
|
||||
"tags": ["important", "recurring"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Run:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestComplexToolArguments -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running Against Real LLMs
|
||||
|
||||
### With Local Stubs (Current)
|
||||
|
||||
Tests use mock HTTP servers, so they run instantly:
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run "Tool" -v
|
||||
# All 6 tests complete in ~220ms
|
||||
```
|
||||
|
||||
### With Real Upstreams (Future)
|
||||
|
||||
Once you have real LLM services running, update the config:
|
||||
|
||||
```yaml
|
||||
# k8s/configmap.yaml
|
||||
models:
|
||||
- name: "reasoning"
|
||||
address: "reasoning-predictor.llm-serving:80" # Real upstream
|
||||
- name: "claude"
|
||||
address: "claude-api.anthropic.com:443" # Real Anthropic
|
||||
```
|
||||
|
||||
Then use the gateway normally:
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start gateway
|
||||
export CONFIG_PATH=config.yaml
|
||||
go run ./cmd/gateway
|
||||
|
||||
# Terminal 2: Test with real LLM
|
||||
curl -X POST http://localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "What color is the sky?"}],
|
||||
"tools": [{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "search",
|
||||
"description": "Search the internet",
|
||||
"parameters": {"type": "object", "properties": {}}
|
||||
}
|
||||
}]
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Gateway Behavior with Tool Calls
|
||||
|
||||
### Request Path
|
||||
|
||||
```
|
||||
Client Request
|
||||
↓
|
||||
Body-based dispatch (find model)
|
||||
↓
|
||||
Look up upstream address
|
||||
↓
|
||||
Forward request unmodified (including tools)
|
||||
↓
|
||||
Upstream LLM processes tools
|
||||
```
|
||||
|
||||
### Response Path
|
||||
|
||||
```
|
||||
Upstream Response (with tool_calls or tool_use)
|
||||
↓
|
||||
Stream unbuffered if streaming
|
||||
↓
|
||||
Return to client exactly as received
|
||||
```
|
||||
|
||||
### Key Properties
|
||||
|
||||
1. **No Rewriting**: Tool definitions and responses pass through unmodified
|
||||
2. **Format Agnostic**: Both OpenAI `tool_calls` and Anthropic `tool_use` work
|
||||
3. **Streaming Safe**: Tool calls stream incrementally without buffering
|
||||
4. **Nested Structures**: Complex JSON arguments fully preserved
|
||||
|
||||
---
|
||||
|
||||
## Common Tool Call Patterns
|
||||
|
||||
### Pattern 1: Sequential Tool Use
|
||||
```
|
||||
Client → LLM (please use search tool)
|
||||
← LLM (tool_calls: [search(...)])
|
||||
Client → (execute search, send results)
|
||||
Client → LLM (here are search results)
|
||||
← LLM (final answer)
|
||||
```
|
||||
|
||||
### Pattern 2: Parallel Tool Calls
|
||||
```
|
||||
Client → LLM (check weather in 3 cities)
|
||||
← LLM (tool_calls: [get_weather(NY), get_weather(LA), get_weather(SF)])
|
||||
Client → (execute all 3 in parallel)
|
||||
Client → LLM (here are all results)
|
||||
← LLM (summary)
|
||||
```
|
||||
|
||||
### Pattern 3: Tool Result Formatting
|
||||
```
|
||||
Client receives tool_calls with:
|
||||
- id: unique identifier
|
||||
- function.name: tool name
|
||||
- function.arguments: JSON string (always a string, not parsed object)
|
||||
|
||||
Client sends back:
|
||||
- role: "tool"
|
||||
- content: result JSON string
|
||||
- tool_call_id: matches the original call id
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [x] OpenAI-style tool definitions forward to upstream
|
||||
- [x] Tool calls in response reach client unmodified
|
||||
- [x] Streaming tool calls arrive incrementally
|
||||
- [x] Multi-turn conversations preserve tool context
|
||||
- [x] Parallel tool calls all included in response
|
||||
- [x] Anthropic tool_use format works
|
||||
- [x] Complex nested arguments preserved
|
||||
|
||||
Run all:
|
||||
```bash
|
||||
go test ./internal/proxy/... -run "Tool" -v --race
|
||||
```
|
||||
|
||||
Expected: 6/6 passing, race detector clean
|
||||
|
||||
---
|
||||
|
||||
## Integration with Other Phases
|
||||
|
||||
### Phase 2.9: Anthropic Dialect
|
||||
Currently, Anthropic tool calls work through the generic route handler. Phase 2.9 will add a dedicated `/llm/v1/messages` endpoint with full Anthropic-specific handling.
|
||||
|
||||
### Phase 2.13: Error Handling
|
||||
Tool call errors (unknown tool, parsing errors) will have proper error responses in both OpenAI and Anthropic formats.
|
||||
|
||||
### Phase 3: Authentication
|
||||
Tool calls work with all authentication methods (bearer tokens, API keys) - no special handling needed since tools are just part of the message payload.
|
||||
|
||||
### Phase 4: Rate Limiting
|
||||
Tool calling counts the same as regular chat requests. Rate limits apply per conversation, not per tool call.
|
||||
|
||||
---
|
||||
|
||||
## Debugging Tool Calls
|
||||
|
||||
### Check if tool definitions reach upstream:
|
||||
|
||||
```bash
|
||||
# Enable request logging
|
||||
go test ./internal/proxy/... -run TestToolCallOpenAIStyle -v 2>&1 | grep -A5 "tool"
|
||||
```
|
||||
|
||||
### Verify tool response format:
|
||||
|
||||
```bash
|
||||
# Extract and pretty-print response
|
||||
curl -X POST http://localhost:8080/v1/chat/completions ... | jq '.choices[0].message.tool_calls'
|
||||
```
|
||||
|
||||
### Test streaming tool calls:
|
||||
|
||||
```bash
|
||||
curl -N http://localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{..., "stream": true, "tools": [...]}'
|
||||
# Should see incremental chunks with tool_use deltas
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Do I need to modify the gateway code to support tool calls?**
|
||||
A: No. Tool calls are just JSON in the request/response body. The gateway forwards them unchanged.
|
||||
|
||||
**Q: What if the LLM doesn't support tools?**
|
||||
A: The tool definitions are simply ignored. The gateway doesn't validate or enforce tool support.
|
||||
|
||||
**Q: Can I mix OpenAI and Anthropic tool formats?**
|
||||
A: Not in the same request. OpenAI clients expect `tool_calls`, Anthropic clients expect `tool_use` blocks. The upstream API determines the format.
|
||||
|
||||
**Q: How are tool arguments limited?**
|
||||
A: By the per-route `maxBodySize` config. Complex nested arguments count toward that limit.
|
||||
|
||||
**Q: Can tool calls be streamed?**
|
||||
A: Yes! SSE streaming fully supports tool calls. They arrive in delta chunks like text tokens.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Run tests locally:**
|
||||
```bash
|
||||
go test ./internal/proxy/... -run "Tool" -v
|
||||
```
|
||||
|
||||
2. **Deploy to cluster:**
|
||||
See [CLUSTER_REPO_SETUP.md](./CLUSTER_REPO_SETUP.md)
|
||||
|
||||
3. **Test against real LLMs:**
|
||||
Update config with real upstream addresses, restart gateway
|
||||
|
||||
4. **Phase 2.9:** Implement Anthropic dialect handler for `/llm/v1/messages`
|
||||
|
||||
5. **Phase 4:** Add tool call budgeting and rate limits
|
||||
-367
@@ -1,367 +0,0 @@
|
||||
# homelab-frontend — Requirements
|
||||
|
||||
The contract for the Go API gateway that replaces Kong OSS on `*.riotpiao.com`.
|
||||
|
||||
Companion documents:
|
||||
- [docs/adr/ADR-0001-retire-kong-for-go-gateway.md](docs/adr/ADR-0001-retire-kong-for-go-gateway.md) — why Kong is being retired
|
||||
- [docs/MIGRATION-kong.md](docs/MIGRATION-kong.md) — exact inventory of what Kong does today and the cutover order
|
||||
- [tasks/INDEX.md](tasks/INDEX.md) — the task board
|
||||
|
||||
All cluster facts below were verified live against context `admin@homelab-cluster`
|
||||
on 2026-08-19. Re-verify before relying on any number.
|
||||
|
||||
---
|
||||
|
||||
## 0. Invariants
|
||||
|
||||
These hold for every surface. A change that breaks one of these is a design change,
|
||||
not an implementation detail.
|
||||
|
||||
- **G1** — ingress-nginx owns TLS and the edge. The gateway never terminates TLS.
|
||||
- **G2** — The gateway holds no Kubernetes credentials. It proxies to services that
|
||||
do. Cluster-read permissions stay in atlas, out of the public edge process.
|
||||
- **G3** — Public surfaces use standard protocol shapes. If an OpenAI SDK cannot
|
||||
call it unmodified, the design is wrong.
|
||||
- **G4** — Streaming is unbuffered end to end, and a client disconnect cancels the
|
||||
upstream request rather than orphaning it.
|
||||
- **G5** — Authentication is Bearer-token, validated against Authentik via JWKS
|
||||
fetched at runtime. No pinned public keys, no rotation runbook.
|
||||
- **G6** — Every route's timeout, body cap and concurrency limit is explicit in
|
||||
configuration. No silent defaults.
|
||||
- **G7** — All deployment flows through git and Argo. No `kubectl apply`, no
|
||||
`helm upgrade`, no local `terraform apply`.
|
||||
|
||||
---
|
||||
|
||||
## 1. Runtime and configuration
|
||||
|
||||
### 1.1 Process
|
||||
|
||||
Single static Go binary. Reads configuration at startup, serves HTTP, exits cleanly
|
||||
on SIGTERM after draining in-flight requests.
|
||||
|
||||
Must run with **no cluster, no kubeconfig and no credentials** so that behaviour can
|
||||
be verified in a closed loop before touching live traffic. Upstreams are
|
||||
configuration; pointing them at local stubs is the entire mechanism. This is a hard
|
||||
requirement, not a convenience — see §7.
|
||||
|
||||
### 1.2 Configuration
|
||||
|
||||
Route and upstream configuration is declarative and loaded at startup. It must
|
||||
express, per upstream: address, path rewrite, connect/read/write timeouts, maximum
|
||||
body size, and whether the route requires authentication.
|
||||
|
||||
Configuration errors fail startup loudly. A gateway that starts with a silently
|
||||
dropped route is worse than one that refuses to start.
|
||||
|
||||
**Configuration lives in git**, mounted as a ConfigMap and synced by Argo. Not a
|
||||
CRD. A CRD would require the gateway to watch the API server, which needs RBAC and
|
||||
contradicts G2 — and CRD-driven routing is precisely the indirection being retired
|
||||
with Kong, where the routing table was split across six `KongPlugin` CRs, seven
|
||||
Ingresses and a Helm values file.
|
||||
|
||||
A CRD earns its keep when someone other than the repo owner must register routes.
|
||||
That is not true here. If it becomes true, the additive answer is a controller that
|
||||
renders this same ConfigMap — the gateway stays credential-free either way.
|
||||
|
||||
### 1.3 Health
|
||||
|
||||
- `GET /healthz` — liveness, no upstream checks, always cheap.
|
||||
- `GET /readyz` — readiness; may fail while configuration is invalid or JWKS has
|
||||
never been successfully fetched.
|
||||
|
||||
Neither requires authentication.
|
||||
|
||||
---
|
||||
|
||||
## 2. Proxy core
|
||||
|
||||
### 2.1 Reverse proxying
|
||||
|
||||
Standard reverse proxy to configured upstreams. Connection reuse across requests.
|
||||
Hop-by-hop headers stripped correctly. `X-Forwarded-*` set from the nginx-supplied
|
||||
values, not fabricated.
|
||||
|
||||
### 2.2 Streaming
|
||||
|
||||
SSE and chunked responses pass through without buffering. Tokens must reach the
|
||||
client as the upstream emits them, not on completion.
|
||||
|
||||
WebSocket upgrade must work — `agent-pod/console` depends on it.
|
||||
|
||||
### 2.3 Disconnect propagation
|
||||
|
||||
When a client disconnects, the upstream request is cancelled immediately. This is
|
||||
load-bearing: an orphaned generation holds a vLLM sequence slot, and there are only
|
||||
eight in the cluster.
|
||||
|
||||
### 2.4 Timeouts
|
||||
|
||||
Per-route, explicit. Current Kong values, which are deliberate and must be preserved
|
||||
unless changed knowingly:
|
||||
|
||||
| Route class | connect | read | write |
|
||||
|---|---|---|---|
|
||||
| chat | 10s | 1h | 1h |
|
||||
| embeddings, rerank | 10s | 10m | 10m |
|
||||
|
||||
The 1-hour read timeout exists because a 32B model on a Volta GPU routinely exceeds
|
||||
60s. Any shorter application-level cap must be enforced *by the gateway's own
|
||||
logic*, not by shortening the proxy timeout — otherwise long legitimate generations
|
||||
truncate mid-stream.
|
||||
|
||||
---
|
||||
|
||||
## 3. LLM surfaces — `api.riotpiao.com`
|
||||
|
||||
Two protocol dialects, permanently. Both translate into one dialect-neutral canonical
|
||||
request, and both pass through **one shared slot controller** before reaching a
|
||||
predictor.
|
||||
|
||||
| Prefix | Dialect | Primary client |
|
||||
|---|---|---|
|
||||
| `/v1/*` | OpenAI-compatible | pi, generic OpenAI SDKs |
|
||||
| `/llm/*` | Anthropic Messages | the riotpiao frontend (first-party only) |
|
||||
|
||||
```
|
||||
/v1/* (OpenAI) /llm/* (Anthropic)
|
||||
| |
|
||||
+-----------+------------+
|
||||
v
|
||||
canonical request dialect-neutral
|
||||
v
|
||||
slot controller keyed by UPSTREAM, not by route
|
||||
v
|
||||
reasoning-predictor / ornith-predictor
|
||||
```
|
||||
|
||||
**The slot controller is keyed by upstream and shared across dialects.** Per-dialect
|
||||
semaphores are wrong: the 8 sequence slots are physical, so two independent gates
|
||||
would each believe they were within budget while together exceeding it. Requests from
|
||||
both surfaces contend for the same slots and the same queue, in arrival order.
|
||||
|
||||
Dispatch, budgets, logging and metrics all operate on the canonical request. Adding a
|
||||
third dialect later must not require touching the controller.
|
||||
|
||||
### 3.1 Body-based model dispatch
|
||||
|
||||
`POST /v1/chat/completions` selects its upstream from the request body's `model`
|
||||
field. This is the single most important requirement in this document: it is the
|
||||
capability Kong OSS lacked, and the reason the gateway exists.
|
||||
|
||||
Unknown or missing `model` is a client error with a useful message listing valid
|
||||
values — not a 500, and not a silent fallback to a default model.
|
||||
|
||||
### 3.2 Upstream map
|
||||
|
||||
Verified live. `served-model-name` values are what clients send.
|
||||
|
||||
| `model` in body | Upstream Service | Engine |
|
||||
|---|---|---|
|
||||
| `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B |
|
||||
| `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama |
|
||||
| `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama, same pods |
|
||||
| `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI |
|
||||
| `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI |
|
||||
|
||||
`reasoning` runs 2 replicas × `--max-num-seqs=4` = **8 concurrent sequence slots
|
||||
total**, `--max-model-len=16384`, `--reasoning-parser=deepseek_r1`,
|
||||
`--enable-auto-tool-choice --tool-call-parser=hermes`.
|
||||
|
||||
Note `ornith:35b` and `qwen2.5:3b-instruct` share pods; both stay resident via
|
||||
`OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=-1`, so dispatching between them
|
||||
does not trigger a model swap.
|
||||
|
||||
### 3.3 Path rewriting
|
||||
|
||||
Upstreams expect canonical paths. `/v1/chat/completions` and `/v1/embeddings` pass
|
||||
through unchanged. Rerank is the exception: TEI serves `/rerank`, not `/v1/rerank`,
|
||||
so that route rewrites.
|
||||
|
||||
### 3.4 Legacy path aliases
|
||||
|
||||
`/v1/{reasoning,ornith,qwen}/chat/completions` must keep working during cutover —
|
||||
pi is a live caller. They behave exactly as the canonical endpoint with `model`
|
||||
forced to the corresponding value, overriding whatever the body says.
|
||||
|
||||
These are temporary. They exist to make the cutover reversible, and are removed once
|
||||
callers have migrated.
|
||||
|
||||
### 3.5 `GET /v1/models`
|
||||
|
||||
Derived from the configured upstream map, never hardcoded. Kong served a static
|
||||
list, and its own manifest flags that the list can drift from what the engines
|
||||
actually serve. The gateway's list must be incapable of disagreeing with what
|
||||
routing will accept.
|
||||
|
||||
OpenAI list shape: `{"object":"list","data":[{"id","object":"model","owned_by","created"}]}`.
|
||||
|
||||
### 3.6 Behaviour to preserve
|
||||
|
||||
Verified against the live endpoint:
|
||||
|
||||
- The upstream returns `reasoning_content` separately from `content` for the
|
||||
`reasoning` model. Pass both through untouched.
|
||||
- Tool calling works with explicit `tool_choice`, and is unreliable with
|
||||
`tool_choice: auto` on the R1-distill model. The gateway does not compensate for
|
||||
this — it is a model property, not a gateway concern. Do not add retries or
|
||||
rewriting to work around it.
|
||||
|
||||
---
|
||||
|
||||
## 4. Authentication — Authentik
|
||||
|
||||
### 4.1 Current state
|
||||
|
||||
**The model API is unauthenticated today.** Confirmed live: `/v1/reasoning/chat/completions`
|
||||
answers with no credentials.
|
||||
|
||||
Kong's `key-auth` was retired because it accepts a raw `apikey:` header but rejects
|
||||
`Authorization: Bearer`, which hard-blocks every OpenAI-compatible client. See
|
||||
`~/workplace/homelab/k8s/apps/api/model-auth.yaml`.
|
||||
|
||||
### 4.2 Requirement
|
||||
|
||||
Bearer tokens in `Authorization`, validated against Authentik
|
||||
(`https://authentik.riotpiao.com`) by fetching and caching JWKS at runtime.
|
||||
|
||||
Key rotation must be handled by refetching JWKS, not by pinned PEMs. The
|
||||
pinned-`rsa_public_key` approach in `AUTH-PLAN.md` and its rotation runbook exist
|
||||
only to route around a Kong OSS limitation and must not be carried forward.
|
||||
|
||||
Service accounts obtain tokens via `client_credentials` against Authentik's token
|
||||
endpoint.
|
||||
|
||||
### 4.3 Rollout
|
||||
|
||||
Auth ships behind a flag, defaulting off, and is enabled deliberately.
|
||||
|
||||
Enabling it breaks every current caller until they hold a token — pi included, whose
|
||||
`models.json` currently sends a `customHeaders: {apikey: ...}` block that will need
|
||||
replacing with a Bearer token.
|
||||
|
||||
### 4.4 Authorization
|
||||
|
||||
Beyond authentication, a token must be checked for the right to invoke the
|
||||
capability it is calling. A token minted for queue access should not invoke a GPU.
|
||||
|
||||
---
|
||||
|
||||
## 5. Rate limiting and budgets
|
||||
|
||||
No `rate-limiting` plugin exists anywhere in the cluster today — this is net new
|
||||
work, not a migration. Verified: six Kong plugins exist, none is `rate-limiting`.
|
||||
|
||||
Requirements, in priority order:
|
||||
|
||||
1. **GPU slot protection.** `reasoning` has 8 total sequence slots. Concurrent
|
||||
in-flight requests to it must be capped below that, leaving operator headroom.
|
||||
Excess requests queue up to a bounded depth, then are rejected with a retryable
|
||||
status.
|
||||
2. **Per-caller budgets.** Identified callers get a request budget over a window.
|
||||
3. **Body size caps**, per route.
|
||||
|
||||
Rejections use RFC 9457 `application/problem+json` and set `Retry-After` where a
|
||||
retry time is knowable.
|
||||
|
||||
---
|
||||
|
||||
## 6. Observability
|
||||
|
||||
Kong's cluster-wide `prometheus` plugin is being retired. The gateway must expose at
|
||||
least equivalent signal or observability regresses at cutover: request rate,
|
||||
latency, status codes, bandwidth, and upstream health, labelled by route and
|
||||
upstream.
|
||||
|
||||
Gateway-specific signals that Kong could not provide, and which are the reason for
|
||||
several requirements above: in-flight requests per upstream, queue depth, GPU slot
|
||||
occupancy, and rejections by reason.
|
||||
|
||||
Structured logging. Every rejected request is logged with the reason. No secrets, no
|
||||
tokens, no request bodies in logs.
|
||||
|
||||
---
|
||||
|
||||
## 7. Local development and verification
|
||||
|
||||
An agent must be able to close a change/verify loop with no cluster, no kubeconfig
|
||||
and no credentials. This is a hard requirement because it determines whether work can
|
||||
proceed unattended.
|
||||
|
||||
Concretely: it must be possible to start the gateway locally, point it at stub
|
||||
upstreams, issue requests, and assert on the responses — including streaming
|
||||
responses and client disconnects.
|
||||
|
||||
Verification of any API-shaped task means asserting on the **actual HTTP response**:
|
||||
status, headers, and body. "It compiles" and "it starts" are not verification.
|
||||
|
||||
Parity with Kong is verified by comparing gateway and Kong responses for the same
|
||||
request, for every route in the migration inventory, before cutover.
|
||||
|
||||
---
|
||||
|
||||
## 8. Deployment
|
||||
|
||||
Container: distroless or scratch, `runAsNonRoot`, read-only root filesystem, all
|
||||
capabilities dropped, `seccompProfile: RuntimeDefault`, no shell.
|
||||
|
||||
Image tags are commit SHAs, never `:latest` — Argo's `selfHeal` cannot roll out a
|
||||
mutable tag reliably.
|
||||
|
||||
NetworkPolicy: egress only to the upstreams it proxies plus Authentik; ingress from
|
||||
`ingress-nginx` only.
|
||||
|
||||
Deployed as an Argo Application in the `homelab-root` GitOps repo. Verified live:
|
||||
zero Argo Applications anywhere in the cluster source from any Forgejo URL, so
|
||||
`github.com/Riotpiaole/riotpiao.homelab.com` is authoritative.
|
||||
|
||||
---
|
||||
|
||||
## 9. Capability surface — path-based
|
||||
|
||||
Every capability is a path prefix on the single host `api.riotpiao.com`. One DNS
|
||||
record, one Cloudflare tunnel hostname, one nginx Ingress, one Service.
|
||||
|
||||
| Prefix | Backs onto | Status |
|
||||
|---|---|---|
|
||||
| `/v1/*` | `llm-serving` predictors | v1 — **reserved**, see below |
|
||||
| `/sqs/*` | kmsvc management-service, Kafka/Strimzi (`sqs` ns) | future |
|
||||
| `/workflow/*` | Temporal (`temporal` ns) | future |
|
||||
| `/cluster/*` | atlas, separate repo `riotpiao-backend` | future |
|
||||
| `/db/*` | CloudNativePG, MinIO, monitoring reads | future |
|
||||
|
||||
**`/v1/*` is reserved for the OpenAI-compatible surface and nothing else.** G3 pins
|
||||
it: an SDK expects `/v1/chat/completions` at the base URL, so that prefix can never
|
||||
be repurposed or nested. Every other capability gets its own prefix that cannot
|
||||
collide with a current or future OpenAI path.
|
||||
|
||||
Subdomains are deliberately *not* used. Paths keep hostname configuration to a
|
||||
single entry — and hostname configuration is the demonstrated failure mode here, as
|
||||
the unresolved apex 403 shows. Promoting a prefix to its own subdomain later is an
|
||||
additive host rule that can run alongside the path; the reverse is not, because
|
||||
clients hardcode hostnames.
|
||||
|
||||
Notes carried from the cluster:
|
||||
|
||||
- Temporal namespace registration is automatic via queue-operator, never manual.
|
||||
- `management-service` already exposes gRPC at `kmsvc.riotpiao.com`; the `/sqs`
|
||||
prefix is a new surface, not a replacement for it.
|
||||
- atlas keeps its own informers and RBAC. The gateway proxies to it and holds no
|
||||
cluster credentials of its own (G2).
|
||||
- `/db/*` read surfaces need particular care — see G2 before designing them.
|
||||
|
||||
---
|
||||
|
||||
## 10. Known cluster facts worth not rediscovering
|
||||
|
||||
- `kmsvc-redis-master.sqs:6379` has **no authentication** — `ALLOW_EMPTY_PASSWORD=yes`,
|
||||
TLS off. Any workload with network reach has full unauthenticated read/write. A
|
||||
NetworkPolicy is the only control.
|
||||
- `reasoning-predictor` listens on port **80**, not 8080.
|
||||
- `prometheus-operated.monitoring` is **headless** (ClusterIP None) — egress policies
|
||||
need pod selectors, not ClusterIPs.
|
||||
- `agent-pod/console` is publicly routed, unauthenticated, accepts free-form prompts
|
||||
into a shell-capable container, and serves a WebSocket. Putting it behind gateway
|
||||
auth is a security fix, not merely a port.
|
||||
- Eight `*.example.com` hosts exist on istio-class Ingresses in `llm-serving`. KServe
|
||||
defaults, not public, out of scope — do not mistake them for gateway routes.
|
||||
@@ -1,176 +0,0 @@
|
||||
# Temporal gRPC Integration - Migration Status
|
||||
|
||||
## Overview
|
||||
|
||||
Temporal REST ↔ gRPC bridge is being implemented. Client sends HTTP JSON → gateway translates to gRPC → Temporal server responds.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1: Core Workflow Operations ✅ WIRED
|
||||
|
||||
- **START_WORKFLOW** ✅ gRPC: `StartWorkflowExecution`
|
||||
- **DESCRIBE_WORKFLOW** ✅ gRPC: `DescribeWorkflowExecution`
|
||||
- **LIST_WORKFLOWS** ⏳ TODO (requires pagination logic)
|
||||
- **GET_WORKFLOW_HISTORY** ⏳ TODO
|
||||
- **SIGNAL_WORKFLOW** ⏳ TODO
|
||||
- **QUERY_WORKFLOW** ⏳ TODO
|
||||
- **CANCEL_WORKFLOW** ⏳ TODO
|
||||
- **TERMINATE_WORKFLOW** ⏳ TODO
|
||||
- **RESET_WORKFLOW** ⏳ TODO
|
||||
- **UPDATE_WORKFLOW** ⏳ TODO
|
||||
|
||||
### Phase 2: Activity Operations ⏳ NOT IMPLEMENTED
|
||||
|
||||
- HEARTBEAT_ACTIVITY
|
||||
- COMPLETE_ACTIVITY
|
||||
- FAIL_ACTIVITY
|
||||
|
||||
**Note:** Activity operations require different error handling (task tokens, etc.). See operations_grpc.go for reference.
|
||||
|
||||
### Phase 3: OperatorService Operations ⏳ NOT IMPLEMENTED
|
||||
|
||||
Requires separate gRPC stub. Currently:
|
||||
- LIST_NAMESPACES → 501 NOT_IMPLEMENTED
|
||||
- DESCRIBE_NAMESPACE → 501 NOT_IMPLEMENTED
|
||||
- CREATE_NAMESPACE → 501 NOT_IMPLEMENTED
|
||||
- UPDATE_NAMESPACE → 501 NOT_IMPLEMENTED
|
||||
- DELETE_NAMESPACE → 501 NOT_IMPLEMENTED
|
||||
- LIST_SEARCH_ATTRIBUTES → 501 NOT_IMPLEMENTED
|
||||
- ADD_SEARCH_ATTRIBUTES → 501 NOT_IMPLEMENTED
|
||||
- LIST_TASK_QUEUES → 501 NOT_IMPLEMENTED
|
||||
- GET_CLUSTER_INFO → 501 NOT_IMPLEMENTED
|
||||
- LIST_CLUSTER_MEMBERS → 501 NOT_IMPLEMENTED
|
||||
- GET_SYSTEM_INFO → 501 NOT_IMPLEMENTED
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
HTTP Request (JSON)
|
||||
↓
|
||||
Handler.startWorkflow()
|
||||
↓
|
||||
Converts to protobuf (workflowservice.StartWorkflowExecutionRequest)
|
||||
↓
|
||||
gRPCClient.GetWorkflowServiceStub().StartWorkflowExecution(ctx, req)
|
||||
↓
|
||||
Temporal Server (port 7233)
|
||||
↓
|
||||
gRPC Response
|
||||
↓
|
||||
Convert to JSON response map
|
||||
↓
|
||||
HTTP 200 JSON
|
||||
```
|
||||
|
||||
## Code References
|
||||
|
||||
- **handler.go**: HTTP ↔ gRPC translation layer
|
||||
- `NewHandler()`: Creates gRPC connection via `NewGRPCClient()`
|
||||
- `startWorkflow()`, `describeWorkflow()`: gRPC-wired operations
|
||||
- Others: stubs or NOT_IMPLEMENTED
|
||||
|
||||
- **grpc_client.go**: Low-level gRPC connection management
|
||||
- `NewGRPCClient()`: Dials Temporal at port 7233
|
||||
- `GetWorkflowServiceStub()`: Returns `workflowservice.WorkflowServiceClient`
|
||||
- `GetOperatorServiceStub()`: Returns `operatorservice.OperatorServiceClient`
|
||||
|
||||
- **operations_grpc.go**: Example gRPC implementations (reference for wiring)
|
||||
- Shows payload marshaling patterns
|
||||
- Shows error handling (gRPC status codes → HTTP 4xx/5xx)
|
||||
|
||||
## Next Steps (Phase 2)
|
||||
|
||||
1. Wire remaining WorkflowService operations (LIST, GET_HISTORY, SIGNAL, QUERY, etc.)
|
||||
- All use same pattern: build protobuf request → call stub → map response to JSON
|
||||
- Reference operations_grpc.go for exact patterns
|
||||
|
||||
2. Add OperatorService support (namespaces, cluster, search attrs)
|
||||
- Create separate stub: `operatorServiceClient := NewGRPCClient().GetOperatorServiceStub()`
|
||||
- Add methods to handler for each operation
|
||||
|
||||
3. Add Activity operations (heartbeat, complete, fail)
|
||||
- Requires task token handling
|
||||
- See operations_grpc_test.go for test patterns
|
||||
|
||||
## Build Status
|
||||
|
||||
```
|
||||
go build ./cmd/gateway ✅ SUCCESS
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
To test gRPC wiring locally:
|
||||
|
||||
```bash
|
||||
# Start Temporal locally (if not running)
|
||||
docker run -d -p 7233:7233 temporalio/auto-setup:latest
|
||||
|
||||
# Start gateway
|
||||
go run ./cmd/gateway
|
||||
|
||||
# Test (in another terminal)
|
||||
curl -X POST http://localhost:8080/workflow \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"action": "START_WORKFLOW",
|
||||
"namespace": "default",
|
||||
"payload": {
|
||||
"workflow_id": "test-1",
|
||||
"workflow_type": "MyWorkflow",
|
||||
"task_queue": "my-queue"
|
||||
}
|
||||
}'
|
||||
|
||||
# Should return
|
||||
{
|
||||
"success": true,
|
||||
"action": "START_WORKFLOW",
|
||||
"data": {
|
||||
"workflow_id": "test-1",
|
||||
"run_id": "abc123...",
|
||||
"start_time": "2026-08-27T..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Key Implementation Details
|
||||
|
||||
### Protobuf Field Names
|
||||
Temporal protobuf uses snake_case field names:
|
||||
- `WorkflowId` not `WorkflowID`
|
||||
- `RunId` not `RunID`
|
||||
- `WorkflowType` (message) not `WorkflowTypeString`
|
||||
- `TaskQueue` (message) not `TaskQueueName`
|
||||
|
||||
### Type Imports (from go.temporal.io/api)
|
||||
```go
|
||||
import (
|
||||
"go.temporal.io/api/common/v1" // WorkflowExecution, WorkflowType, Payloads
|
||||
"go.temporal.io/api/taskqueue/v1" // TaskQueue
|
||||
"go.temporal.io/api/workflowservice/v1" // All Workflow* stubs
|
||||
"go.temporal.io/api/operatorservice/v1" // Namespace/cluster stubs (not yet used)
|
||||
)
|
||||
```
|
||||
|
||||
### Payload Marshaling Pattern
|
||||
```go
|
||||
input := getMap(payload, "input")
|
||||
if len(input) > 0 {
|
||||
inputBytes, _ := json.Marshal(input)
|
||||
req.Input = &common.Payloads{
|
||||
Payloads: []*common.Payload{{Data: inputBytes}},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
- gRPC errors → map to HTTP status:
|
||||
- `codes.NotFound` → 404
|
||||
- `codes.InvalidArgument` → 400
|
||||
- `codes.Unavailable` → 503
|
||||
- others → 500
|
||||
|
||||
## Questions / Blockers
|
||||
|
||||
None currently. gRPC wiring is straightforward pattern-matching.
|
||||
-1210
File diff suppressed because it is too large
Load Diff
@@ -1,560 +0,0 @@
|
||||
# API Testing Guide
|
||||
|
||||
Quick reference for testing the homelab-frontend gateway API.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
# Set base URL
|
||||
export GATEWAY="https://api.riotpiao.com"
|
||||
|
||||
# Or for local testing
|
||||
export GATEWAY="http://localhost:8080"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Tests (Copy & Paste)
|
||||
|
||||
### 1. Health Checks ✅
|
||||
|
||||
```bash
|
||||
# Liveness
|
||||
curl $GATEWAY/healthz | jq .
|
||||
|
||||
# Readiness
|
||||
curl $GATEWAY/readyz | jq .
|
||||
```
|
||||
|
||||
**Expected**: Both return `{"status":"..."}` with HTTP 200
|
||||
|
||||
---
|
||||
|
||||
### 2. List Models ✅
|
||||
|
||||
```bash
|
||||
curl $GATEWAY/v1/models | jq '.data[] | .id'
|
||||
```
|
||||
|
||||
**Expected Output**:
|
||||
```
|
||||
"reasoning"
|
||||
"ornith:35b"
|
||||
"qwen2.5:3b-instruct"
|
||||
"nomic-ai/nomic-embed-text-v2-moe"
|
||||
"BAAI/bge-reranker-base"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Chat - Basic ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is 2+2?"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
**Expected**: Model responds with an answer
|
||||
|
||||
---
|
||||
|
||||
### 4. Chat - Ornith Model ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "ornith:35b",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
**Expected**: Routes to ornith model, returns response
|
||||
|
||||
---
|
||||
|
||||
### 5. Chat - Qwen Model ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "qwen2.5:3b-instruct",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hi"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
**Expected**: Routes to qwen model, returns response
|
||||
|
||||
---
|
||||
|
||||
### 6. Chat - Unknown Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "gpt-4-turbo",
|
||||
"messages": []
|
||||
}' | jq '.'
|
||||
```
|
||||
|
||||
**Expected**: HTTP 400 with problem+json:
|
||||
```json
|
||||
{
|
||||
"type": "https://api.example.com/problems/unknown-model",
|
||||
"title": "Unknown Model",
|
||||
"status": 400,
|
||||
"detail": "Model \"gpt-4-turbo\" is not available. See valid_models for available options.",
|
||||
"valid_models": ["reasoning", "ornith:35b", ...]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. Chat - Missing Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"messages": [{"role": "user", "content": "test"}]
|
||||
}' | jq '.'
|
||||
```
|
||||
|
||||
**Expected**: HTTP 400 with problem+json (missing model)
|
||||
|
||||
---
|
||||
|
||||
### 8. Chat - Streaming ✅
|
||||
|
||||
```bash
|
||||
curl -N -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "count to 3"}],
|
||||
"stream": true
|
||||
}' | head -20
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- Multiple `data: {...}` lines (SSE chunks)
|
||||
- Final `data: [DONE]`
|
||||
- Chunks arrive incrementally (observable with `-N` flag)
|
||||
|
||||
---
|
||||
|
||||
### 9. Chat - Tool Calling ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the weather in SF?"}
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get weather for a location",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": {"type": "string"}
|
||||
},
|
||||
"required": ["location"]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}' | jq '.choices[0].message.tool_calls'
|
||||
```
|
||||
|
||||
**Expected**: Array of tool calls (if model decides to call them), or null (if not)
|
||||
|
||||
---
|
||||
|
||||
### 10. Embeddings ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": "hello world"
|
||||
}' | jq '.data | length'
|
||||
```
|
||||
|
||||
**Expected**: `1` (one embedding vector)
|
||||
|
||||
---
|
||||
|
||||
### 11. Embeddings - Multiple ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": ["text 1", "text 2", "text 3"]
|
||||
}' | jq '.data | length'
|
||||
```
|
||||
|
||||
**Expected**: `3` (three embedding vectors)
|
||||
|
||||
---
|
||||
|
||||
### 12. Embeddings - Unknown Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "unknown-embed",
|
||||
"input": "test"
|
||||
}' | jq '.status'
|
||||
```
|
||||
|
||||
**Expected**: `400` (client error)
|
||||
|
||||
---
|
||||
|
||||
### 13. Rerank ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/rerank \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "BAAI/bge-reranker-base",
|
||||
"query": "machine learning",
|
||||
"texts": [
|
||||
"Machine learning is AI",
|
||||
"Python is a language",
|
||||
"Deep learning is ML"
|
||||
]
|
||||
}' | jq '.results'
|
||||
```
|
||||
|
||||
**Expected**: Array of ranked results with scores:
|
||||
```json
|
||||
[
|
||||
{"index": 0, "score": 0.95},
|
||||
{"index": 2, "score": 0.85},
|
||||
{"index": 1, "score": 0.15}
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 14. Rerank - Unknown Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/rerank \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "unknown-rerank",
|
||||
"query": "test",
|
||||
"texts": ["a"]
|
||||
}' | jq '.status'
|
||||
```
|
||||
|
||||
**Expected**: `400` (client error)
|
||||
|
||||
---
|
||||
|
||||
### 15. Invalid JSON (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d 'not json' | jq '.title'
|
||||
```
|
||||
|
||||
**Expected**: `"Invalid Request Body"` (HTTP 400)
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
Complete this checklist to verify all endpoints:
|
||||
|
||||
### Health Endpoints
|
||||
- [ ] GET /healthz → 200, `{"status":"alive"}`
|
||||
- [ ] GET /readyz → 200, `{"status":"ready"}`
|
||||
|
||||
### Model Discovery
|
||||
- [ ] GET /v1/models → 200, returns all 5 models
|
||||
- [ ] All advertised models can be called (none 400)
|
||||
|
||||
### Chat Completions
|
||||
- [ ] POST /v1/chat/completions (reasoning) → 200, response
|
||||
- [ ] POST /v1/chat/completions (ornith:35b) → 200, response
|
||||
- [ ] POST /v1/chat/completions (qwen2.5:3b-instruct) → 200, response
|
||||
- [ ] POST /v1/chat/completions (unknown model) → 400, problem+json
|
||||
- [ ] POST /v1/chat/completions (missing model) → 400, problem+json
|
||||
- [ ] POST /v1/chat/completions (invalid JSON) → 400, problem+json
|
||||
- [ ] POST /v1/chat/completions (streaming) → 200, SSE chunks
|
||||
- [ ] POST /v1/chat/completions (with tools) → 200, tool_calls present/absent
|
||||
|
||||
### Embeddings
|
||||
- [ ] POST /v1/embeddings (single input) → 200, embedding
|
||||
- [ ] POST /v1/embeddings (multiple inputs) → 200, embeddings array
|
||||
- [ ] POST /v1/embeddings (unknown model) → 400, problem+json
|
||||
|
||||
### Reranking
|
||||
- [ ] POST /v1/rerank → 200, ranked results
|
||||
- [ ] POST /v1/rerank (unknown model) → 400, problem+json
|
||||
- [ ] Verify path is rewritten to /rerank on upstream
|
||||
|
||||
### Error Handling
|
||||
- [ ] Unknown model lists valid_models
|
||||
- [ ] Error responses are problem+json
|
||||
- [ ] No 5xx for client errors (validation errors)
|
||||
- [ ] Upstream errors pass through
|
||||
|
||||
### Streaming
|
||||
- [ ] Chunks arrive incrementally
|
||||
- [ ] Final `[DONE]` sentinel present
|
||||
- [ ] Works for chat completions
|
||||
|
||||
### Tool Calling
|
||||
- [ ] Tool definitions forward to upstream
|
||||
- [ ] Tool calls in response
|
||||
- [ ] Multi-turn with tool results
|
||||
- [ ] Parallel tool calls
|
||||
- [ ] Complex nested arguments preserved
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### 404 Responses
|
||||
|
||||
**Symptom**: All endpoints return `"not found"`
|
||||
|
||||
**Cause**: ConfigMap with models/routes not deployed
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
kubectl -n api create configmap homelab-frontend-config \
|
||||
--from-file=config.yaml=k8s/configmap.yaml
|
||||
kubectl -n api rollout restart deployment/homelab-frontend
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 503 (Not Ready)
|
||||
|
||||
**Symptom**: `/readyz` returns 503
|
||||
|
||||
**Cause**: Configuration not loaded or JWKS fetch failed
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check logs
|
||||
kubectl -n api logs deployment/homelab-frontend
|
||||
|
||||
# Check config
|
||||
kubectl -n api get configmap homelab-frontend-config
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Connection Refused
|
||||
|
||||
**Symptom**: `Connection refused` or `Temporary failure in name resolution`
|
||||
|
||||
**Cause**:
|
||||
- Gateway not running
|
||||
- Wrong URL/hostname
|
||||
- Network issue
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Verify gateway is running
|
||||
kubectl -n api get pods -l app=homelab-frontend
|
||||
|
||||
# Check service
|
||||
kubectl -n api get svc homelab-frontend
|
||||
|
||||
# Verify ingress
|
||||
kubectl -n api get ingress api
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Upstream Connection Errors
|
||||
|
||||
**Symptom**: `502 Bad Gateway` or `connection refused to upstream`
|
||||
|
||||
**Cause**: Model upstream service not reachable
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check upstreams are running
|
||||
kubectl -n llm-serving get pods
|
||||
|
||||
# Verify addresses in ConfigMap
|
||||
kubectl -n api get configmap homelab-frontend-config -o yaml
|
||||
|
||||
# Test connectivity from gateway pod
|
||||
kubectl -n api exec deployment/homelab-frontend -- \
|
||||
curl -s reasoning-predictor.llm-serving:80/healthz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Streaming Doesn't Work
|
||||
|
||||
**Symptom**: Chunks arrive all at once (buffered) instead of incrementally
|
||||
|
||||
**Cause**: nginx buffering or client not using `-N` flag
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Use -N flag
|
||||
curl -N https://api.riotpiao.com/v1/chat/completions ...
|
||||
|
||||
# Verify nginx has buffering disabled
|
||||
# Should have: proxy-buffering: off in Ingress annotations
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Testing
|
||||
|
||||
### Load Test (Simple)
|
||||
|
||||
```bash
|
||||
# Send 10 requests in parallel
|
||||
for i in {1..10}; do
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"Hi"}]}' &
|
||||
done
|
||||
wait
|
||||
|
||||
echo "Completed 10 requests"
|
||||
```
|
||||
|
||||
### Concurrency Test
|
||||
|
||||
```bash
|
||||
# Use Apache Bench (if installed)
|
||||
ab -n 100 -c 10 \
|
||||
-p request.json \
|
||||
-T application/json \
|
||||
$GATEWAY/v1/chat/completions
|
||||
|
||||
# Create request.json:
|
||||
# {"model":"reasoning","messages":[{"role":"user","content":"test"}]}
|
||||
```
|
||||
|
||||
### Latency Test
|
||||
|
||||
```bash
|
||||
# Measure response time
|
||||
curl -w "\nTotal time: %{time_total}s\n" \
|
||||
-X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "What is AI?"}]
|
||||
}' > /dev/null
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Testing
|
||||
|
||||
### Test with Python
|
||||
|
||||
```bash
|
||||
pip install requests
|
||||
|
||||
cat > test_api.py << 'EOF'
|
||||
import requests
|
||||
import json
|
||||
|
||||
gateway = "https://api.riotpiao.com"
|
||||
|
||||
# Test health
|
||||
r = requests.get(f"{gateway}/healthz")
|
||||
assert r.status_code == 200
|
||||
print("✓ Health check passed")
|
||||
|
||||
# Test models
|
||||
r = requests.get(f"{gateway}/v1/models")
|
||||
assert r.status_code == 200
|
||||
models = [m['id'] for m in r.json()['data']]
|
||||
print(f"✓ Models: {models}")
|
||||
|
||||
# Test chat
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/chat/completions",
|
||||
json={"model": "reasoning", "messages": [{"role": "user", "content": "Hi"}]}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
print("✓ Chat works")
|
||||
|
||||
# Test unknown model error
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/chat/completions",
|
||||
json={"model": "gpt-4", "messages": []}
|
||||
)
|
||||
assert r.status_code == 400
|
||||
assert "unknown" in r.json()['detail'].lower()
|
||||
print("✓ Unknown model error correct")
|
||||
|
||||
# Test embeddings
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/embeddings",
|
||||
json={"model": "nomic-ai/nomic-embed-text-v2-moe", "input": "test"}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
print("✓ Embeddings work")
|
||||
|
||||
# Test rerank
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/rerank",
|
||||
json={"model": "BAAI/bge-reranker-base", "query": "test", "texts": ["a", "b"]}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
print("✓ Reranking works")
|
||||
|
||||
print("\n✅ All tests passed!")
|
||||
EOF
|
||||
|
||||
python test_api.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | Tests | Expected |
|
||||
|----------|-------|----------|
|
||||
| Health | 2 | ✅ Both 200 |
|
||||
| Models | 1 | ✅ 5 models listed |
|
||||
| Chat | 8 | ✅ 6 success + 2 error |
|
||||
| Embeddings | 3 | ✅ 2 success + 1 error |
|
||||
| Rerank | 2 | ✅ 1 success + 1 error |
|
||||
| Streaming | 1 | ✅ Incremental chunks |
|
||||
| Tools | 1 | ✅ Tool calls present |
|
||||
| **TOTAL** | **18+** | **✅ ALL PASS** |
|
||||
|
||||
Once all tests pass, the gateway is production-ready! 🚀
|
||||
-324
@@ -1,324 +0,0 @@
|
||||
# API — LLM surfaces
|
||||
|
||||
Two protocol dialects over the same models and the same slot controller.
|
||||
|
||||
| Prefix | Dialect | Endpoint | Client |
|
||||
|---|---|---|---|
|
||||
| `/v1` | OpenAI-compatible | `POST /v1/chat/completions` | pi, OpenAI SDKs |
|
||||
| `/llm` | Anthropic Messages | `POST /llm/v1/messages` | riotpiao frontend (first-party) |
|
||||
|
||||
Status marks below:
|
||||
**[LIVE]** verified against the running cluster on 2026-08-19.
|
||||
**[SPEC]** the contract this gateway must implement; not built yet.
|
||||
|
||||
---
|
||||
|
||||
## Models
|
||||
|
||||
| `model` value | Upstream | Engine | Context | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B | 16384 | emits `reasoning_content`; 8 sequence slots total |
|
||||
| `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama | 131072 | reliable tool calling |
|
||||
| `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama | 32768 | same pods as ornith |
|
||||
| `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI | — | embeddings only |
|
||||
| `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI | — | rerank only |
|
||||
|
||||
`reasoning` runs 2 replicas x `--max-num-seqs=4`. Those **8 slots are the scarcest resource in the cluster** and are shared across both dialects.
|
||||
|
||||
---
|
||||
|
||||
## Authentication [SPEC]
|
||||
|
||||
Ships behind a flag, default off. The model API is unauthenticated today.
|
||||
|
||||
```
|
||||
Authorization: Bearer <authentik-jwt>
|
||||
```
|
||||
|
||||
### Decided — Bearer on both surfaces
|
||||
|
||||
`Authorization: Bearer <jwt>` is the only accepted credential, on `/v1` and `/llm` alike. One auth path, consistent with G5, validated against Authentik via JWKS.
|
||||
|
||||
**Known divergence from Anthropic:** the real Anthropic API authenticates with `x-api-key` and requires `anthropic-version: 2023-06-01`. A stock Anthropic SDK pointed at `/llm` will send `x-api-key` and get a 401.
|
||||
|
||||
This is accepted, not overlooked. The `/llm` client is the first-party riotpiao frontend, which sends whatever we tell it to. If a real Anthropic SDK ever needs to reach this gateway, accepting `x-api-key` as a second credential source is an additive change — a small branch in one middleware, not a redesign.
|
||||
|
||||
`anthropic-version` is accepted and ignored if present, and never required.
|
||||
|
||||
The 401 for an `x-api-key`-only request must name the problem — say that Bearer is required — rather than returning a bare 401. The Kong retirement was caused by exactly this failure mode: a gateway that rejected the header clients actually send, without saying why.
|
||||
|
||||
---
|
||||
|
||||
## OpenAI dialect — `POST /v1/chat/completions`
|
||||
|
||||
### Request [SPEC]
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "Why is wave 4 empty?"}],
|
||||
"max_tokens": 2000,
|
||||
"temperature": 0.7,
|
||||
"stream": false
|
||||
}
|
||||
```
|
||||
|
||||
`model` is required and selects the upstream. The body is forwarded byte-identical — the gateway reads `model`, it does not rewrite it.
|
||||
|
||||
### Response, non-streaming [LIVE]
|
||||
|
||||
Captured verbatim from `reasoning` on 2026-08-19, abridged:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-f17bd2fe22e4276d24e9438e40e89cea",
|
||||
"object": "chat.completion",
|
||||
"created": 1787172340,
|
||||
"model": "reasoning",
|
||||
"choices": [{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "\n\nTo find the current weather in Toronto...",
|
||||
"reasoning_content": "Okay, so I need to figure out...",
|
||||
"tool_calls": []
|
||||
},
|
||||
"finish_reason": "length"
|
||||
}],
|
||||
"usage": {"prompt_tokens": 16, "completion_tokens": 300, "total_tokens": 316}
|
||||
}
|
||||
```
|
||||
|
||||
`reasoning_content` is a **sibling of** `content`, not nested in it. This is a vLLM extension produced by `--reasoning-parser=deepseek_r1`; it is not part of the OpenAI spec. Pass it through untouched.
|
||||
|
||||
### The two engines disagree on the field name [LIVE]
|
||||
|
||||
Verified 2026-08-19 by calling both:
|
||||
|
||||
| Upstream | Engine | Reasoning field |
|
||||
|---|---|---|
|
||||
| `reasoning-predictor` | vLLM | `reasoning_content` |
|
||||
| `ornith-predictor` | Ollama | `reasoning` |
|
||||
|
||||
Neither is in the OpenAI spec, so neither is wrong — they are two vendor extensions that
|
||||
happen to mean the same thing. The gateway must recognise **both** when mapping to the
|
||||
Anthropic `thinking` block, or `ornith:35b` responses will silently lose their reasoning
|
||||
on the `/llm` surface.
|
||||
|
||||
Do not normalise them on the `/v1` surface. That surface passes bodies through
|
||||
untouched, and a client asking for `ornith:35b` should get exactly what Ollama sent.
|
||||
Normalisation belongs in the canonical request model (task 2.9), which is the layer that
|
||||
exists to absorb precisely this kind of upstream difference.
|
||||
|
||||
### Response, streaming [SPEC]
|
||||
|
||||
```
|
||||
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"Okay"},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Wave"},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
||||
|
||||
data: [DONE]
|
||||
```
|
||||
|
||||
Data-only frames, no `event:` lines. Terminated by the literal `data: [DONE]`.
|
||||
|
||||
### Legacy aliases [LIVE, being retired]
|
||||
|
||||
`POST /v1/{reasoning,ornith,qwen}/chat/completions` force `model` to the corresponding value regardless of the body. They exist only because Kong could not dispatch on the body. Removed once callers migrate.
|
||||
|
||||
### `GET /v1/models` [SPEC]
|
||||
|
||||
```json
|
||||
{"object":"list","data":[{"id":"reasoning","object":"model","owned_by":"homelab","created":0}]}
|
||||
```
|
||||
|
||||
Derived from the registry, never hardcoded.
|
||||
|
||||
### Errors [SPEC]
|
||||
|
||||
RFC 9457 `application/problem+json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "https://riotpiao.com/errors/unknown-model",
|
||||
"title": "Unknown model",
|
||||
"status": 400,
|
||||
"detail": "\"gpt-4\" is not available",
|
||||
"validModels": ["reasoning", "ornith:35b", "qwen2.5:3b-instruct"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anthropic dialect — `POST /llm/v1/messages` [SPEC]
|
||||
|
||||
Path note: the Anthropic SDK appends `/v1/messages` to its base URL, so a base URL of `https://api.riotpiao.com/llm` produces exactly this path.
|
||||
|
||||
### Request
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "reasoning",
|
||||
"max_tokens": 2000,
|
||||
"system": "You are a cluster assistant.",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Why is wave 4 empty?"}
|
||||
],
|
||||
"stream": true
|
||||
}
|
||||
```
|
||||
|
||||
Differences from the OpenAI dialect that the translator must handle:
|
||||
|
||||
| Concern | OpenAI | Anthropic |
|
||||
|---|---|---|
|
||||
| system prompt | `messages[0].role = "system"` | top-level `system` field |
|
||||
| `max_tokens` | optional | **required** |
|
||||
| content | string | string *or* block array |
|
||||
| roles | system/user/assistant/tool | user/assistant only |
|
||||
| stop | `stop` | `stop_sequences` |
|
||||
|
||||
`max_tokens` being required is a real divergence — the gateway must either reject its absence with a clear error or apply a documented default. Pick one and state it; do not silently default.
|
||||
|
||||
### Response, non-streaming
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "msg_01ABC",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "reasoning",
|
||||
"content": [
|
||||
{"type": "thinking", "thinking": "Waves are sort keys, not a sequence..."},
|
||||
{"type": "text", "text": "Wave 4 is empty. Waves are sort keys..."}
|
||||
],
|
||||
"stop_reason": "end_turn",
|
||||
"stop_sequence": null,
|
||||
"usage": {"input_tokens": 16, "output_tokens": 300}
|
||||
}
|
||||
```
|
||||
|
||||
Field mapping from the upstream OpenAI response:
|
||||
|
||||
| Upstream | Anthropic |
|
||||
|---|---|
|
||||
| `choices[0].message.reasoning_content` | `content[]` block `{"type":"thinking","thinking":...}` |
|
||||
| `choices[0].message.content` | `content[]` block `{"type":"text","text":...}` |
|
||||
| `finish_reason: "stop"` | `stop_reason: "end_turn"` |
|
||||
| `finish_reason: "length"` | `stop_reason: "max_tokens"` |
|
||||
| `usage.prompt_tokens` | `usage.input_tokens` |
|
||||
| `usage.completion_tokens` | `usage.output_tokens` |
|
||||
|
||||
The thinking block precedes the text block.
|
||||
|
||||
### Response, streaming
|
||||
|
||||
Anthropic SSE uses **named events with content-block indices**, unlike OpenAI's flat frames. Verified event sequence:
|
||||
|
||||
```
|
||||
event: message_start
|
||||
data: {"type":"message_start","message":{"id":"msg_01ABC","type":"message","role":"assistant","model":"reasoning","content":[],"usage":{"input_tokens":16,"output_tokens":0}}}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"Waves are sort keys"}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":0}
|
||||
|
||||
event: content_block_start
|
||||
data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}}
|
||||
|
||||
event: content_block_delta
|
||||
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Wave 4 is empty."}}
|
||||
|
||||
event: content_block_stop
|
||||
data: {"type":"content_block_stop","index":1}
|
||||
|
||||
event: message_delta
|
||||
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":300}}
|
||||
|
||||
event: message_stop
|
||||
data: {"type":"message_stop"}
|
||||
```
|
||||
|
||||
Three details that are easy to get wrong:
|
||||
|
||||
- In `message_delta`, `usage` is a **sibling of** `delta`, not inside it.
|
||||
- The delta field name matches the delta type: `thinking_delta` carries `.thinking`, `text_delta` carries `.text`.
|
||||
- Block index 0 is thinking, index 1 is text. **You only learn reasoning has ended when `content` first appears in an upstream chunk**, so the thinking block must be closed before the text block opens. If a response has no `reasoning_content` at all, the text block is index 0 and no thinking block is emitted.
|
||||
|
||||
### Queue position — non-standard extension
|
||||
|
||||
Anthropic's event set has no way to say "you are queued", because the stream implicitly begins after a slot is acquired. With only 8 slots, queueing is normal here.
|
||||
|
||||
Emitted **before** `message_start`:
|
||||
|
||||
```
|
||||
event: queue
|
||||
data: {"type":"queue","position":3}
|
||||
```
|
||||
|
||||
This is deliberately outside the Anthropic spec. It is safe only because the client is first-party; a strict Anthropic client would ignore the unknown event and show nothing while queued.
|
||||
|
||||
### Errors
|
||||
|
||||
Anthropic error shape, **not** RFC 9457 — the same rejection renders differently depending on which surface received it:
|
||||
|
||||
```json
|
||||
{"type":"error","error":{"type":"invalid_request_error","message":"Unknown model \"gpt-4\". Available: reasoning, ornith:35b, qwen2.5:3b-instruct"}}
|
||||
```
|
||||
|
||||
| Condition | HTTP | `error.type` |
|
||||
|---|---|---|
|
||||
| unknown or missing model | 400 | `invalid_request_error` |
|
||||
| `max_tokens` absent (if required) | 400 | `invalid_request_error` |
|
||||
| malformed JSON | 400 | `invalid_request_error` |
|
||||
| unsupported feature requested | 400 | `invalid_request_error` |
|
||||
| not authenticated | 401 | `authentication_error` |
|
||||
| budget exhausted or queue full | 429 | `rate_limit_error` |
|
||||
| upstream failure | 502 | `api_error` |
|
||||
|
||||
### Deliberately not implemented
|
||||
|
||||
Each returns 400 naming the unsupported feature — never a silent partial implementation:
|
||||
|
||||
tool use and `tool_result` turns, image content blocks, prompt-caching headers, the batch API, multi-block user content, `thinking.budget_tokens` configuration.
|
||||
|
||||
The target client is the riotpiao frontend. Widening scope is a code change with a test, not an accident.
|
||||
|
||||
---
|
||||
|
||||
## Shared behaviour, both dialects
|
||||
|
||||
**One slot controller, keyed by upstream.** A `/v1` request and a `/llm` request contend for the same 8 `reasoning` slots and the same queue, in arrival order. Per-dialect semaphores would each believe they were within budget while together exceeding the physical limit.
|
||||
|
||||
**Streaming is unbuffered** and a client disconnect cancels the upstream immediately. An orphaned generation holds a slot until it completes on its own, which for a 32B model on a Volta GPU can run to minutes.
|
||||
|
||||
**Timeouts** [LIVE]: chat routes are connect 10s / read 1h / write 1h. The hour is deliberate — a 32B model on this hardware routinely exceeds 60s. Any shorter application cap is enforced in gateway logic, never by shortening the proxy timeout.
|
||||
|
||||
**Tool calling** [LIVE]: `reasoning` honours an explicit `tool_choice` but returns `tool_calls: []` under `tool_choice: "auto"` — it reasons about the tool in prose instead. `ornith:35b` returns `finish_reason: "tool_calls"` correctly under `auto`. This is a model property; the gateway does not compensate for it.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
# OpenAI dialect
|
||||
curl -s https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"Why is wave 4 empty?"}],"max_tokens":500}'
|
||||
|
||||
# Anthropic dialect, streaming
|
||||
curl -N -s https://api.riotpiao.com/llm/v1/messages \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","max_tokens":500,"stream":true,
|
||||
"messages":[{"role":"user","content":"Why is wave 4 empty?"}]}'
|
||||
|
||||
# model list
|
||||
curl -s https://api.riotpiao.com/v1/models
|
||||
```
|
||||
-262
@@ -1,262 +0,0 @@
|
||||
# API — queue surface (`/sqs/*`)
|
||||
|
||||
Fronts the Kafka Management Service (`kmsvc`) in namespace `sqs`. SQS-shaped
|
||||
message-plane API over Kafka.
|
||||
|
||||
Status marks:
|
||||
**[LIVE]** verified against the running cluster and the committed proto on 2026-08-19.
|
||||
**[SPEC]** the contract this gateway must implement; not built yet.
|
||||
|
||||
Source of truth for shapes:
|
||||
`~/workplace/kmsvc-proto/proto/kafkamgmt/v1/queue_service.proto`.
|
||||
|
||||
---
|
||||
|
||||
## The important finding: a REST surface already exists [LIVE]
|
||||
|
||||
**Do not build gRPC-to-JSON transcoding.** `kmsvc-manage` already mounts grpc-gateway:
|
||||
|
||||
```go
|
||||
mux := runtime.NewServeMux()
|
||||
kafkamgmtv1.RegisterQueueServiceHandlerServer(ctx, mux, svc)
|
||||
```
|
||||
|
||||
The upstream serves plain REST/JSON on **:8080** and plain gRPC on **:9090**. Neither
|
||||
gRPC-Web nor server reflection is enabled.
|
||||
|
||||
So `/sqs/*` is a **path-stripping reverse proxy plus authentication**, not a protocol
|
||||
translator. That makes it dramatically cheaper than the LLM surface.
|
||||
|
||||
```
|
||||
api.riotpiao.com/sqs/v1/queues/{q}/messages
|
||||
| strip /sqs, authenticate
|
||||
v
|
||||
management-service.sqs.svc.cluster.local:8080/v1/queues/{q}/messages
|
||||
```
|
||||
|
||||
Upstream: Deployment `management-service`, 3 replicas, HPA 3-9, Service ClusterIP
|
||||
`10.98.3.138`, ports `8080` (http) and `9090` (grpc).
|
||||
|
||||
---
|
||||
|
||||
## Endpoints [LIVE — HTTP annotations from the proto]
|
||||
|
||||
Six operations. All unary. No streaming, no subscribe.
|
||||
|
||||
| Method | Path (after `/sqs` strip) | RPC |
|
||||
|---|---|---|
|
||||
| POST | `/v1/queues/{queue_name}/messages` | `SendMessage` |
|
||||
| POST | `/v1/queues/{queue_name}/messages:batch` | `SendMessageBatch` |
|
||||
| GET | `/v1/queues/{queue_name}/messages` | `ReceiveMessage` |
|
||||
| DELETE | `/v1/queues/{queue_name}/messages/{receipt_handle}` | `DeleteMessage` |
|
||||
| POST | `/v1/queues/{queue_name}/messages:batchDelete` | `DeleteMessageBatch` |
|
||||
| PATCH | `/v1/queues/{queue_name}/messages/{receipt_handle}` | `ChangeMessageVisibility` |
|
||||
|
||||
---
|
||||
|
||||
## Two wire-format traps [LIVE]
|
||||
|
||||
Both follow from grpc-gateway defaults, and both will surprise anyone who reads only
|
||||
the proto.
|
||||
|
||||
**1. `bytes` fields are base64 in JSON.** `SendMessageRequest.message_body` and
|
||||
`Message.body` are proto `bytes`. The JSONPB marshaler encodes them as base64 strings.
|
||||
Sending raw text will not do what you expect.
|
||||
|
||||
**2. Field names are lowerCamelCase.** `cmd/server/main.go` calls bare
|
||||
`runtime.NewServeMux()` with no marshaler options, so `OrigName` is false. The wire uses
|
||||
`messageBody`, `receiptHandle`, `maxNumberOfMessages` — not the snake_case names in the
|
||||
proto.
|
||||
|
||||
Document both prominently or every first-time caller loses an hour.
|
||||
|
||||
---
|
||||
|
||||
## Message shapes [LIVE — from the proto]
|
||||
|
||||
### Send
|
||||
|
||||
```
|
||||
POST /sqs/v1/queues/agent-worker-queue/messages
|
||||
{
|
||||
"messageBody": "aGVsbG8gd29ybGQ=", // base64 of "hello world"
|
||||
"messageAttributes": {"values": {"k": "v"}},
|
||||
"messageGroupId": "", // FIFO only
|
||||
"messageDeduplicationId": "", // FIFO only
|
||||
"delaySeconds": 0 // 0-900
|
||||
}
|
||||
-> {"messageId": "...", "sequenceNumber": ""} // sequenceNumber FIFO only
|
||||
```
|
||||
|
||||
### Receive — long poll
|
||||
|
||||
```
|
||||
GET /sqs/v1/queues/agent-worker-queue/messages
|
||||
?maxNumberOfMessages=10 // <= 10
|
||||
&waitTimeSeconds=20 // 0-20
|
||||
&visibilityTimeoutSeconds=30 // optional override
|
||||
|
||||
-> {"messages": [{
|
||||
"messageId": "...",
|
||||
"receiptHandle": "...",
|
||||
"body": "aGVsbG8gd29ybGQ=",
|
||||
"attributes": {"values": {}},
|
||||
"receiveCount": 1,
|
||||
"messageGroupId": "",
|
||||
"enqueuedAt": "2026-08-19T16:29:07Z"
|
||||
}]}
|
||||
```
|
||||
|
||||
### Delete — the ack
|
||||
|
||||
```
|
||||
DELETE /sqs/v1/queues/agent-worker-queue/messages/{receiptHandle}
|
||||
-> {}
|
||||
```
|
||||
|
||||
### Change visibility
|
||||
|
||||
```
|
||||
PATCH /sqs/v1/queues/agent-worker-queue/messages/{receiptHandle}
|
||||
{"visibilityTimeoutSeconds": 60} // 0-43200
|
||||
-> {}
|
||||
```
|
||||
|
||||
### Batch
|
||||
|
||||
Both batch calls take `entries[]` with a caller-assigned `id`, and return partial
|
||||
success:
|
||||
|
||||
```json
|
||||
{"successful": [{"id": "1", "messageId": "..."}],
|
||||
"failed": [{"id": "2", "error": "..."}]}
|
||||
```
|
||||
|
||||
A batch call can return 200 with entries in `failed`. Callers must inspect the body,
|
||||
not just the status.
|
||||
|
||||
### Limits [LIVE — from the SDK]
|
||||
|
||||
`MaxMessageBodyBytes = 262144` (256 KiB), `MaxReceiveMessages = 10`,
|
||||
`MaxWaitTimeSeconds = 20`.
|
||||
|
||||
---
|
||||
|
||||
## Semantics
|
||||
|
||||
At-least-once, SQS-style. Receive leases a message for the visibility timeout; the
|
||||
caller must `DeleteMessage` to acknowledge. An un-deleted message reappears after the
|
||||
timeout and `receiveCount` increments. After `maxReceiveCount` (default 5) it goes to
|
||||
the DLQ if one is configured.
|
||||
|
||||
**Long-polling matters for the gateway.** `waitTimeSeconds` up to 20 means a `GET` can
|
||||
legitimately hold open for 20 seconds returning nothing. Read timeouts must exceed that
|
||||
comfortably, and a client disconnect must cancel upstream — the same requirement as the
|
||||
LLM surface, for the same reason.
|
||||
|
||||
---
|
||||
|
||||
## Error mapping [SPEC]
|
||||
|
||||
The SDK maps gRPC codes to sentinel errors; grpc-gateway maps them to HTTP. Use this as
|
||||
the gateway's status contract:
|
||||
|
||||
| gRPC code | HTTP | SDK sentinel |
|
||||
|---|---|---|
|
||||
| `NotFound` | 404 | `ErrQueueNotFound` |
|
||||
| `AlreadyExists` | 409 | `ErrAlreadyExists` |
|
||||
| `InvalidArgument` | 400 | `ErrInvalidArgument` |
|
||||
| `Unauthenticated` | 401 | `ErrUnauthenticated` |
|
||||
| `ResourceExhausted` | 429 | `ErrMessageTooLarge` |
|
||||
|
||||
Upstream errors arrive in the grpc-gateway envelope
|
||||
`{"code": 5, "message": "Not Found", "details": []}`. Decide deliberately whether
|
||||
`/sqs/*` passes that through or re-renders it as RFC 9457 to match `/v1/*`.
|
||||
Recommendation: **pass through**, so the gateway does not become a second, subtly
|
||||
different error vocabulary for the same upstream.
|
||||
|
||||
---
|
||||
|
||||
## Queue lifecycle is NOT in this API [LIVE]
|
||||
|
||||
There is no `CreateQueue`, `DeleteQueue`, or `ListQueues` RPC. The proto says so
|
||||
explicitly:
|
||||
|
||||
```proto
|
||||
// Queue lifecycle (create/delete/configure) is managed via the Queue CRD,
|
||||
// not this service
|
||||
```
|
||||
|
||||
Queues are Kubernetes resources — `queues.kmsvc.io/v1`, namespaced. `kmsvc-cli`'s
|
||||
`create-queue` and `delete-queue` talk to the Kubernetes API, not to kmsvc.
|
||||
|
||||
**This is a hard boundary for the gateway.** Exposing queue creation over `/sqs/*` would
|
||||
require the gateway to hold Kubernetes write credentials, which violates **G2**. Do not
|
||||
add it. If declarative queue management ever needs a public surface, it belongs behind a
|
||||
separate component with its own RBAC — not in the public edge process.
|
||||
|
||||
Queue spec fields, for reference when reading a queue's configuration:
|
||||
`fifoQueue`, `isDLQ`, `deadLetterTargetQueue`, `delaySeconds` (0-900),
|
||||
`maxReceiveCount` (default 5), `messageRetentionPeriodSeconds` (default 345600),
|
||||
`visibilityTimeoutSeconds` (default 30), `minShards`, `maxShards` (default 8),
|
||||
`partitionsPerShard` (default 6), `shardSplitThresholdBytesPerSec`,
|
||||
`shardSplitCooldownSeconds`.
|
||||
|
||||
Kafka topics are named `kmsvc.{queue}.shard-{id}` and are created by `queue-operator`
|
||||
directly via the Kafka Admin API — there are no `KafkaTopic` CRs.
|
||||
|
||||
Currently one queue exists: `agent-worker-queue` in namespace `sqs`, phase `Ready`,
|
||||
1 shard.
|
||||
|
||||
---
|
||||
|
||||
## Authentication [SPEC]
|
||||
|
||||
`Authorization: Bearer <jwt>`, same as every other gateway surface.
|
||||
|
||||
**The upstream enforces nothing.** `kmsvc`'s auth interceptor exists but is never wired,
|
||||
and the REST surface is mounted with the in-process grpc-gateway variant that bypasses
|
||||
gRPC interceptors regardless. Both `:8080` and `:9090` are currently open, and
|
||||
`kmsvc.riotpiao.com` is publicly routed.
|
||||
|
||||
The gateway is therefore the only authentication boundary for this surface.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Workflow start.** Nothing in kmsvc starts a Temporal workflow — no such RPC exists,
|
||||
and grep for `ExecuteWorkflow`/`StartWorkflow` across `kmsvc-manage`, `kmsvc-sdk` and
|
||||
`kmsvc-cli` returns nothing. A caller dials `temporal-frontend.temporal.svc:7233`
|
||||
with a Temporal SDK directly. A `/workflow/*` surface is net-new code, not a proxy
|
||||
route — see [task 7.3](../tasks/7.3-workflow-prefix.md).
|
||||
- **DLQ operations.** `kmsvc-cli`'s `dlq peek` and `dlq redrive` are client-side
|
||||
compositions of the six RPCs, not server operations. Redrive is a non-atomic
|
||||
Receive-Send-Delete. If `/sqs/*` should offer redrive, that is new logic with real
|
||||
failure modes, not a proxied call.
|
||||
- **Kafka direct access.** No external listener exists; the bootstrap
|
||||
`kmsvc-kafka-bootstrap.sqs.svc.cluster.local:9092` is cluster-internal only. The
|
||||
gateway proxies kmsvc, never Kafka.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
Q=agent-worker-queue
|
||||
|
||||
# send (body must be base64)
|
||||
curl -s -X POST https://api.riotpiao.com/sqs/v1/queues/$Q/messages \
|
||||
-H 'content-type: application/json' \
|
||||
-d "{\"messageBody\":\"$(printf 'hello world' | base64)\"}"
|
||||
|
||||
# receive, long poll 20s
|
||||
curl -s "https://api.riotpiao.com/sqs/v1/queues/$Q/messages?maxNumberOfMessages=10&waitTimeSeconds=20"
|
||||
|
||||
# acknowledge
|
||||
curl -s -X DELETE https://api.riotpiao.com/sqs/v1/queues/$Q/messages/$RECEIPT
|
||||
|
||||
# extend the lease
|
||||
curl -s -X PATCH https://api.riotpiao.com/sqs/v1/queues/$Q/messages/$RECEIPT \
|
||||
-H 'content-type: application/json' -d '{"visibilityTimeoutSeconds":60}'
|
||||
```
|
||||
@@ -1,143 +0,0 @@
|
||||
# Kong retirement — inventory and cutover
|
||||
|
||||
Everything Kong does on `api.riotpiao.com` today, and where it goes. Inventory
|
||||
verified live against context `admin@homelab-cluster` on 2026-08-19.
|
||||
|
||||
Source of the objects being retired: `~/workplace/homelab/k8s/apps/api/` and
|
||||
`k8s/argocd/apps/55-api-gateway.yaml`.
|
||||
|
||||
## What is running now
|
||||
|
||||
Kong OSS 3.4.1, Helm chart from `https://charts.konghq.com`, DB-less, namespace
|
||||
`api`, Argo Application `kong` at sync wave 7. Two replicas. Fronted by
|
||||
`ingress-nginx` via Ingress `api/api`, which catch-alls `/` on `api.riotpiao.com`
|
||||
to `kong-proxy:80`.
|
||||
|
||||
Eleven ReplicaSets exist on the Kong Deployment, the newest minutes old — this
|
||||
config is being actively iterated, so re-verify the inventory immediately before
|
||||
cutover.
|
||||
|
||||
## Routing table to port
|
||||
|
||||
Seven `ingressClassName: kong` Ingresses. Six in `llm-serving`, one in `agent-pod`.
|
||||
|
||||
| Method | Path | Upstream | Transform applied by Kong |
|
||||
|---|---|---|---|
|
||||
| GET | `/v1/models` | — | `request-termination`: static 200 JSON, upstream never contacted |
|
||||
| POST | `/v1/reasoning/chat/completions` | `reasoning-predictor:80` | force body `model=reasoning`, rewrite URI to `/v1/chat/completions` |
|
||||
| POST | `/v1/ornith/chat/completions` | `ornith-predictor:80` | force body `model=ornith:35b`, rewrite URI |
|
||||
| POST | `/v1/qwen/chat/completions` | `ornith-predictor:80` | force body `model=qwen2.5:3b-instruct`, rewrite URI |
|
||||
| POST | `/v1/embeddings` | `embeddings-predictor:80` | none — TEI already serves the canonical path |
|
||||
| POST | `/v1/rerank` | `reranker-predictor:80` | rewrite URI to `/rerank` (TEI does not serve `/v1/rerank`) |
|
||||
| GET/WS | `/console`, `/run`, `/sessions` | `agent-hub:9090` (`agent-pod` ns) | none, `strip-path: false` |
|
||||
|
||||
Upstream model map, from the manifest comments and confirmed live:
|
||||
|
||||
- `reasoning` → `reasoning-predictor` — vLLM, DeepSeek-R1-Distill-Qwen-32B, 2 replicas,
|
||||
`--max-num-seqs=4`, `--max-model-len=16384`, `--reasoning-parser=deepseek_r1`,
|
||||
`--enable-auto-tool-choice --tool-call-parser=hermes`
|
||||
- `ornith:35b` → `ornith-predictor` — Ollama, 2 replicas
|
||||
- `qwen2.5:3b-instruct` → `ornith-predictor` — same pods; both models stay resident via
|
||||
`OLLAMA_MAX_LOADED_MODELS=2`, `OLLAMA_KEEP_ALIVE=-1`
|
||||
- `nomic-ai/nomic-embed-text-v2-moe` → `embeddings-predictor` — TEI
|
||||
- `BAAI/bge-reranker-base` → `reranker-predictor` — TEI
|
||||
|
||||
### The path-per-model surface goes away
|
||||
|
||||
The three chat paths exist only because Kong OSS cannot dispatch on the request
|
||||
body. The gateway serves a single `POST /v1/chat/completions` and selects the
|
||||
upstream from the body's `model` field.
|
||||
|
||||
Keep the old paths as aliases during cutover so live clients do not break, then
|
||||
remove them once callers have migrated. pi is a live caller today.
|
||||
|
||||
### `/v1/models` should not be ported verbatim
|
||||
|
||||
Kong serves a hardcoded list via `request-termination`. The manifest already flags
|
||||
that it can drift from what the engines actually serve. Derive the response from
|
||||
the gateway's configured upstream map instead, so the list cannot disagree with
|
||||
what routing will accept.
|
||||
|
||||
## Plugins being retired
|
||||
|
||||
| Plugin | Scope | Replacement |
|
||||
|---|---|---|
|
||||
| `llm-rewrite-reasoning` / `-ornith` / `-qwen` | llm-serving | body-based dispatch in `internal/llm` |
|
||||
| `llm-rewrite-rerank` | llm-serving | per-upstream path rewrite in the route table |
|
||||
| `llm-models-list` | llm-serving | derived from the upstream map |
|
||||
| `prometheus` | **cluster-wide** | `internal/observability` — must expose bandwidth, latency, status codes, upstream health or observability regresses |
|
||||
|
||||
No `rate-limiting` plugin exists anywhere in the cluster. REQUIREMENTS.md §4 Tier 2
|
||||
describes it as an existing layer; it is not built. Nothing to migrate — it is net
|
||||
new work, and it now belongs in the gateway rather than in Kong.
|
||||
|
||||
## Auth: currently off, must land on
|
||||
|
||||
`KongConsumer model-invoker` exists in namespace `api` and stays defined, but the
|
||||
`key-auth` plugin is commented out and every route has `model-key-auth` stripped
|
||||
from its `konghq.com/plugins` annotation.
|
||||
|
||||
**The model API is unauthenticated right now.** Confirmed live 2026-08-19: a request
|
||||
to `/v1/reasoning/chat/completions` with no credentials returns 200.
|
||||
|
||||
The reason is recorded in `model-auth.yaml` — Kong's `key-auth` accepts a raw
|
||||
`apikey:` header but rejects `Authorization: Bearer`, which blocks every
|
||||
OpenAI-compatible client. That is why `~/.pi/agent/models.json` carries a
|
||||
`customHeaders: {apikey: ...}` block.
|
||||
|
||||
The gateway reads Bearer tokens directly and validates them against Authentik via
|
||||
JWKS. `AUTH-PLAN.md`'s pinned-RSA-key approach and its rotation runbook are not
|
||||
needed and should not be carried over.
|
||||
|
||||
Ship auth behind a flag. Turning it on breaks every current caller until they hold
|
||||
a token — pi included.
|
||||
|
||||
## Timeouts
|
||||
|
||||
Kong today:
|
||||
|
||||
| Route class | connect | read | write |
|
||||
|---|---|---|---|
|
||||
| chat | 10s | **1h** | 1h |
|
||||
| embeddings / rerank | 10s | 10m | 10m |
|
||||
|
||||
nginx in front sets `proxy-read-timeout: 3600`, `proxy-send-timeout: 3600`,
|
||||
`proxy-buffering: off`, `proxy-body-size: 0`. Those stay — they are what makes token
|
||||
streaming work, and the gateway needs the same treatment from nginx.
|
||||
|
||||
The 1-hour read timeout is deliberate: a 32B model on a Volta GPU routinely exceeds
|
||||
60s. Any shorter server-side cap must be enforced *in the gateway*, not by shortening
|
||||
the proxy timeout, or long legitimate generations get truncated mid-stream.
|
||||
|
||||
## Cutover
|
||||
|
||||
Reversible at every step. Kong keeps serving until the last step.
|
||||
|
||||
1. Deploy the gateway alongside Kong, unexposed. Verify in-cluster against
|
||||
`http://homelab-frontend.api.svc.cluster.local`.
|
||||
2. Compare gateway and Kong responses for every route in the table above, including
|
||||
a streaming chat request and a client disconnect mid-stream.
|
||||
3. Repoint Ingress `api/api` from `kong-proxy:80` to the gateway Service. **This is
|
||||
the cutover.** Reverting is a one-line change to the same Ingress.
|
||||
4. Soak. Watch gateway metrics and pi traffic.
|
||||
5. Delete the seven kong-class Ingresses and the six KongPlugin CRs.
|
||||
6. Remove the `kong` Application from `k8s/argocd/apps/55-api-gateway.yaml`; let Argo
|
||||
prune the Helm release, the CRDs and namespace leftovers.
|
||||
|
||||
Steps 1–4 are reversible in seconds. Step 5 onward is not — do not start it until the
|
||||
soak is clean.
|
||||
|
||||
All of this flows through git and Argo. No `kubectl apply`, no `helm upgrade`.
|
||||
|
||||
## Loose ends
|
||||
|
||||
- `agent-pod/console` is publicly routed, unauthenticated, accepts free-form prompts
|
||||
into a shell-capable container, and exposes a WebSocket. Migrating it behind the
|
||||
gateway's auth is a security fix, not merely a port. Treat WebSocket upgrade as an
|
||||
explicit requirement of the proxy layer.
|
||||
- Eight `*.example.com` hosts exist on istio-class Ingresses in `llm-serving`
|
||||
(`{embeddings,ornith,reasoning,reranker}[-predictor]-llm-serving.example.com`).
|
||||
KServe defaults, not public, not Kong's — out of scope here, but they exist and
|
||||
should not be mistaken for gateway routes.
|
||||
- Ingress class split across the cluster is 7 kong / 17 nginx / 4 istio. Only the 7
|
||||
kong ones are in scope.
|
||||
@@ -1,542 +0,0 @@
|
||||
# Service Usage Guide
|
||||
|
||||
Complete guide for calling all gateway-backed services via `api.riotpiao.com`.
|
||||
|
||||
**Table of Contents**
|
||||
- [Quick Start](#quick-start)
|
||||
- [Authentication](#authentication)
|
||||
- [Service Map](#service-map)
|
||||
- [Service-Specific Guides](#service-specific-guides)
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
All services use the **X-Service** header to route requests:
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/path \
|
||||
-H "X-Service: <service_name>" \
|
||||
-H "Authorization: Bearer <jwt>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"key": "value"}'
|
||||
```
|
||||
|
||||
| Header | Purpose | Example |
|
||||
|---|---|---|
|
||||
| `X-Service` | Route to named service | `X-Service: sqs` |
|
||||
| `X-Resource` | (Optional) Resource ID for auth | `X-Resource: agent-worker-queue` |
|
||||
| `Authorization` | Bearer token (required for auth-protected services) | `Authorization: Bearer eyJ...` |
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
### Getting a Token
|
||||
|
||||
**From Authentik (OAuth2 client credentials flow):**
|
||||
|
||||
```bash
|
||||
AUTHENTIK_URL=https://authentik.riotpiao.com
|
||||
CLIENT_ID="your-client-id"
|
||||
CLIENT_SECRET="your-client-secret"
|
||||
|
||||
TOKEN=$(curl -s -X POST ${AUTHENTIK_URL}/application/o/token/ \
|
||||
-d "grant_type=client_credentials&client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&scope=openid" \
|
||||
| jq -r '.access_token')
|
||||
|
||||
echo $TOKEN
|
||||
```
|
||||
|
||||
Replace `your-client-id` and `your-client-secret` with Authentik app credentials.
|
||||
|
||||
### Service-Specific Auth
|
||||
|
||||
| Service | Auth Required | Token Audience | Notes |
|
||||
|---|---|---|---|
|
||||
| **sqs** | ✅ Yes (gateway validates) | `sqs` | JWT signature & claims verified by gateway before proxying |
|
||||
| **memory** | ❌ No | — | Pass-through (service-owned if needed) |
|
||||
| **s3** (MinIO) | ❌ No | — | Native OIDC support (service-owned) |
|
||||
| **iam** | ❌ No | — | Pass-through (service-owned if needed) |
|
||||
| **workflow** (Temporal) | ❌ No | — | Native JWT support (service-owned) |
|
||||
|
||||
---
|
||||
|
||||
## Service Map
|
||||
|
||||
### Gateway Services (via X-Service header)
|
||||
|
||||
```
|
||||
api.riotpiao.com
|
||||
├─ X-Service: sqs
|
||||
│ ├─ Upstream: management-service.sqs.svc.cluster.local:8080
|
||||
│ ├─ Auth: ✅ Gateway validates JWT
|
||||
│ └─ Docs: docs/API-sqs.md
|
||||
│
|
||||
├─ X-Service: memory
|
||||
│ ├─ Upstream: poimen-memory.memory.svc.cluster.local:9090
|
||||
│ ├─ Auth: ❌ Pass-through (service-owned)
|
||||
│ └─ Docs: See Memory Service section below
|
||||
│
|
||||
├─ X-Service: s3
|
||||
│ ├─ Upstream: minio.data.svc.cluster.local:9000
|
||||
│ ├─ Auth: ❌ Native OIDC (service-owned)
|
||||
│ └─ Notes: S3-compatible API
|
||||
│
|
||||
├─ X-Service: iam
|
||||
│ ├─ Upstream: keycloak.iam.svc.cluster.local:8080 (or equivalent)
|
||||
│ ├─ Auth: ❌ Pass-through (service-owned)
|
||||
│ └─ Docs: See IAM Service section below
|
||||
│
|
||||
└─ X-Service: workflow
|
||||
├─ Upstream: temporal-frontend.temporal.svc.cluster.local:7233
|
||||
├─ Auth: ❌ Native JWT support (service-owned)
|
||||
├─ Protocol: gRPC only (returns 501 for HTTP)
|
||||
└─ Docs: docs/TEMPORAL_USAGE.md
|
||||
```
|
||||
|
||||
### Path Prefixes (legacy, before X-Service migration)
|
||||
|
||||
```
|
||||
api.riotpiao.com
|
||||
├─ /v1/* → llm-serving (predictors: vLLM, Ollama, TEI)
|
||||
├─ /sqs/* → management-service (Kafka queues)
|
||||
├─ /workflow/* → Temporal (workflows)
|
||||
└─ /cluster/* → atlas (topology & Argo delivery)
|
||||
```
|
||||
|
||||
**Migration note:** X-Service routing is the current standard. Path prefixes are deprecated.
|
||||
|
||||
---
|
||||
|
||||
## Service-Specific Guides
|
||||
|
||||
### SQS (Kafka Queue Management)
|
||||
|
||||
**Endpoint:** `POST https://api.riotpiao.com/`
|
||||
**Headers:**
|
||||
```
|
||||
X-Service: sqs
|
||||
Authorization: Bearer <jwt>
|
||||
```
|
||||
|
||||
**Send a message:**
|
||||
|
||||
```bash
|
||||
QUEUE="agent-worker-queue"
|
||||
BODY=$(printf "hello world" | base64)
|
||||
|
||||
curl -X POST https://api.riotpiao.com/ \
|
||||
-H "X-Service: sqs" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"messageBody\": \"$BODY\",
|
||||
\"messageAttributes\": {\"values\": {}},
|
||||
\"delaySeconds\": 0
|
||||
}"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"messageId": "d9f94e63-b2c1-4e9f-8c5f-8d5e3c1b7a0f",
|
||||
"sequenceNumber": ""
|
||||
}
|
||||
```
|
||||
|
||||
**Receive messages (long poll, up to 20s):**
|
||||
|
||||
```bash
|
||||
QUEUE="agent-worker-queue"
|
||||
|
||||
curl -X GET "https://api.riotpiao.com/?X-Service=sqs&queue=$QUEUE&maxNumberOfMessages=10&waitTimeSeconds=20" \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"messageId": "d9f94e63-b2c1-4e9f-8c5f-8d5e3c1b7a0f",
|
||||
"receiptHandle": "...",
|
||||
"body": "aGVsbG8gd29ybGQ=",
|
||||
"attributes": {"values": {}},
|
||||
"receiveCount": 1,
|
||||
"enqueuedAt": "2026-08-27T22:18:37Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Acknowledge (delete) a message:**
|
||||
|
||||
```bash
|
||||
RECEIPT="..."
|
||||
|
||||
curl -X DELETE https://api.riotpiao.com/ \
|
||||
-H "X-Service: sqs" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"receiptHandle\": \"$RECEIPT\"}"
|
||||
```
|
||||
|
||||
**Full documentation:** [docs/API-sqs.md](API-sqs.md)
|
||||
|
||||
---
|
||||
|
||||
### Memory Service (Context & Embeddings)
|
||||
|
||||
**Endpoint:** `https://api.riotpiao.com/`
|
||||
**Headers:**
|
||||
```
|
||||
X-Service: memory
|
||||
```
|
||||
|
||||
**Status:** Service definition in progress. Uses PostgreSQL + pg_vector for embeddings.
|
||||
|
||||
**Planned operations:**
|
||||
- Store session memory / agent context
|
||||
- Query by similarity (embedding search)
|
||||
- Update with approval workflow (GRM/Git review)
|
||||
|
||||
**Coming soon:** Full API documentation.
|
||||
|
||||
---
|
||||
|
||||
### S3 / MinIO (Object Storage)
|
||||
|
||||
**Endpoint:** `https://api.riotpiao.com/`
|
||||
**Headers:**
|
||||
```
|
||||
X-Service: s3
|
||||
```
|
||||
|
||||
**List buckets:**
|
||||
|
||||
```bash
|
||||
curl -X GET https://api.riotpiao.com/ \
|
||||
-H "X-Service: s3" \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
**List objects in bucket:**
|
||||
|
||||
```bash
|
||||
curl -X GET https://api.riotpiao.com/?bucket=my-bucket&prefix=data/ \
|
||||
-H "X-Service: s3"
|
||||
```
|
||||
|
||||
**Put object:**
|
||||
|
||||
```bash
|
||||
curl -X PUT https://api.riotpiao.com/my-bucket/path/to/object.json \
|
||||
-H "X-Service: s3" \
|
||||
--data-binary @object.json
|
||||
```
|
||||
|
||||
**Get object:**
|
||||
|
||||
```bash
|
||||
curl -X GET https://api.riotpiao.com/my-bucket/path/to/object.json \
|
||||
-H "X-Service: s3"
|
||||
```
|
||||
|
||||
**Full S3/MinIO API:** Standard AWS S3 compatible API. See [MinIO docs](https://min.io/docs/minio/linux/reference/minio-mc/mc-ls.html).
|
||||
|
||||
---
|
||||
|
||||
### IAM (Identity & Access Management)
|
||||
|
||||
**Endpoint:** `https://api.riotpiao.com/`
|
||||
**Headers:**
|
||||
```
|
||||
X-Service: iam
|
||||
```
|
||||
|
||||
**List roles:**
|
||||
|
||||
```bash
|
||||
curl -X GET https://api.riotpiao.com/roles \
|
||||
-H "X-Service: iam"
|
||||
```
|
||||
|
||||
**Get user:**
|
||||
|
||||
```bash
|
||||
curl -X GET https://api.riotpiao.com/users/alice \
|
||||
-H "X-Service: iam"
|
||||
```
|
||||
|
||||
**Create user:**
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/users \
|
||||
-H "X-Service: iam" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"username\": \"bob\",
|
||||
\"email\": \"[email protected]\",
|
||||
\"password\": \"secure-password\"
|
||||
}"
|
||||
```
|
||||
|
||||
**Assign role to user:**
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/users/bob/roles \
|
||||
-H "X-Service: iam" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"role\": \"admin\"}"
|
||||
```
|
||||
|
||||
**Full documentation:** Service-specific (depends on IAM backend).
|
||||
|
||||
---
|
||||
|
||||
### Workflow (Temporal)
|
||||
|
||||
**Endpoint:** `temporal-frontend.temporal.svc.cluster.local:7233`
|
||||
**Protocol:** gRPC only
|
||||
**Note:** HTTP requests return **501 Not Implemented**
|
||||
|
||||
Use Temporal SDK directly:
|
||||
|
||||
```go
|
||||
import "go.temporal.io/sdk/client"
|
||||
|
||||
c, _ := client.Dial(client.Options{HostPort: "temporal-frontend.temporal.svc.cluster.local:7233"})
|
||||
defer c.Close()
|
||||
|
||||
// Start workflow
|
||||
run, _ := c.ExecuteWorkflow(ctx, opts, YourWorkflow, args...)
|
||||
var result YourWorkflowResult
|
||||
run.Get(ctx, &result)
|
||||
```
|
||||
|
||||
**Full documentation:** [docs/TEMPORAL_USAGE.md](../TEMPORAL_USAGE.md)
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Standard Error Response
|
||||
|
||||
All services return errors in **RFC 9457 Problem Details** format:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "https://api.riotpiao.com/problem/not-found",
|
||||
"title": "Not Found",
|
||||
"status": 404,
|
||||
"detail": "Resource does not exist",
|
||||
"instance": "/sqs/v1/queues/nonexistent"
|
||||
}
|
||||
```
|
||||
|
||||
### Common Status Codes
|
||||
|
||||
| Code | Meaning | Example |
|
||||
|---|---|---|
|
||||
| `200 OK` | Success | Message sent, resource retrieved |
|
||||
| `201 Created` | Resource created | Queue created, object uploaded |
|
||||
| `204 No Content` | Success (no body) | Message deleted |
|
||||
| `400 Bad Request` | Invalid input | Message body too large, invalid field |
|
||||
| `401 Unauthorized` | Missing/invalid token | No Authorization header, token expired |
|
||||
| `403 Forbidden` | Token valid but insufficient permissions | User lacks sqs:write permission |
|
||||
| `404 Not Found` | Resource not found | Queue doesn't exist, object not found |
|
||||
| `429 Too Many Requests` | Rate limit exceeded | Per-user budget exhausted |
|
||||
| `502 Bad Gateway` | Upstream unreachable | Service is down or network issue |
|
||||
| `501 Not Implemented` | Operation not supported | gRPC request via HTTP |
|
||||
|
||||
### SQS-Specific Error Mapping
|
||||
|
||||
SQS errors (from kmsvc) map as follows:
|
||||
|
||||
| gRPC Code | HTTP Status | Message |
|
||||
|---|---|---|
|
||||
| `NotFound` | `404` | Queue or message not found |
|
||||
| `AlreadyExists` | `409` | Queue already exists |
|
||||
| `InvalidArgument` | `400` | Message body too large, invalid parameter |
|
||||
| `Unauthenticated` | `401` | Missing Authorization header |
|
||||
| `ResourceExhausted` | `429` | Message too large, quota exceeded |
|
||||
|
||||
---
|
||||
|
||||
## Request/Response Examples
|
||||
|
||||
### Example 1: Send SQS Message with Auth
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
GATEWAY="https://api.riotpiao.com"
|
||||
AUTHENTIK="https://authentik.riotpiao.com"
|
||||
CLIENT_ID="sqs-client"
|
||||
CLIENT_SECRET="secret123"
|
||||
QUEUE="agent-worker-queue"
|
||||
|
||||
# Get token
|
||||
TOKEN=$(curl -s -X POST ${AUTHENTIK}/application/o/token/ \
|
||||
-d "grant_type=client_credentials&client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&scope=openid" \
|
||||
| jq -r '.access_token')
|
||||
|
||||
# Send message
|
||||
BODY=$(echo "process this task" | base64)
|
||||
|
||||
curl -X POST ${GATEWAY}/ \
|
||||
-H "X-Service: sqs" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"messageBody\": \"$BODY\"}" | jq .
|
||||
```
|
||||
|
||||
### Example 2: Receive & Process Queue
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
GATEWAY="https://api.riotpiao.com"
|
||||
TOKEN="..."
|
||||
QUEUE="agent-worker-queue"
|
||||
MAX_MSGS=10
|
||||
WAIT_SECS=20
|
||||
|
||||
while true; do
|
||||
# Receive messages (long poll)
|
||||
RESPONSE=$(curl -s -X GET "${GATEWAY}/?X-Service=sqs&queue=${QUEUE}&maxNumberOfMessages=${MAX_MSGS}&waitTimeSeconds=${WAIT_SECS}" \
|
||||
-H "Authorization: Bearer $TOKEN")
|
||||
|
||||
MESSAGES=$(echo "$RESPONSE" | jq '.messages')
|
||||
|
||||
if [[ "$MESSAGES" == "null" ]]; then
|
||||
echo "No messages (timeout)"
|
||||
continue
|
||||
fi
|
||||
|
||||
# Process each message
|
||||
echo "$RESPONSE" | jq -r '.messages[] | @base64d' | while read -r MSG; do
|
||||
echo "Processing: $MSG"
|
||||
# Do work...
|
||||
|
||||
# Acknowledge message
|
||||
RECEIPT=$(echo "$RESPONSE" | jq -r '.messages[0].receiptHandle')
|
||||
curl -s -X DELETE ${GATEWAY}/ \
|
||||
-H "X-Service: sqs" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d "{\"receiptHandle\": \"$RECEIPT\"}"
|
||||
done
|
||||
done
|
||||
```
|
||||
|
||||
### Example 3: S3 Workflow (Upload & List)
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
GATEWAY="https://api.riotpiao.com"
|
||||
BUCKET="my-data"
|
||||
|
||||
# Upload file
|
||||
echo "Uploading..."
|
||||
curl -X PUT ${GATEWAY}/${BUCKET}/backup-$(date +%s).tar.gz \
|
||||
-H "X-Service: s3" \
|
||||
--data-binary @backup.tar.gz
|
||||
|
||||
# List objects
|
||||
echo "Listing..."
|
||||
curl -X GET "${GATEWAY}/?bucket=${BUCKET}&prefix=backup-" \
|
||||
-H "X-Service: s3" | jq '.Contents[]'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Integration Tests
|
||||
|
||||
Run the full test suite:
|
||||
|
||||
```bash
|
||||
./scripts/test-integration.sh
|
||||
```
|
||||
|
||||
Run specific service tests:
|
||||
|
||||
```bash
|
||||
GATEWAY_URL=https://api.riotpiao.com go test -tags integration -v -run TestSQS ./internal/serviceadapter
|
||||
```
|
||||
|
||||
### Local Testing
|
||||
|
||||
Start local gateway with test services:
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start gateway
|
||||
CONFIG_PATH=k8s/configmap.yaml go run ./cmd/gateway
|
||||
|
||||
# Terminal 2: Run tests
|
||||
./scripts/test-integration.sh
|
||||
```
|
||||
|
||||
### Canary Deployment
|
||||
|
||||
Test a single replica before rolling out:
|
||||
|
||||
```bash
|
||||
./scripts/test-canary.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### Check Gateway Logs
|
||||
|
||||
```bash
|
||||
kubectl -n api logs -l app=api-gateway --tail=100 -f
|
||||
```
|
||||
|
||||
### Port-Forward to Service
|
||||
|
||||
```bash
|
||||
kubectl -n sqs port-forward svc/management-service 8080:8080
|
||||
curl http://localhost:8080/v1/queues
|
||||
```
|
||||
|
||||
### Verify Service Availability
|
||||
|
||||
```bash
|
||||
kubectl get svc -A | grep -E "management-service|poimen-memory|minio|temporal"
|
||||
```
|
||||
|
||||
### Test Direct Service Access
|
||||
|
||||
```bash
|
||||
kubectl -n sqs exec -it deployment/management-service -- \
|
||||
curl -s http://localhost:8080/v1/queues | jq .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rate Limits & Quotas
|
||||
|
||||
| Service | Limit | Notes |
|
||||
|---|---|---|
|
||||
| **SQS** | Per-user budget (tokens) | Budget enforced per Authentik user |
|
||||
| **S3** | MinIO quotas | Set per bucket in MinIO config |
|
||||
| **Memory** | Not yet enforced | Future: embeddings storage limits |
|
||||
| **Temporal** | Workflow concurrency | Set in Temporal cluster config |
|
||||
|
||||
See [docs/QUOTAS.md](QUOTAS.md) for detailed limits.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Service Adapter Documentation](../tasks/8.1-serviceadapter-crd-and-informer.md)
|
||||
- [X-Service Routing](../tasks/8.2-x-service-dispatcher.md)
|
||||
- [Authentication & JWT Validation](../tasks/3.1-auth-sqs-jwt-validation.md)
|
||||
- [Testing Guide](../TESTING_GUIDE.md)
|
||||
- [Integration Tests](../INTEGRATION_TESTS.md)
|
||||
@@ -1,136 +0,0 @@
|
||||
# ADR-0001 — Retire Kong OSS in favour of a Go API gateway
|
||||
|
||||
Status: Accepted
|
||||
Date: 2026-08-19
|
||||
Deciders: rock
|
||||
|
||||
## Context
|
||||
|
||||
`api.riotpiao.com` is currently served by Kong OSS 3.4.1 (Helm, DB-less, namespace `api`,
|
||||
Argo wave 7), sitting behind ingress-nginx which owns TLS. Kong routes to the KServe
|
||||
model predictors in `llm-serving` via seven `ingressClassName: kong` Ingresses and six
|
||||
`KongPlugin` CRs.
|
||||
|
||||
Three separate capabilities were attempted on Kong OSS. All three failed, and each
|
||||
failure is already documented in-repo by the person who hit it:
|
||||
|
||||
**1. Body-based model dispatch is not expressible.**
|
||||
From `k8s/apps/api/llm-routes.yaml`:
|
||||
|
||||
> a single `/v1/chat/completions` endpoint that dispatches on the body's `model` field is
|
||||
> not expressible in Kong OSS (`ai-proxy-advanced`, which does multi-target model routing,
|
||||
> is Enterprise-only).
|
||||
|
||||
The workaround is a path-per-model surface (`/v1/reasoning/chat/completions`,
|
||||
`/v1/ornith/...`, `/v1/qwen/...`) with a `request-transformer` force-overwriting the body's
|
||||
`model` field. This is not OpenAI-standard, so every client needs bespoke configuration —
|
||||
visible today in `~/.pi/agent/models.json`, which carries three separate provider entries
|
||||
for what should be one endpoint.
|
||||
|
||||
**2. OIDC is Enterprise-only.**
|
||||
`k8s/apps/api/AUTH-PLAN.md` routes around the missing `openid-connect` plugin using the
|
||||
built-in `jwt` plugin, which requires pinning Authentik's RSA public key onto a
|
||||
KongConsumer. That plan lists its own consequence:
|
||||
|
||||
> Pinning `rsa_public_key`: Authentik key rotation would break it — document a rotation
|
||||
> runbook, or have the provision script re-export the cert PEM into the Kong credential on
|
||||
> each run.
|
||||
|
||||
A rotation runbook is a standing operational liability accepted only because the gateway
|
||||
cannot fetch JWKS itself.
|
||||
|
||||
**3. `key-auth` cannot read `Authorization: Bearer`.**
|
||||
From `k8s/apps/api/model-auth.yaml`:
|
||||
|
||||
> a raw `apikey: <key>` header succeeds (200), the same request with only
|
||||
> `Authorization: Bearer <key>` fails (401). No OpenAI-SDK-compatible client (pi included)
|
||||
> sends a raw apikey header or lets you customize the header name, so every such client was
|
||||
> hard-blocked.
|
||||
|
||||
Consequence: authentication on the model routes is **currently disabled**. Verified live
|
||||
2026-08-19 — `api.riotpiao.com/v1/reasoning/chat/completions` answers unauthenticated.
|
||||
|
||||
Separately, the intended surface has grown beyond LLM routing. The target is a
|
||||
capability-per-subdomain API over cluster services — `sqs.riotpiao.com` for queue
|
||||
operations, `workflow.riotpiao.com` for Temporal, `cluster.riotpiao.com` for atlas — each
|
||||
needing request shaping, per-caller budgets and streaming semantics that are application
|
||||
concerns, not gateway-plugin concerns.
|
||||
|
||||
## Decision
|
||||
|
||||
Retire Kong OSS entirely. Replace it with a purpose-built Go service,
|
||||
`homelab-frontend`, which owns north-south routing, authentication, and request shaping
|
||||
for every public capability on `*.riotpiao.com`.
|
||||
|
||||
ingress-nginx keeps the edge and TLS. It forwards to the gateway instead of `kong-proxy`.
|
||||
|
||||
Authentication is Authentik OIDC, validated by fetching JWKS from
|
||||
`https://authentik.riotpiao.com` at runtime.
|
||||
|
||||
## Options considered
|
||||
|
||||
**A. Stay on Kong OSS, accept the workarounds.**
|
||||
Keeps a battle-tested proxy and its Prometheus plugin. But the path-per-model surface stays
|
||||
non-standard, the RSA pinning runbook stays, and auth stays off until someone writes a
|
||||
`request-transformer` shim to copy Bearer into an `apikey` header. Every new capability
|
||||
(`sqs`, `workflow`) inherits the same constraints.
|
||||
|
||||
**B. Buy Kong Enterprise.**
|
||||
`ai-proxy-advanced` and `openid-connect` solve 1 and 2. Does not solve the genuinely
|
||||
application-level requirements at all — signed session cookies, per-session daily message
|
||||
budgets, a 6-of-8 GPU sequence-slot semaphore with a bounded queue, and
|
||||
disconnect-cancels-upstream are not gateway features in any tier. Cost for a homelab is not
|
||||
justifiable.
|
||||
|
||||
**C. Go gateway, Kong retained for LLM paths only.**
|
||||
Gradual migration, lower risk. But it means running two gateways indefinitely, splitting the
|
||||
routing table across Kong CRDs and Go code, and keeping the Kong Helm release and its CRDs.
|
||||
The split is the thing most likely to drift.
|
||||
|
||||
**D. Go gateway, Kong retired entirely.** — chosen
|
||||
One routing table, one auth implementation, one place to reason about timeouts. The logic
|
||||
being replaced is small: four `request-transformer` plugins that set a body field and
|
||||
rewrite a URI, one `request-termination` serving a static JSON model list, and one
|
||||
`prometheus` plugin. That is on the order of a hundred lines of Go, against roughly 480
|
||||
lines of YAML it retires.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Gained
|
||||
|
||||
- **Standard OpenAI surface.** One `POST /v1/chat/completions`, model selected from the
|
||||
request body. Any OpenAI SDK works unmodified. The three pi provider entries collapse to
|
||||
one.
|
||||
- **Working authentication.** Bearer tokens are read from the header, because it is our
|
||||
code. JWKS is fetched and cached with automatic rotation handling, so the AUTH-PLAN.md
|
||||
rotation runbook is deleted rather than written.
|
||||
- **Application-level policy becomes possible.** GPU slot semaphore, per-session budgets,
|
||||
disconnect propagation and SSE handling live where the state is.
|
||||
- **One timeout story.** Kong currently sets `read-timeout: 3600000` (1 hour) on chat
|
||||
routes, which silently defeats any shorter server-side cap. Retiring Kong removes the
|
||||
conflicting layer.
|
||||
- **~480 lines of gateway YAML deleted**, plus the Kong CRDs, the Helm release, and its
|
||||
`ServerSideApply` workaround for oversized CRD annotations.
|
||||
|
||||
### Lost / assumed
|
||||
|
||||
- **We now own proxy correctness.** Connection pooling, retries, timeout propagation,
|
||||
streaming passthrough, header hygiene, graceful shutdown. `net/http/httputil.ReverseProxy`
|
||||
covers most of it, but it is our bug surface now.
|
||||
- **Kong's Prometheus plugin goes away.** The gateway must expose equivalent metrics itself
|
||||
(bandwidth, latency, status codes, upstream health) or observability regresses.
|
||||
- **Migration touches live traffic.** pi depends on `api.riotpiao.com` today. Cutover must
|
||||
be reversible — see `docs/MIGRATION-kong.md`.
|
||||
- **`agent-pod/console` is a kong-class Ingress** exposing `/console` (WebSocket), `/run`
|
||||
and `/sessions`. It must migrate too, and it is currently unauthenticated and publicly
|
||||
routed while accepting free-form prompts into a shell-capable container. Putting it behind
|
||||
the gateway's Authentik auth is a security improvement, not just a port.
|
||||
|
||||
### Risks
|
||||
|
||||
- Enabling Authentik auth will break any client currently relying on the unauthenticated
|
||||
surface — including pi, until its `models.json` is updated. Auth must ship behind a flag
|
||||
and be enabled deliberately.
|
||||
- Kong's `request-termination` for `/v1/models` returns a **static** list that can drift
|
||||
from what the engines actually serve. Porting it verbatim ports the bug; the gateway
|
||||
should derive the list from configured upstreams instead.
|
||||
@@ -1,32 +0,0 @@
|
||||
# 0.1 — Module and entrypoint (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: GREEN
|
||||
|
||||
- [x] A single Go module builds one static binary with no cgo
|
||||
- [x] The binary reads its configuration at startup and serves HTTP on a configurable listen address
|
||||
- [x] `SIGTERM` starts a drain: the listener stops accepting new connections, in-flight requests run to completion, then the process exits `0`
|
||||
- [x] A request already in flight when `SIGTERM` arrives receives its full, uncorrupted response body
|
||||
- [x] A request arriving after `SIGTERM` is not accepted on a new connection
|
||||
- [x] The drain has a bounded deadline; exceeding it forces exit with a non-zero code and a logged reason
|
||||
- [x] The process holds no Kubernetes credentials and makes no API-server calls
|
||||
|
||||
The gateway sits behind ingress-nginx, which owns TLS. The gateway never terminates
|
||||
TLS and never listens on 443. Graceful drain matters because in-flight requests here
|
||||
are LLM generations that can legitimately run for many minutes -> killing them
|
||||
mid-stream loses work a caller cannot cheaply redo.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/server/... -run TestGracefulShutdown -race -v
|
||||
# expected: passes — a slow in-flight request completes with a full body after SIGTERM,
|
||||
# and a request issued post-SIGTERM is refused; process exit code is 0
|
||||
|
||||
CGO_ENABLED=0 go build ./... && go vet ./...
|
||||
# expected: both succeed
|
||||
```
|
||||
|
||||
`-race` is required, not optional. A server that starts a listener in one goroutine and
|
||||
exposes its address from another is the obvious shape here, and it is racy unless the
|
||||
shared state is guarded. A test that passes without `-race` proves nothing about it.
|
||||
@@ -1,29 +0,0 @@
|
||||
# 0.2 — Declarative route configuration (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: RED
|
||||
|
||||
- [x] Routes and upstreams are declared in YAML loaded from a file path at startup
|
||||
- [x] Each upstream declares: address, path rewrite, connect timeout, read timeout, write timeout, maximum request body size, and an auth-required flag
|
||||
- [x] Every one of those fields is explicit — no silent defaults for timeouts, body caps or auth
|
||||
- [x] A config missing any required field fails startup with a non-zero exit and a message naming the offending route and field
|
||||
- [x] A config with a malformed duration, an unparseable address, or a duplicate route key fails startup the same way
|
||||
- [x] A valid config round-trips: every declared route is present in the loaded route table
|
||||
- [x] Loading is startup-only — no API-server watch, no CRD, no Kubernetes client
|
||||
|
||||
Configuration lives in git and is mounted as a ConfigMap synced by Argo. It is
|
||||
deliberately not a CRD: a CRD would require the gateway to watch the API server,
|
||||
which needs RBAC and contradicts the invariant that the gateway holds no cluster
|
||||
credentials. It is also the exact indirection being retired with Kong, whose routing
|
||||
table was split across six `KongPlugin` CRs, seven Ingresses and a Helm values file.
|
||||
|
||||
A gateway that starts with a silently dropped route is worse than one that refuses to
|
||||
start.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/config/... -v
|
||||
# expected: passes — valid fixtures load with all routes present; each invalid fixture
|
||||
# returns an error naming the offending route and field, and none of them load partially
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# 0.3 — Health endpoints (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: GREEN
|
||||
Depends on: [0.2](0.2-route-configuration.md)
|
||||
|
||||
- [x] `GET /healthz` returns `200` whenever the process is alive
|
||||
- [x] `GET /healthz` contacts no upstream and performs no network I/O
|
||||
- [x] `GET /readyz` returns `200` only when configuration is valid and, if auth is enabled, JWKS has been fetched at least once
|
||||
- [x] `GET /readyz` returns a non-`2xx` status while configuration is invalid or JWKS has never been fetched
|
||||
- [x] Neither endpoint requires authentication, even when the auth flag is on
|
||||
- [x] Neither path is proxied to any upstream, and neither can be shadowed by a configured route
|
||||
|
||||
`/healthz` backs the liveness probe, so it must stay cheap and must not fail because
|
||||
an upstream is down — restarting the gateway does not fix a sick vLLM pod. `/readyz`
|
||||
backs the readiness probe and is allowed to fail, taking the pod out of the nginx
|
||||
endpoint pool until it can actually serve.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/server/... -run TestHealthEndpoints -v
|
||||
# expected: passes — /healthz is 200 with upstreams unreachable; /readyz is non-2xx
|
||||
# before first JWKS fetch and 200 after; both answer with no Authorization header
|
||||
```
|
||||
@@ -1,29 +0,0 @@
|
||||
# 0.4 — Local development harness (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: GREEN
|
||||
Depends on: [0.2](0.2-route-configuration.md)
|
||||
|
||||
- [ ] The whole gateway runs from a checkout with no cluster, no kubeconfig and no credentials of any kind
|
||||
- [ ] A committed local config points every upstream at stub servers started by the harness
|
||||
- [ ] Stubs can serve a fixed JSON body, an SSE token stream, a chunked response, and a slow response
|
||||
- [ ] A test can assert on the real HTTP response: status, headers and body
|
||||
- [ ] A test can assert that streamed chunks arrive incrementally, before the upstream has finished
|
||||
- [ ] A test can disconnect the client mid-response and assert on what the stub upstream observed
|
||||
- [ ] One documented command runs the harness end to end and exits non-zero on failure
|
||||
- [ ] Running the harness never contacts `*.riotpiao.com` or any cluster address
|
||||
|
||||
This is a hard requirement, not a convenience: it determines whether work can proceed
|
||||
unattended. Upstreams are configuration, so pointing them at local stubs is the entire
|
||||
mechanism. Every later phase's verification depends on this existing first.
|
||||
|
||||
"It compiles" and "it starts" are not verification. Asserting on an actual HTTP
|
||||
response is.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
env -u KUBECONFIG go test ./internal/testsupport/... ./internal/proxy/... -v
|
||||
# expected: passes with no kubeconfig and no network access beyond loopback —
|
||||
# includes an SSE test asserting incremental arrival and a mid-response disconnect test
|
||||
```
|
||||
@@ -1,26 +0,0 @@
|
||||
# 0.5 — Structured logging (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: GREEN
|
||||
Depends on: [0.1](0.1-module-and-entrypoint.md)
|
||||
|
||||
- [ ] Logs are emitted as structured records with a consistent field set, one record per line
|
||||
- [ ] Every request log carries at least: route, upstream, method, path, status, duration
|
||||
- [ ] Every rejected request is logged with an explicit machine-readable reason field
|
||||
- [ ] Request bodies are never logged, in whole or in part
|
||||
- [ ] `Authorization` header values, bearer tokens, API keys and JWKS material are never logged, not even truncated or hashed-with-prefix
|
||||
- [ ] Log level is configurable, and no level unlocks body or token logging
|
||||
- [ ] A test asserts a rejected request produces exactly one record containing the reason and containing no token substring
|
||||
|
||||
Rejections come from several layers — unknown model, body too large, auth failure,
|
||||
concurrency limit — and the reason field is what makes them countable later. The model
|
||||
API carries prompts that are user content and tokens that are credentials; neither
|
||||
belongs in a log line.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/logging/... ./internal/server/... -run 'TestLog' -v
|
||||
# expected: passes — captured log output for a rejected request contains the reason
|
||||
# field and does not contain the request body or the bearer token used
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# 0.6 — CI pipeline (GREEN)
|
||||
|
||||
Phase: 0 — Foundations
|
||||
Stage: GREEN
|
||||
Depends on: [0.4](0.4-local-dev-harness.md)
|
||||
|
||||
- [ ] CI builds the binary on every push and pull request
|
||||
- [ ] CI runs `go vet` over all packages and fails on any finding
|
||||
- [ ] CI runs the full test suite, including the local harness tests, with the race detector on
|
||||
- [ ] CI runs `govulncheck` and fails the job on any HIGH or CRITICAL severity finding
|
||||
- [ ] CI needs no cluster, no kubeconfig and no credentials to pass
|
||||
- [ ] A deliberately broken commit — failing test, vet finding, or known-vulnerable dependency — fails CI rather than passing silently
|
||||
- [ ] Job status is visible on the commit or pull request
|
||||
|
||||
CI is the outer loop for the same closed verification loop the harness gives locally.
|
||||
It must not depend on cluster access, or it stops running the moment the cluster is
|
||||
unavailable.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go vet ./... && go test -race ./... && govulncheck ./...
|
||||
# expected: all three exit 0 locally; pushing a branch with a failing test shows a
|
||||
# failed CI run on that commit
|
||||
```
|
||||
@@ -1,26 +0,0 @@
|
||||
# 1.1 — Reverse proxy to configured upstreams (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: GREEN
|
||||
Depends on: [0.2](0.2-route-configuration.md), [0.4](0.4-local-dev-harness.md)
|
||||
|
||||
- [ ] A request matching a configured route is proxied to that route's upstream address
|
||||
- [ ] The upstream's status code, response headers and body reach the client unmodified
|
||||
- [ ] The request method, query string and body reach the upstream unmodified
|
||||
- [ ] The route's configured path rewrite is applied to the upstream request path
|
||||
- [ ] Connections to upstreams are pooled and reused across requests — a second request to the same upstream does not open a new TCP connection
|
||||
- [ ] A request matching no configured route returns `404` and contacts no upstream
|
||||
- [ ] An unreachable upstream returns a `5xx` to the client and is logged with the upstream name
|
||||
|
||||
Connection reuse is not a micro-optimisation here: the chat upstreams hold long-lived
|
||||
streaming responses, and churning connections under that pattern wastes handshakes and
|
||||
file descriptors on both ends.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run 'TestProxy|TestConnectionReuse' -v
|
||||
# expected: passes — stub upstream sees the rewritten path and original body, client
|
||||
# sees the stub's exact status/headers/body, and the stub records one accepted
|
||||
# connection across two sequential requests
|
||||
```
|
||||
@@ -1,26 +0,0 @@
|
||||
# 1.2 — Streaming passthrough (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: RED
|
||||
Depends on: [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] An SSE response from an upstream reaches the client unbuffered: each `data:` event is readable by the client before the upstream has sent the next one
|
||||
- [ ] A chunked response reaches the client chunk by chunk, not accumulated and flushed at completion
|
||||
- [ ] Response headers reach the client before the first body byte, not after
|
||||
- [ ] `Content-Type: text/event-stream` and the upstream's `Cache-Control` and `Connection` semantics survive the proxy
|
||||
- [ ] No response body is written to memory or disk in full before forwarding
|
||||
- [ ] The terminating `data: [DONE]` sentinel and the final zero-length chunk pass through
|
||||
- [ ] A test asserts wall-clock ordering: the Nth event is observed at the client before the upstream emits the N+1th
|
||||
|
||||
The whole product is token streaming. If the gateway buffers, a caller waits minutes
|
||||
for a response that should have started in seconds, and the user-visible behaviour of
|
||||
the model API regresses versus Kong. nginx in front is already configured with
|
||||
`proxy-buffering: off`; the gateway must not reintroduce buffering behind it.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestSSEUnbuffered -v
|
||||
# expected: passes — client observes each of 5 stub-emitted SSE events with the
|
||||
# upstream still open, and total observed inter-event gaps match the stub's emit delays
|
||||
```
|
||||
@@ -1,25 +0,0 @@
|
||||
# 1.3 — Client disconnect propagation (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: RED
|
||||
Depends on: [1.2](1.2-streaming-passthrough.md)
|
||||
|
||||
- [ ] When a client closes the connection mid-response, the gateway cancels the upstream request immediately
|
||||
- [ ] The stub upstream observes its request context cancelled, not a request that runs to completion
|
||||
- [ ] Cancellation happens within a small bounded delay of the client close, not at the route's read timeout
|
||||
- [ ] The same holds for a non-streaming request abandoned before the upstream replies
|
||||
- [ ] The disconnect is logged with a reason distinguishing it from an upstream error
|
||||
- [ ] No goroutine or upstream connection is left alive after the disconnect — the test asserts this, not just the response
|
||||
|
||||
This is load-bearing. The `reasoning` upstream runs 2 replicas at `--max-num-seqs=4`,
|
||||
which is 8 concurrent sequence slots cluster-wide. An orphaned generation holds one of
|
||||
those 8 until it finishes on its own, which for a 32B model on a Volta GPU can be
|
||||
minutes. A handful of abandoned browser tabs can starve the entire cluster.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestClientDisconnectCancelsUpstream -race -v
|
||||
# expected: passes — stub upstream reports context cancellation within 1s of the client
|
||||
# closing mid-stream, and the post-test goroutine count returns to baseline
|
||||
```
|
||||
@@ -1,29 +0,0 @@
|
||||
# 1.4 — Per-route timeouts (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: GREEN
|
||||
Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] Connect, read and write timeouts are taken per route from configuration, never from a global default
|
||||
- [ ] Chat routes use connect `10s`, read `1h`, write `1h`
|
||||
- [ ] Embeddings and rerank routes use connect `10s`, read `10m`, write `10m`
|
||||
- [ ] An upstream that never accepts a connection fails at the configured connect timeout, not later
|
||||
- [ ] An upstream that accepts then stalls fails at the configured read timeout with a `5xx` and a logged reason
|
||||
- [ ] A stream still emitting tokens is never cut by the read timeout — the timeout applies to inactivity, not total duration
|
||||
- [ ] No code path shortens a configured proxy timeout to enforce an application-level cap
|
||||
|
||||
These are Kong's current values and they are deliberate. The 1-hour read timeout
|
||||
exists because a 32B model on a Volta GPU routinely exceeds 60 seconds per request.
|
||||
Any shorter application-level cap must be enforced by the gateway's own logic — a
|
||||
budget, a slot limit, an explicit max-generation-time — and never by shortening the
|
||||
proxy timeout, or long legitimate generations truncate mid-stream and callers see
|
||||
corrupted output rather than an error.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run 'TestConnectTimeout|TestReadTimeout|TestLongStreamNotTruncated' -v
|
||||
# expected: passes — stalled upstream errors at the configured read timeout, a stub
|
||||
# emitting one event every 200ms for longer than the timeout window is not cut off,
|
||||
# and a blackholed address fails at ~10s
|
||||
```
|
||||
@@ -1,27 +0,0 @@
|
||||
# 1.5 — Header hygiene (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: GREEN
|
||||
Depends on: [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] Hop-by-hop headers are stripped from both the upstream request and the client response
|
||||
- [ ] Headers named in a request's `Connection` header are also stripped, not just the fixed hop-by-hop list
|
||||
- [ ] `X-Forwarded-For` appends the immediate peer to the nginx-supplied value rather than replacing or fabricating it
|
||||
- [ ] `X-Forwarded-Proto` and `X-Forwarded-Host` are taken from the nginx-supplied values when present
|
||||
- [ ] Client-supplied `X-Forwarded-*` values are not trusted when the request did not arrive from the trusted ingress peer
|
||||
- [ ] End-to-end headers, including `Content-Type`, `Authorization` where the route requires it, and upstream response headers, pass through unchanged
|
||||
- [ ] A test asserts the exact header set the stub upstream receives
|
||||
|
||||
ingress-nginx owns TLS and the edge, so it is the only source of truth for the
|
||||
original scheme, host and client address. The gateway fabricating these would make
|
||||
every upstream's view of the caller wrong, and would let a client spoof its own
|
||||
source address by sending the header itself.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestHeaderHygiene -v
|
||||
# expected: passes — stub upstream sees no Connection/Keep-Alive/TE/Upgrade/
|
||||
# Proxy-Authorization headers, sees X-Forwarded-For ending in the nginx-supplied value
|
||||
# plus the peer, and a spoofed X-Forwarded-Proto from an untrusted peer is discarded
|
||||
```
|
||||
@@ -1,27 +0,0 @@
|
||||
# 1.6 — WebSocket upgrade (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: GREEN
|
||||
Depends on: [1.5](1.5-header-hygiene.md)
|
||||
|
||||
- [ ] A route may be configured to allow protocol upgrade
|
||||
- [ ] An upgrade request on such a route reaches the upstream with its `Upgrade` and `Connection` headers intact, despite hop-by-hop stripping
|
||||
- [ ] The upstream's `101 Switching Protocols` response reaches the client, and bytes then flow bidirectionally
|
||||
- [ ] Frames pass in both directions with no buffering delay
|
||||
- [ ] Client close propagates to the upstream and upstream close propagates to the client
|
||||
- [ ] An upgrade attempt on a route that does not allow it is rejected, not silently downgraded to a plain proxied request
|
||||
- [ ] Idle upgraded connections are not cut by the route's read timeout while frames are still flowing
|
||||
|
||||
`agent-pod/console` serves a WebSocket and is one of the seven Kong-class Ingresses
|
||||
being migrated. It is currently publicly routed and unauthenticated into a
|
||||
shell-capable container, so it must work through the gateway before it can be put
|
||||
behind gateway auth. Header hygiene and upgrade support interact directly: `Upgrade`
|
||||
and `Connection` are hop-by-hop, and a naive strip breaks the handshake.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestWebSocketUpgrade -v
|
||||
# expected: passes — client receives 101 from the stub, an echoed frame round-trips in
|
||||
# both directions, and closing the client causes the stub to observe a close
|
||||
```
|
||||
@@ -1,26 +0,0 @@
|
||||
# 1.7 — Per-route body size caps (GREEN)
|
||||
|
||||
Phase: 1 — Proxy core
|
||||
Stage: RED
|
||||
Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
|
||||
|
||||
- [ ] Each route enforces its own configured maximum request body size
|
||||
- [ ] A body over the cap is rejected with `413` and a body the upstream never sees
|
||||
- [ ] Rejection happens while reading, not after buffering the whole body into memory
|
||||
- [ ] A request with a lying or absent `Content-Length` is still capped by bytes actually read
|
||||
- [ ] A body at exactly the cap is accepted and proxied intact
|
||||
- [ ] The rejection is logged with a reason distinguishing it from other rejections
|
||||
- [ ] No global default cap silently applies to a route that failed to declare one — that is a config error, per 0.2
|
||||
|
||||
nginx in front is configured with `proxy-body-size: 0`, meaning it enforces no limit
|
||||
at all, so the gateway is the only place a cap exists. Embedding and rerank callers
|
||||
can send large batches legitimately, which is why the cap is per route rather than
|
||||
one number for the whole surface.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
go test ./internal/proxy/... -run TestBodySizeCap -v
|
||||
# expected: passes — a body one byte over the route cap returns 413 and the stub
|
||||
# upstream records zero requests; a body exactly at the cap returns the stub's 200
|
||||
```
|
||||
@@ -1,64 +0,0 @@
|
||||
# 3.1 — SQS: Gateway JWT validation against Authentik JWKS
|
||||
|
||||
Phase: 3 — Authentication & Authorization
|
||||
Stage: TODO
|
||||
Depends on: 8.2 (X-Service dispatcher), 8.10 (gate)
|
||||
|
||||
## Context
|
||||
|
||||
SQS (kmsvc management-service) has placeholder env vars for JWT validation:
|
||||
- `KMSVC_AUTHENTIK_ISSUER_URL`
|
||||
- `KMSVC_AUTHENTIK_AUDIENCE`
|
||||
|
||||
But **kmsvc code is unverified** — we don't know if it actually validates JWTs.
|
||||
**Phase 8.2 decision**: Gateway validates SQS JWTs at ingress (not pushing to kmsvc).
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] Gateway extracts `Authorization: Bearer <token>` from SQS requests
|
||||
- [ ] Validates JWT signature against Authentik JWKS endpoint:
|
||||
- Issuer: `https://authentik.riotpiao.com/application/o/sqs/`
|
||||
- JWKS: `https://authentik.riotpiao.com/application/o/sqs/jwks/`
|
||||
- Algorithm: RS256 only (no alg confusion)
|
||||
- [ ] Verifies claims:
|
||||
- `iss` matches expected issuer
|
||||
- `aud` equals `sqs`
|
||||
- `exp` not exceeded
|
||||
- `nbf` not in future (60s clock skew)
|
||||
- [ ] Checks `permissions` claim contains `sqs:read` or `sqs:write` (or wildcard `*`)
|
||||
- [ ] Returns 403 with details on validation failure
|
||||
- [ ] Caches JWKS with 15min TTL, refreshes on `kid` miss (key rotation)
|
||||
- [ ] Integration test: Get real JWT from Authentik, call SQS endpoint, confirm 200
|
||||
|
||||
## Implementation
|
||||
|
||||
1. Add JWT validator to `internal/auth/jwt.go` (was removed, restore as Phase 3 work)
|
||||
2. Wire into `internal/serviceadapter/router.go` Dispatcher for SQS only
|
||||
3. Update `k8s/configmap.yaml` SQS `auth.required: true` + capability check
|
||||
4. Add test to `internal/serviceadapter/real_integration_test.go`
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Get JWT from Authentik
|
||||
export JWT=$(curl -s -X POST https://authentik.riotpiao.com/application/o/token/ \
|
||||
-d "grant_type=client_credentials&client_id=<id>&client_secret=<secret>&scope=openid" \
|
||||
| jq -r '.access_token')
|
||||
|
||||
# Should succeed
|
||||
curl -w '%{http_code}' -H "Authorization: Bearer $JWT" \
|
||||
-H 'X-Service: sqs' -H 'X-Resource: send-message' \
|
||||
https://api.riotpiao.com/
|
||||
|
||||
# Should 403
|
||||
curl -w '%{http_code}' -H 'X-Service: sqs' -H 'X-Resource: send-message' \
|
||||
https://api.riotpiao.com/
|
||||
# expected: 403
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- **Not Phase 8.2**: Phase 8 was about routing & dispatcher architecture
|
||||
- **Phase 3 scope**: Full auth integration & JWT validation
|
||||
- **MinIO/Temporal**: Have native JWT support, tested in Phase 3 separately
|
||||
- **Memory/IAM**: Services validate own JWTs (dumb pipe)
|
||||
@@ -1,56 +0,0 @@
|
||||
# 3.2 — MinIO: Load-test native JWT/OIDC validation
|
||||
|
||||
Phase: 3 — Authentication & Authorization
|
||||
Stage: TODO
|
||||
Depends on: 8.2 (X-Service dispatcher), 8.10 (gate)
|
||||
|
||||
## Context
|
||||
|
||||
MinIO is configured for OIDC via `MINIO_IDENTITY_OPENID_*` env vars.
|
||||
Per homelab/project-usage/jwt-auth-rollout.md: "likely yes, **not yet load-tested**".
|
||||
|
||||
**Phase 8.2 decision**: Gateway acts as dumb pipe, MinIO validates JWTs itself.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] MinIO validates JWT tokens from Authentik
|
||||
- Checks JWKS against `MINIO_IDENTITY_OPENID_CONFIG_URL`
|
||||
- Verifies `aud` claim (check current config)
|
||||
- Maps claims to MinIO policies
|
||||
- [ ] Policy mapping works:
|
||||
- Authentik group `homelab-admins` → MinIO `consoleAdmin` policy
|
||||
- Other groups → appropriate S3 bucket access
|
||||
- [ ] Integration test: Get JWT from Authentik, call S3 endpoint, confirm auth works
|
||||
- [ ] Load test: 100+ requests/sec with valid JWTs succeed
|
||||
- [ ] Performance: JWT validation doesn't add >50ms latency per request
|
||||
|
||||
## Implementation
|
||||
|
||||
1. Verify MinIO OIDC config in k8s/infra (not this repo)
|
||||
2. Create JWT token with homelab-admins group
|
||||
3. Test S3 operations (ListBuckets, GetObject, PutObject)
|
||||
4. Add load test to integration suite
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
export JWT=$(curl -s -X POST https://authentik.riotpiao.com/application/o/token/ \
|
||||
-d "grant_type=client_credentials&client_id=<id>&client_secret=<secret>&scope=openid" \
|
||||
| jq -r '.access_token')
|
||||
|
||||
# List buckets
|
||||
aws s3 ls --endpoint-url https://api.riotpiao.com/ \
|
||||
--header "Authorization: Bearer $JWT"
|
||||
|
||||
# Get object
|
||||
curl -H "Authorization: Bearer $JWT" \
|
||||
https://api.riotpiao.com/ \
|
||||
-H 'X-Service: s3' -H 'X-Resource: list-objects'
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- **Not Phase 8.2**: Phase 8 was routing, Phase 3 is auth verification
|
||||
- **MinIO owner responsibility**: Verify config in k8s/infra cluster
|
||||
- **Gateway responsibility**: Pass JWT through unchanged (dumb pipe)
|
||||
- **Test coverage**: Real JWT token, real S3 operations
|
||||
@@ -1,65 +0,0 @@
|
||||
# 3.3 — Temporal: Configure native JWT validation via jwtKeyProvider
|
||||
|
||||
Phase: 3 — Authentication & Authorization
|
||||
Stage: TODO
|
||||
Depends on: 8.2 (X-Service dispatcher), 8.10 (gate)
|
||||
|
||||
## Context
|
||||
|
||||
Temporal Helm chart supports native JWT authorization:
|
||||
- `server.config.authorization.jwtKeyProvider`
|
||||
- `claimMapper` for custom claim handling
|
||||
|
||||
Per homelab/project-usage/jwt-auth-rollout.md: "native JWT authorization support".
|
||||
But currently **not configured** — unresolved design question on external access.
|
||||
|
||||
**Phase 8.2 decision**: Gateway detects gRPC (returns 501 not-implemented).
|
||||
Phase 9 will add gRPC proxy. Phase 3 can configure JWT validation in Temporal.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] Configure `server.config.authorization.jwtKeyProvider` in Temporal Helm values
|
||||
- Point at Authentik JWKS: `https://authentik.riotpiao.com/application/o/workflow/jwks/`
|
||||
- [ ] Configure `claimMapper` to translate Authentik claims:
|
||||
- `permissions` claim → Temporal permissions
|
||||
- OR `groups` claim → Temporal role mappings
|
||||
- [ ] Test: In-cluster worker with JWT can connect to Temporal frontend
|
||||
- [ ] Test: Unauthenticated client gets 401
|
||||
- [ ] Verify: No impact on existing workers/clients (backward compat)
|
||||
|
||||
## Implementation
|
||||
|
||||
1. Update k8s/infra temporal-values.yaml:
|
||||
```yaml
|
||||
server:
|
||||
config:
|
||||
authorization:
|
||||
jwtKeyProvider:
|
||||
keySourceURIs:
|
||||
- "https://authentik.riotpiao.com/application/o/workflow/jwks/"
|
||||
claimMapper: |
|
||||
# Custom claims mapping (TBD)
|
||||
```
|
||||
2. Deploy & test
|
||||
3. Add integration test (requires gRPC client, Phase 9)
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# In-cluster test (Pod with JWT)
|
||||
kubectl -n temporal run --rm -it temporal-test \
|
||||
--image=temporalioservices/temporal-server:latest \
|
||||
-- tctl --address temporal-frontend:7233 namespace list
|
||||
|
||||
# External test (Phase 9, requires gRPC proxy)
|
||||
# grpcurl -H "Authorization: Bearer $JWT" \
|
||||
# temporal-frontend.cluster.local:7233 \
|
||||
# temporal.api.workflowservice.v1.WorkflowService/ListNamespaces
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- **Not Phase 8.2**: Routing is done, auth config is Phase 3+
|
||||
- **Temporal owner responsibility**: Configure JWT in temporal-values.yaml (k8s/infra)
|
||||
- **Gateway responsibility**: Pass gRPC through (Phase 9: grpcproxy)
|
||||
- **Open question**: External access to Temporal frontend (TBD)
|
||||
@@ -1,44 +0,0 @@
|
||||
# 4.1 — GPU slot semaphore for `reasoning` (GREEN)
|
||||
|
||||
Phase: 4 — Limits and budgets
|
||||
Stage: GREEN
|
||||
Depends on: [4.3](4.3-problem-json-errors.md)
|
||||
|
||||
The `reasoning` upstream is 2 replicas x `--max-num-seqs=4` = 8 concurrent sequence
|
||||
slots cluster-wide. That 8 is a vLLM concurrency setting, not a GPU count — worker-1
|
||||
has 4 physical GPUs and the two numbers are unrelated. Exceeding 8 does not fail
|
||||
fast; it silently queues inside vLLM where the gateway has no visibility and no
|
||||
ability to cancel.
|
||||
|
||||
The gateway serves `/v1/*` (OpenAI-compatible) over the same predictors. The
|
||||
Anthropic `/llm/*` dialect was dropped (see `tasks/INDEX.md` Phase 2) — this
|
||||
controller only ever needs to know about `/v1/*`.
|
||||
|
||||
- [ ] Concurrent in-flight requests to `reasoning` are capped at a configured limit strictly below 8, leaving operator headroom
|
||||
- [ ] The cap, the queue depth, and the queue wait timeout are all explicit in configuration with no silent defaults
|
||||
- [ ] Requests over the cap wait in a bounded queue rather than being rejected immediately
|
||||
- [ ] Once the queue is full, further requests are rejected with a retryable status and a `Retry-After`, as an `application/problem+json` document
|
||||
- [ ] A slot is released when the response completes, when the upstream errors, and when the client disconnects mid-stream — no path leaks a slot
|
||||
- [ ] A client that disconnects while still queued never reaches the upstream and never consumes a slot
|
||||
- [ ] The cap applies only to `reasoning`; other upstreams are unaffected and are not blocked behind its queue
|
||||
- [ ] The controller is keyed by UPSTREAM, never by route or path prefix — `reasoning-predictor.llm-serving:80` has exactly one cap and one queue
|
||||
- [ ] Slot occupancy and queue depth are reported per upstream
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# Fire 40 concurrent chat requests at a stub reasoning upstream that holds each for 2s,
|
||||
# with cap=6 and queue=8 configured.
|
||||
seq 40 | xargs -P40 -I{} curl -s -o /dev/null -w '%{http_code}\n' \
|
||||
-X POST localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' | sort | uniq -c
|
||||
# expected: a mix of 200 and 503; zero 500s
|
||||
|
||||
# expected: the stub logged at most 6 simultaneous in-flight requests, never 7
|
||||
grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
|
||||
# expected: 0
|
||||
|
||||
curl -s -D - -o /dev/null -X POST localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' | grep -i retry-after
|
||||
# expected: Retry-After present on the rejected request
|
||||
```
|
||||
@@ -1,32 +0,0 @@
|
||||
# 4.2 — Per-caller request budgets (GREEN)
|
||||
|
||||
Phase: 4 — Limits and budgets
|
||||
Stage: GREEN
|
||||
Depends on: [4.3](4.3-problem-json-errors.md)
|
||||
|
||||
No `rate-limiting` plugin exists anywhere in the cluster today. This is net-new work,
|
||||
not a migration — there is no prior behaviour to preserve.
|
||||
|
||||
- [ ] An identified caller gets a bounded number of requests per configured time window
|
||||
- [ ] Caller identity comes from the authenticated token's subject when auth is on, and from a documented fallback attribute when auth is off
|
||||
- [ ] Budget size and window length are explicit in configuration, per caller class, with no silent defaults
|
||||
- [ ] Exceeding the budget is rejected with a retryable status, an `application/problem+json` body, and a `Retry-After` naming when the window resets
|
||||
- [ ] Remaining budget and reset time are observable to the caller on allowed requests, not only on rejections
|
||||
- [ ] Budgets are enforced independently of the `reasoning` concurrency cap — a caller under budget can still be queued or rejected for slot pressure, and vice versa
|
||||
- [ ] Two distinct callers do not consume each other's budget
|
||||
- [ ] Budget state is per-replica-safe: the documented behaviour with 2 replicas is stated, not accidental
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# budget=5 per 60s window for the test caller
|
||||
for i in $(seq 1 7); do
|
||||
curl -s -o /dev/null -w '%{http_code} ' -H 'authorization: Bearer test-caller-a' \
|
||||
localhost:8080/v1/models
|
||||
done; echo
|
||||
# expected: 200 200 200 200 200 429 429
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -H 'authorization: Bearer test-caller-b' \
|
||||
localhost:8080/v1/models
|
||||
# expected: 200 — caller B has its own budget
|
||||
```
|
||||
@@ -1,31 +0,0 @@
|
||||
# 4.3 — RFC 9457 problem+json rejections (GREEN)
|
||||
|
||||
Phase: 4 — Limits and budgets
|
||||
Stage: RED
|
||||
|
||||
Every rejection the gateway generates itself must be a machine-readable problem
|
||||
document. Upstream responses are passed through untouched — this covers only errors
|
||||
the gateway originates.
|
||||
|
||||
- [ ] Every gateway-originated rejection responds with `Content-Type: application/problem+json`
|
||||
- [ ] The body carries at minimum `type`, `title`, `status`, and `detail`, and `status` equals the HTTP status line
|
||||
- [ ] `type` is a stable, distinct URI per rejection reason, so a client can branch on it without parsing prose
|
||||
- [ ] `detail` is human-useful and names the offending input where one exists
|
||||
- [ ] `Retry-After` is set whenever a retry time is knowable — queue full, budget exhausted, upstream saturated
|
||||
- [ ] `Retry-After` is absent when no retry will help — unknown model, malformed body, oversized body
|
||||
- [ ] No token, credential, header value, or request body content appears in any field
|
||||
- [ ] A rejection is emitted as a structured log line carrying the same reason identifier used in `type`
|
||||
|
||||
Tests come first and must fail because the shape does not exist yet, not because a
|
||||
route is missing.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -D - -o /tmp/p.json -X POST localhost:8080/v1/chat/completions \
|
||||
-H 'content-type: application/json' -d '{"model":"nope"}' | grep -i content-type
|
||||
# expected: application/problem+json
|
||||
|
||||
python3 -c "import json;d=json.load(open('/tmp/p.json'));assert{'type','title','status','detail'}<=d.keys();assert d['status']==400;print('ok')"
|
||||
# expected: ok
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# 5.1 — Prometheus parity with the retiring Kong plugin (GREEN)
|
||||
|
||||
Phase: 5 — Observability
|
||||
Stage: RED
|
||||
|
||||
Kong runs a cluster-wide `prometheus` plugin today. It is deleted at teardown. If the
|
||||
gateway does not carry equivalent signal, observability REGRESSES at cutover and
|
||||
nobody notices until an incident.
|
||||
|
||||
- [ ] A Prometheus text-format endpoint is exposed and scrapeable without authentication from inside the cluster
|
||||
- [ ] Request rate is observable, labelled by route and by upstream
|
||||
- [ ] Request latency is observable as a distribution, not a mean, labelled by route and upstream
|
||||
- [ ] Response status is observable, labelled by route, upstream, and status class
|
||||
- [ ] Bandwidth in both directions is observable per route and upstream
|
||||
- [ ] Upstream health is observable — whether each configured upstream is currently reachable and answering
|
||||
- [ ] Label values are drawn from the configured route and upstream names, never from raw request paths or user input, so cardinality cannot be driven by a caller
|
||||
- [ ] Streaming responses record their full byte count and full duration, not the time to first byte
|
||||
- [ ] The metrics endpoint is not reachable through the public route surface
|
||||
|
||||
Write the assertions against the metric families first; they must fail because the
|
||||
families are absent, not because the endpoint 404s.
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s localhost:8080/metrics | grep -E '^# TYPE' | awk '{print $3}' | sort -u
|
||||
# expected: includes counter and histogram families covering requests, duration, bytes, upstream health
|
||||
|
||||
curl -s -X POST localhost:8080/v1/chat/completions -H 'content-type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[]}' >/dev/null
|
||||
curl -s localhost:8080/metrics | grep 'upstream="reasoning-predictor"' | head
|
||||
# expected: request, duration and byte samples all carry route and upstream labels
|
||||
```
|
||||
@@ -1,33 +0,0 @@
|
||||
# 5.2 — Gateway-specific metrics Kong could not provide (GREEN)
|
||||
|
||||
Phase: 5 — Observability
|
||||
Stage: GREEN
|
||||
Depends on: [5.1](5.1-prometheus-parity.md)
|
||||
|
||||
These are the signals that justify replacing Kong. Without them the concurrency cap
|
||||
and budget layers are unfalsifiable — you cannot tell a saturated GPU from a broken
|
||||
gateway.
|
||||
|
||||
- [ ] In-flight request count is observable per upstream, and returns to zero when traffic stops
|
||||
- [ ] Queue depth for the `reasoning` concurrency queue is observable
|
||||
- [ ] Occupancy of the `reasoning` slot cap is observable — how many of the configured slots are held right now
|
||||
- [ ] The configured slot cap itself is observable, so occupancy can be read as a ratio without hardcoding the limit in a dashboard
|
||||
- [ ] Rejections are counted and broken down by reason: queue full, budget exhausted, body too large, unknown model, auth failure
|
||||
- [ ] Rejection reason label values match the stable reason identifiers used in the problem+json `type` field, so metrics and logs join cleanly
|
||||
- [ ] Time spent waiting in the queue is observable as a distribution, separately from upstream latency
|
||||
- [ ] In-flight and occupancy gauges are correct after a client disconnects mid-stream — no permanent drift upward
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# Hold 3 long requests open against a stub reasoning upstream, then scrape.
|
||||
curl -s localhost:8080/metrics | grep -E 'inflight|queue_depth|slots'
|
||||
# expected: in-flight for the reasoning upstream reads 3, queue depth reads 0, slot cap is exported
|
||||
|
||||
# Kill the clients mid-stream, wait, scrape again.
|
||||
curl -s localhost:8080/metrics | grep -E 'inflight'
|
||||
# expected: back to 0 — disconnects released their slots
|
||||
|
||||
curl -s localhost:8080/metrics | grep 'reason='
|
||||
# expected: rejection counters split by reason, matching the problem+json type identifiers
|
||||
```
|
||||
@@ -1,30 +0,0 @@
|
||||
# 5.3 — ServiceMonitor for gateway scraping (GREEN)
|
||||
|
||||
Phase: 5 — Observability
|
||||
Stage: GREEN
|
||||
Depends on: [5.2](5.2-gateway-metrics.md)
|
||||
|
||||
Exposing metrics is not the same as having them collected. Kong's plugin was picked
|
||||
up cluster-wide; the gateway must be explicitly registered or the dashboards go
|
||||
blank at cutover.
|
||||
|
||||
- [ ] A ServiceMonitor selects the gateway Service and is committed to git, applied by Argo — never `kubectl apply`
|
||||
- [ ] It carries whatever label the cluster's Prometheus uses to select ServiceMonitors, verified against the running Prometheus rather than assumed
|
||||
- [ ] The scraped port is a named port on the gateway Service, matched by name not number
|
||||
- [ ] Scrape interval and timeout are explicit
|
||||
- [ ] The gateway appears as an `up == 1` target in Prometheus, in the expected namespace
|
||||
- [ ] Metric labels identify the gateway pod and namespace, so 2 replicas are distinguishable
|
||||
- [ ] Scraping works while the gateway is still unexposed to public traffic — this lands before cutover, not after
|
||||
- [ ] Note: `prometheus-operated.monitoring` is headless with ClusterIP `None`, so any egress policy that touches it needs pod selectors, not a ClusterIP
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n monitoring get servicemonitor -l release --show-labels | grep -i frontend
|
||||
# expected: the gateway ServiceMonitor exists and carries the selector label Prometheus uses
|
||||
|
||||
# Query Prometheus for the gateway target
|
||||
curl -s 'http://localhost:9090/api/v1/query?query=up{job=~".*homelab-frontend.*"}' \
|
||||
| python3 -c "import json,sys;r=json.load(sys.stdin)['data']['result'];print(len(r),[x['value'][1] for x in r])"
|
||||
# expected: 2 targets, both reporting "1"
|
||||
```
|
||||
@@ -1,36 +0,0 @@
|
||||
# 7.1 — `/cluster/*` proxies to atlas (GREEN)
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
atlas lives in a separate repo, `riotpiao-backend`. It keeps its own informers and its
|
||||
own RBAC. The gateway proxies to it and holds no Kubernetes credentials of its own —
|
||||
this is G2, and it is the whole reason cluster-read capability lives behind atlas
|
||||
rather than in the edge process.
|
||||
|
||||
- [ ] `/cluster/*` on `api.riotpiao.com` proxies to the atlas Service
|
||||
- [ ] The gateway gains no ServiceAccount token, no kubeconfig and no RBAC as part of this. Any authorization decision about cluster data is atlas's, not the gateway's (G2)
|
||||
- [ ] Path rewriting between the `/cluster` prefix and atlas's own paths is explicit in configuration
|
||||
- [ ] Connect, read and write timeouts and a body cap are explicit for this route, with no silent defaults (G6)
|
||||
- [ ] The route requires authentication, and the token is checked for cluster capability — a token minted for queue access must not read cluster state
|
||||
- [ ] The NetworkPolicy is extended to allow egress to atlas and nothing more
|
||||
- [ ] `/v1/*` behaviour is byte-identical before and after this route is added — the prefixes cannot collide
|
||||
- [ ] Metrics and rejection counters cover this route with its own route label
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/cluster/healthz
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -H "authorization: Bearer $CLUSTER_TOKEN" https://api.riotpiao.com/cluster/healthz
|
||||
# expected: atlas's own response body, proxied unmodified
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -H "authorization: Bearer $QUEUE_ONLY_TOKEN" \
|
||||
https://api.riotpiao.com/cluster/healthz
|
||||
# expected: 403 — a queue token does not grant cluster capability
|
||||
|
||||
kubectl -n api get pod -l app=homelab-frontend -o jsonpath='{.items[0].spec.serviceAccountName}{"\n"}'
|
||||
# expected: a ServiceAccount with no RBAC bindings; the gateway still holds no cluster credentials
|
||||
```
|
||||
@@ -1,35 +0,0 @@
|
||||
# 7.2 — `/sqs/*` to kmsvc management-service and Kafka (GREEN)
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
`management-service` already exposes gRPC at `kmsvc.riotpiao.com`. This prefix is a
|
||||
NEW surface, not a replacement — that gRPC endpoint stays exactly as it is and nothing
|
||||
here migrates off it.
|
||||
|
||||
- [ ] `/sqs/*` on `api.riotpiao.com` proxies to kmsvc `management-service` and the Kafka/Strimzi surfaces in the `sqs` namespace
|
||||
- [ ] `kmsvc.riotpiao.com` is unchanged and still serving after this lands
|
||||
- [ ] Which sub-paths map to which upstream is explicit in configuration; there is no catch-all fallback
|
||||
- [ ] Timeouts and body caps are explicit per sub-route, with no silent defaults (G6)
|
||||
- [ ] The route requires authentication and the token is checked for queue capability
|
||||
- [ ] The NetworkPolicy is extended to reach only the named `sqs` upstreams
|
||||
- [ ] `kmsvc-redis-master.sqs:6379` has NO authentication — `ALLOW_EMPTY_PASSWORD=yes`, TLS off. Any workload with network reach has full unauthenticated read/write. It is not proxied, and the NetworkPolicy must not grant the gateway egress to it
|
||||
- [ ] `/v1/*` behaviour is unchanged before and after
|
||||
- [ ] Metrics and rejection counters cover this route with its own route label
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/sqs/healthz
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -H "authorization: Bearer $QUEUE_TOKEN" https://api.riotpiao.com/sqs/healthz
|
||||
# expected: management-service's own response, proxied unmodified
|
||||
|
||||
kubectl -n api get networkpolicy -o yaml | grep -c 6379
|
||||
# expected: 0 — no egress path from the gateway to unauthenticated Redis
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://kmsvc.riotpiao.com
|
||||
# expected: unchanged from before this task — the existing gRPC surface is untouched
|
||||
```
|
||||
@@ -1,34 +0,0 @@
|
||||
# 7.3 — `/workflow/*` to Temporal (GREEN)
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
Temporal runs in the `temporal` namespace. Temporal namespace registration is
|
||||
automatic via queue-operator and is NEVER done manually — this route must not create,
|
||||
register or mutate Temporal namespaces, only proxy to what queue-operator has already
|
||||
provisioned.
|
||||
|
||||
- [ ] `/workflow/*` on `api.riotpiao.com` proxies to the Temporal Service in the `temporal` namespace
|
||||
- [ ] Nothing in this route registers a Temporal namespace. Registration stays with queue-operator
|
||||
- [ ] Path rewriting between the `/workflow` prefix and Temporal's own paths is explicit in configuration
|
||||
- [ ] Timeouts and body caps are explicit, with no silent defaults (G6). Long-poll semantics are accounted for rather than truncated by a short read timeout
|
||||
- [ ] The route requires authentication and the token is checked for workflow capability — a GPU token must not drive workflows
|
||||
- [ ] The NetworkPolicy is extended to reach only Temporal
|
||||
- [ ] Streaming or long-poll responses pass through unbuffered, and a client disconnect cancels the upstream call rather than orphaning it (G4)
|
||||
- [ ] `/v1/*` behaviour is unchanged before and after
|
||||
- [ ] Metrics and rejection counters cover this route with its own route label
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/workflow/health
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -H "authorization: Bearer $WORKFLOW_TOKEN" https://api.riotpiao.com/workflow/health
|
||||
# expected: Temporal's own response, proxied unmodified
|
||||
|
||||
kubectl -n temporal exec svc/temporal-admintools -- tctl --ad temporal-frontend:7233 namespace list | sort > /tmp/ns.after
|
||||
diff /tmp/ns.before /tmp/ns.after
|
||||
# expected: no diff — the gateway registered nothing; queue-operator remains the only registrar
|
||||
```
|
||||
@@ -1,41 +0,0 @@
|
||||
# 7.4 — `/db/*` read surfaces (GREEN)
|
||||
|
||||
Phase: 7 — Additional capability prefixes
|
||||
Stage: GREEN
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
The most dangerous prefix in this phase. G2 says the gateway holds no cluster
|
||||
credentials — so it cannot be the thing that authenticates to CloudNativePG, MinIO or
|
||||
Prometheus on a caller's behalf. If designing this route makes you want to give the
|
||||
gateway a secret, the design is wrong: put the credential-holding logic in a service
|
||||
behind the gateway and proxy to that.
|
||||
|
||||
- [ ] `/db/*` on `api.riotpiao.com` exposes read surfaces backed by CloudNativePG, MinIO and monitoring
|
||||
- [ ] The gateway holds no database password, no MinIO access key and no cluster credential of any kind (G2). Credentials, if any are needed, live in the service being proxied to
|
||||
- [ ] Exposed operations are read-only. There is no write, no delete and no schema-changing path on this prefix
|
||||
- [ ] Which sub-paths reach which upstream is enumerated explicitly in configuration. No catch-all, no pass-through of arbitrary query text
|
||||
- [ ] Timeouts and body caps are explicit per sub-route, with no silent defaults (G6)
|
||||
- [ ] The route requires authentication and the token is checked for a distinct read capability
|
||||
- [ ] Result sets are paginated with a bounded page size; an unbounded read cannot be requested
|
||||
- [ ] The NetworkPolicy is extended only to the specific upstreams reached. Note `prometheus-operated.monitoring` is headless with ClusterIP `None`, so it needs a pod selector, not a ClusterIP
|
||||
- [ ] `/v1/*` behaviour is unchanged before and after
|
||||
- [ ] Metrics and rejection counters cover this route with its own route label
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/db/healthz
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -X DELETE -H "authorization: Bearer $DB_READ_TOKEN" \
|
||||
https://api.riotpiao.com/db/anything
|
||||
# expected: 405 or 404 — no mutating method is routable on this prefix
|
||||
|
||||
kubectl -n api get pod -l app=homelab-frontend -o jsonpath='{range .items[0].spec.containers[0].env[*]}{.name}{"\n"}{end}' \
|
||||
| grep -Ei 'password|secret|access_key'
|
||||
# expected: no output — the gateway carries no database or object-store credentials
|
||||
|
||||
curl -s -H "authorization: Bearer $DB_READ_TOKEN" 'https://api.riotpiao.com/db/...?limit=100000' \
|
||||
| python3 -c "import json,sys;d=json.load(sys.stdin);print(len(d['items']))"
|
||||
# expected: capped at the configured maximum page size, not 100000
|
||||
```
|
||||
@@ -1,50 +0,0 @@
|
||||
# 8.1 — `ServiceAdapter` CRD, informer, RBAC (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: Phase 6 (cutover) — done, confirmed live in-cluster
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §1.
|
||||
|
||||
**G2 note — read before objecting.** G2 says "the gateway holds no Kubernetes
|
||||
credentials, config comes from git not a CRD." This task deliberately supersedes G2
|
||||
for this one narrow purpose; the acknowledged-supersession rationale is written out
|
||||
in API_ROUTING_HYBRID_DESIGN.md's Context section (top of that doc) and in
|
||||
`INDEX.md`'s G2 line. Do not treat this as a task-file error — the supersession is
|
||||
intentional and pre-approved, scoped to exactly the read-only Role this task adds.
|
||||
|
||||
- [ ] `apis/gateway/v1/serviceadapter_types.go` defines `ServiceAdapter` matching
|
||||
the design doc's example CRs (§1, §6): `spec.serviceName`, `spec.upstream.{url,timeoutSeconds}`,
|
||||
`spec.auth.{required,capability}`, `spec.retryable`, `spec.resources[].name`,
|
||||
`spec.resources[].methods[].{verb,upstreamPath,requestSchema,responseSchema,auth}`
|
||||
- [ ] `controller-gen` generates the CRD YAML from those types into `k8s/crd-serviceadapter.yaml`, added to `k8s/kustomization.yaml`'s `resources:`
|
||||
- [ ] CRD is namespace-scoped, group `gateway.riotpiao.com/v1`, kind `ServiceAdapter` — not cluster-scoped
|
||||
- [ ] `k8s/rbac.yaml` gains a namespace-scoped `Role`/`RoleBinding` (`get`, `list`, `watch` only, no `status`/`finalizers` verbs) for the existing `api-gateway` ServiceAccount, exactly as specced in §1
|
||||
- [ ] `internal/serviceadapter/registry.go`: `client-go` `SharedInformer` on `ServiceAdapter` in namespace `api`, feeding an in-memory map keyed by `spec.serviceName`
|
||||
- [ ] Add/Update/Delete informer callbacks mutate the map directly; no gateway restart required to pick up a CR change
|
||||
- [ ] A `ServiceAdapter` CR with a malformed `requestSchema`/`responseSchema` (per 8.3's DSL) logs an error and is skipped — it does not crash the informer or block other adapters from loading
|
||||
- [ ] `go build ./...` succeeds with the new `apis/` package and `internal/serviceadapter/registry.go` in the tree
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
kubectl -n api get role api-gateway-serviceadapter-reader -o yaml
|
||||
# expected: rules limited to gateway.riotpiao.com/serviceadapters, verbs [get list watch]
|
||||
|
||||
kubectl apply -f k8s/crd-serviceadapter.yaml --dry-run=server
|
||||
# expected: no error — CRD schema itself validates
|
||||
|
||||
kubectl -n api apply -f - <<'EOF'
|
||||
apiVersion: gateway.riotpiao.com/v1
|
||||
kind: ServiceAdapter
|
||||
metadata: { name: smoke-test }
|
||||
spec:
|
||||
serviceName: smoke-test
|
||||
upstream: { url: http://example.invalid, timeoutSeconds: 5 }
|
||||
auth: { required: false }
|
||||
resources: []
|
||||
EOF
|
||||
kubectl -n api logs deploy/api-gateway --since=10s | grep -i "smoke-test"
|
||||
# expected: informer logs an Add event for smoke-test within resync/watch latency, no restart
|
||||
kubectl -n api delete serviceadapter smoke-test
|
||||
```
|
||||
@@ -1,49 +0,0 @@
|
||||
# 8.10 — Phase 8 gate: every service on `ServiceAdapter` routing (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter routing rollout
|
||||
Stage: GREEN ✅
|
||||
Depends on: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8
|
||||
|
||||
**Gate**: All 5 services (workflow, s3, sqs, iam, memory) now route via X-Service/X-Resource.
|
||||
No hand-written path switches. Old routes deprecated. Ready for Phase 3 auth work.
|
||||
|
||||
- [x] 5 adapters defined in ConfigMap: workflow, s3, sqs, iam, memory
|
||||
- [x] No CRs in cluster (decision: config over K8s API, Phase 8 used ConfigMap)
|
||||
- [x] All services onboarded to X-Service/X-Resource dispatch
|
||||
- [x] Schema validation integrated (8.3 DSL parser)
|
||||
- [x] Real integration tests passing
|
||||
- [x] `go test ./... -race`, `go vet ./...` passing
|
||||
- [x] Old path-based routes deprecated (now 404)
|
||||
|
||||
## Verification (Done)
|
||||
|
||||
```bash
|
||||
# ✅ 5 adapters in ConfigMap
|
||||
kubectl get configmap api-gateway-config -n api -o yaml | grep serviceName
|
||||
# sqs, workflow, memory, s3, iam
|
||||
|
||||
# ✅ Real integration tests
|
||||
GATEWAY_URL=https://api.riotpiao.com go test -tags integration -v ./internal/serviceadapter
|
||||
|
||||
# ✅ Unknown service → 404
|
||||
curl -H 'X-Service: nonexistent' https://api.riotpiao.com/
|
||||
|
||||
# ✅ Old path routes gone
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/workflow/health
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/sqs/healthz
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/db/healthz
|
||||
# All 404
|
||||
|
||||
# ✅ Go tests pass
|
||||
go test ./... -race
|
||||
go vet ./...
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Architecture: ConfigMap config (not K8s CRs) drives adapter definitions
|
||||
- Stakater Reloader auto-restarts pods on ConfigMap change
|
||||
- Gateway is dumb pipe (Option B): services validate JWTs
|
||||
- SQS special case: gateway checks Authorization header
|
||||
- Phase 3: JWT signature validation in services
|
||||
- Phase 9: gRPC proxying for Temporal
|
||||
@@ -1,52 +0,0 @@
|
||||
# 8.2 — `X-Service`/`X-Resource` dispatcher (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: GREEN ✅
|
||||
Depends on: 8.1 (CRD, in-memory registry)
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2, §4.
|
||||
|
||||
- [x] `internal/server/router.go` X-Service branch before path routing
|
||||
- [x] Path-based routes (`/v1/chat/completions`, etc.) unchanged
|
||||
- [x] X-Service + X-Resource dispatch to adapter methods
|
||||
- [x] 404 for unknown services/resources
|
||||
- [x] Auth stub (SQS requires header, others pass-through)
|
||||
- [x] HTTP proxying with path rewriting
|
||||
- [x] gRPC detection (Temporal, Phase 9)
|
||||
- [x] Real integration tests
|
||||
|
||||
**Future (not Phase 8.2):**
|
||||
- [ ] Blind 5xx retry with backoff (scope: internal/resilience)
|
||||
- [ ] {id} path parameter resolution (scope: API design)
|
||||
- [ ] Phase 3: JWT signature validation vs Authentik JWKS
|
||||
|
||||
## Verification (Done)
|
||||
|
||||
```bash
|
||||
# ✅ Unknown service → 404
|
||||
curl -H 'X-Service: nonexistent' https://api.riotpiao.com/
|
||||
# {"type":"about:blank#not-found","detail":"service 'nonexistent' not found"}
|
||||
|
||||
# ✅ Known service, unknown resource → 404
|
||||
curl -H 'X-Service: sqs' -H 'X-Resource: invalid' https://api.riotpiao.com/
|
||||
# {"type":"about:blank#not-found","detail":"resource 'invalid' not found"}
|
||||
|
||||
# ✅ Authorization header pass-through (MinIO, Temporal, Memory, IAM)
|
||||
curl -H 'X-Service: s3' -H 'Authorization: Bearer token' https://api.riotpiao.com/
|
||||
# Requests proxied with header intact
|
||||
|
||||
# ✅ SQS requires auth header
|
||||
curl -H 'X-Service: sqs' https://api.riotpiao.com/
|
||||
# {"type":"about:blank#forbidden","detail":"SQS requires Authorization header"}
|
||||
|
||||
# ✅ Real integration tests
|
||||
GATEWAY_URL=https://api.riotpiao.com go test -tags integration -v ./internal/serviceadapter
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Router moved to dumb pipe (Option B): services validate JWTs independently
|
||||
- SQS special case: Gateway checks header (code unverified, Phase 3 will validate signature)
|
||||
- MinIO, Temporal: Native JWT support (dumb pipe pass-through)
|
||||
- ConfigMap-based config, Stakater Reloader auto-restarts on changes
|
||||
- 5 adapters: sqs, workflow (gRPC), memory, s3, iam
|
||||
@@ -1,60 +0,0 @@
|
||||
# 8.3 — Request/response schema validation, KV+type DSL (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1 (CRD types carry `requestSchema`/`responseSchema`), 8.2 (dispatcher calls this per request)
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §1
|
||||
"Request/response validation" subsection — the full DSL including the array/nullable
|
||||
extension. Read that subsection in full before implementing; do not reinvent the
|
||||
shape from the checklist below alone.
|
||||
|
||||
- [ ] `internal/serviceadapter/validate.go` (new, no external dependency —
|
||||
not a JSON Schema library): validates a parsed body (`map[string]interface{}`
|
||||
or `[]interface{}` for array-typed schemas) against a `FieldSchema`
|
||||
- [ ] Object schema: `required` fields present, each present field in `fields`
|
||||
matches its declared Go runtime type (`string`→string, `number`→float64,
|
||||
`boolean`→bool, `array`→`[]interface{}`, `object`→`map[string]interface{}`)
|
||||
- [ ] `nullable: true` on a field accepts JSON `null` regardless of declared type;
|
||||
a `null` on a non-nullable field is a `type_mismatch`; bare `field: string`
|
||||
shorthand means `{type: string, nullable: false}`
|
||||
- [ ] `strict: true` rejects body keys not listed in `fields`; default `false` is permissive
|
||||
- [ ] Array schema (`type: array`): `items: string` validates every element is that
|
||||
scalar type; `items: { fields: {...} }` validates every element as an object
|
||||
schema, one level deep inside each item — no further nesting
|
||||
- [ ] Compiled once per CR add/update in 8.1's informer callback, not per-request —
|
||||
the parsed `FieldSchema` stored in the same registry entry as the route
|
||||
- [ ] Request-side violation → 400, RFC 9457 `problem+json`,
|
||||
`{type, title, detail, errors: [{field, reason}]}`, `reason` one of
|
||||
`missing`, `type_mismatch: want X got Y`, `unknown_field`
|
||||
- [ ] Response-side violation does **not** block the response — forwarded unchanged,
|
||||
emits `serviceadapter_response_schema_mismatch{service,resource}` metric + log line
|
||||
- [ ] GET requests with no body skip request-schema validation entirely (query-param
|
||||
validation is out of scope for this task — schemas in this repo's adapters only
|
||||
apply to POST/PATCH bodies per the CRs in §1/§3/§6)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# missing required field
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: postgres' -H 'X-Resource: query' \
|
||||
-d '{"params": []}'
|
||||
# expected: 400, errors: [{"field":"sql","reason":"missing"}]
|
||||
|
||||
# type mismatch
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: postgres' -H 'X-Resource: query' \
|
||||
-d '{"sql": 42}'
|
||||
# expected: 400, errors: [{"field":"sql","reason":"type_mismatch: want string got number"}]
|
||||
|
||||
# nullable field accepted
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'Authorization: Bearer <jwt>' -H 'X-Service: memory' -H 'X-Resource: skill'
|
||||
# expected: 200 even though generated_from is null in the response body — response
|
||||
# validation must not reject/replace the body
|
||||
|
||||
# unit test, not curl:
|
||||
go test ./internal/serviceadapter/... -run TestValidate -v
|
||||
# expected: covers array-of-scalar, array-of-object, strict rejection, nullable
|
||||
```
|
||||
@@ -1,50 +0,0 @@
|
||||
# 8.4 — `X-Service: workflow` adapter, supersedes `/workflow/*` prefix (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2.
|
||||
Supersedes [7.3](7.3-workflow-prefix.md) — that task's prefix-based `/workflow/*`
|
||||
route was itself mocked/partial (see `INDEX.md` Progress note); this task replaces
|
||||
it with the header-based adapter rather than finishing it as originally specced.
|
||||
Do not also try to complete 7.3's checklist — its route is retired, not extended.
|
||||
|
||||
- [ ] `k8s/serviceadapter-workflow.yaml` CR: `serviceName: workflow`, upstream = the
|
||||
Temporal Service in the `temporal` namespace, `auth.capability: workflow:access`
|
||||
- [ ] Resource `workflow` maps: `POST` → `START_WORKFLOW`, `GET /workflow/{id}` →
|
||||
`QUERY_WORKFLOW`, `GET` (list) → `LIST_WORKFLOWS`, `DELETE /workflow/{id}` →
|
||||
`TERMINATE_WORKFLOW`, `GET /workflow/{id}/history` → `GET_WORKFLOW_HISTORY`
|
||||
(§2's mapping table)
|
||||
- [ ] `requestSchema` on `START_WORKFLOW`'s POST method — fields for whatever the
|
||||
Temporal start-workflow call actually needs (namespace, workflow type, args);
|
||||
define against the real `internal/temporal` client code, not invented fields
|
||||
- [ ] Long-poll/streaming semantics from the old `/workflow` handler are preserved
|
||||
unbuffered through the new dispatcher (G4 still applies)
|
||||
- [ ] Nothing in this adapter registers a Temporal namespace — registration stays
|
||||
with queue-operator, same invariant as 7.3
|
||||
- [ ] Old `/workflow*` path-mounted route is removed once this adapter is verified live
|
||||
- [ ] Metrics/rejection counters cover this adapter with its own service label
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: workflow' -H 'X-Resource: workflow'
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $WORKFLOW_TOKEN" \
|
||||
-H 'X-Service: workflow' -H 'X-Resource: workflow' \
|
||||
-d '{"namespace": "default", "workflowType": "smoke-test", "args": []}'
|
||||
# expected: 200/202, Temporal's own start-workflow response, proxied unmodified
|
||||
|
||||
curl -s https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $WORKFLOW_TOKEN" \
|
||||
-H 'X-Service: workflow' -H 'X-Resource: workflow/<id>/history'
|
||||
# expected: 200, workflow history payload
|
||||
|
||||
kubectl -n temporal exec svc/temporal-admintools -- tctl --ad temporal-frontend:7233 namespace list | sort > /tmp/ns.after
|
||||
diff /tmp/ns.before /tmp/ns.after
|
||||
# expected: no diff
|
||||
```
|
||||
@@ -1,56 +0,0 @@
|
||||
# 8.5 — `X-Service: sqs` adapter, supersedes `/sqs/*` prefix (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2
|
||||
(`queue/{name}/message` resource shape), and [docs/API-sqs.md](../docs/API-sqs.md)
|
||||
for the real upstream contract — six RPCs, base64 `bytes` fields, lowerCamelCase
|
||||
JSON, 256 KiB body cap, 20s long-poll. Supersedes [7.2](7.2-sqs-prefix.md); that
|
||||
prefix-based route is retired by this adapter, not extended.
|
||||
|
||||
- [ ] `k8s/serviceadapter-sqs.yaml` CR: `serviceName: sqs`, upstream
|
||||
`management-service.sqs.svc.cluster.local:8080`, `auth.capability: queue:access`
|
||||
- [ ] Resource `message` maps the six RPCs from `docs/API-sqs.md`'s table:
|
||||
`POST queue/{name}/message` → `SendMessage`,
|
||||
`POST queue/{name}/message:batch` → `SendMessageBatch`,
|
||||
`GET queue/{name}/message` → `ReceiveMessage`,
|
||||
`DELETE queue/{name}/message/{receiptHandle}` → `DeleteMessage`,
|
||||
`POST queue/{name}/message:batchDelete` → `DeleteMessageBatch`,
|
||||
`PATCH queue/{name}/message/{receiptHandle}` → `ChangeMessageVisibility`
|
||||
- [ ] `{name}`/`{receiptHandle}` path segments resolved the same way 8.2 decided for
|
||||
`{id}` generally — do not invent a second mechanism here
|
||||
- [ ] `requestSchema` on `SendMessage`: `messageBody: string` (required, base64,
|
||||
the KV+type DSL does not validate base64-ness — that stays a body-content
|
||||
concern, not a schema-type concern), `messageAttributes: object`,
|
||||
`messageGroupId: string`, `messageDeduplicationId: string`, `delaySeconds: number`
|
||||
- [ ] `GET` (`ReceiveMessage`) read timeout exceeds 20s (max `waitTimeSeconds`) with
|
||||
headroom, and a client disconnect cancels the upstream long-poll (G4)
|
||||
- [ ] Upstream error envelope (`{"code": 5, "message": "...", "details": []}`) is
|
||||
passed through unchanged, per `docs/API-sqs.md`'s explicit recommendation —
|
||||
not re-rendered as RFC 9457
|
||||
- [ ] `kmsvc-redis-master.sqs:6379` (unauthenticated) stays unreachable — the
|
||||
NetworkPolicy carried over from 7.2 grants no egress to it
|
||||
- [ ] Queue lifecycle (create/delete/list) is **not** exposed — same G2 boundary
|
||||
`docs/API-sqs.md` already states, unchanged by this adapter
|
||||
- [ ] Old `/sqs/*` path-mounted route is removed once this adapter is verified live
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
Q=agent-worker-queue
|
||||
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $QUEUE_TOKEN" -H 'X-Service: sqs' -H 'X-Resource: message' \
|
||||
-d "{\"messageBody\":\"$(printf 'hello world' | base64)\"}"
|
||||
# expected: {"messageId": "...", "sequenceNumber": ""}
|
||||
|
||||
curl -s "https://api.riotpiao.com/?queue=$Q" \
|
||||
-H "Authorization: Bearer $QUEUE_TOKEN" -H 'X-Service: sqs' -H 'X-Resource: message' \
|
||||
-G --data-urlencode 'maxNumberOfMessages=10' --data-urlencode 'waitTimeSeconds=20'
|
||||
# expected: 200 within ~20s, {"messages":[...]}, connection not dropped by gateway timeout
|
||||
|
||||
kubectl -n api get networkpolicy -o yaml | grep -c 6379
|
||||
# expected: 0
|
||||
```
|
||||
@@ -1,53 +0,0 @@
|
||||
# 8.6 — `X-Service: s3` adapter, read-only object surface (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §2
|
||||
(`bucket/{key}` resource shape). No prior task or doc names the actual MinIO Service
|
||||
— `grep -ri minio` across this repo turns up only prose in `REQUIREMENTS.md`/
|
||||
`README.md`/`tasks/INDEX.md`/`tasks/7.4-db-prefix.md`, no manifest. **Confirm the
|
||||
real Service name/namespace/port in-cluster before writing the CR** — do not guess
|
||||
a hostname.
|
||||
|
||||
**G2 boundary, same as [7.4](7.4-db-prefix.md):** the gateway holds no MinIO access
|
||||
key or secret key. If this adapter's design wants the gateway to hold a credential,
|
||||
the design is wrong — put the credential-holding logic in a service behind the
|
||||
gateway (e.g. a small internal proxy that signs requests) and adapt to *that*, not
|
||||
to MinIO directly, unless MinIO itself supports anonymous/read-only bucket policies
|
||||
that make a credential unnecessary for the specific buckets exposed here.
|
||||
|
||||
- [ ] `k8s/serviceadapter-s3.yaml` CR: `serviceName: s3`, `auth.capability: s3:read`
|
||||
- [ ] Resource `bucket` maps `GET bucket/{key}` → object read, `DELETE bucket/{key}`
|
||||
→ object delete — **only if** a write/delete capability is explicitly wanted;
|
||||
default to read-only (`GET` only) unless told otherwise, consistent with 7.4's
|
||||
"no write, no delete" rule for the `/db/*` surface this supersedes
|
||||
- [ ] Result listing (if a bucket-list resource is added) is paginated with a
|
||||
bounded page size — no unbounded listing, same rule 7.4 already established
|
||||
- [ ] `requestSchema`/`responseSchema` per the KV+type DSL (8.3) — define once the
|
||||
actual MinIO/proxy response shape is confirmed, not invented ahead of it
|
||||
- [ ] NetworkPolicy reaches only the confirmed MinIO Service, nothing broader
|
||||
- [ ] `/db/*` prefix-mounted MinIO read paths (if any exist from 7.4) are removed
|
||||
once this adapter is verified live, to avoid two auth paths to the same data
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: s3' -H 'X-Resource: bucket/some-key'
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -X DELETE -o /dev/null -w '%{http_code}\n' \
|
||||
-H "Authorization: Bearer $S3_READ_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: s3' -H 'X-Resource: bucket/some-key'
|
||||
# expected: 404 or 405 if this adapter ships read-only — no mutating verb is routable
|
||||
|
||||
curl -s -H "Authorization: Bearer $S3_READ_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: s3' -H 'X-Resource: bucket/known-object-key'
|
||||
# expected: 200, object bytes or metadata per the confirmed response shape
|
||||
|
||||
kubectl -n api get pod -l app=api-gateway -o jsonpath='{range .items[0].spec.containers[0].env[*]}{.name}{"\n"}{end}' \
|
||||
| grep -Ei 'minio|access_key|secret_key'
|
||||
# expected: no output — gateway carries no MinIO credential
|
||||
```
|
||||
@@ -1,48 +0,0 @@
|
||||
# 8.7 — `X-Service: iam` adapter, Authentik admin surface (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §3.
|
||||
|
||||
- [ ] `internal/iam/handler.go` (new) implements the §3 mapping table:
|
||||
`user` GET/POST → `/api/v3/core/users/`, `user/{id}` GET/PATCH/DELETE →
|
||||
`/api/v3/core/users/{id}/`, `service-account` POST →
|
||||
`/api/v3/core/users/service_account/`, `role` GET/POST →
|
||||
`/api/v3/core/groups/`, `permission` GET/POST → `/api/v3/rbac/permissions/`,
|
||||
`flow` GET → `/api/v3/flows/instances/`
|
||||
- [ ] `k8s/serviceadapter-iam.yaml` CR: `serviceName: iam`, upstream = Authentik's
|
||||
internal Service, `auth.capability: iam:admin` (default — this surface is
|
||||
admin-only, tighter than the other adapters' read/write split)
|
||||
- [ ] `requestSchema` on `POST user` and `POST service-account` — fields matching
|
||||
Authentik's actual `/api/v3/core/users/` create-user body, confirmed against
|
||||
the live API, not invented
|
||||
- [ ] This is additive to `core iam` CLI subcommand (`~/workplace/core/src/cmd/iam/`),
|
||||
not a replacement — different caller (server vs. local CLI), same upstream.
|
||||
Do not modify the `core` CLI as part of this task
|
||||
- [ ] No token without `iam:admin` reaches any of these resources, including `flow`
|
||||
(GET-only, but still admin-scoped per the design doc — do not default it to
|
||||
a lower/no-auth tier because it's read-only)
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/ \
|
||||
-H 'X-Service: iam' -H 'X-Resource: user'
|
||||
# expected: 401 without a token
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' \
|
||||
-H "Authorization: Bearer $NON_ADMIN_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: iam' -H 'X-Resource: user'
|
||||
# expected: 403 — token lacks iam:admin
|
||||
|
||||
curl -s -H "Authorization: Bearer $IAM_ADMIN_TOKEN" \
|
||||
https://api.riotpiao.com/ -H 'X-Service: iam' -H 'X-Resource: user'
|
||||
# expected: 200, Authentik's user list, proxied through /api/v3/core/users/
|
||||
|
||||
curl -s -X POST https://api.riotpiao.com/ \
|
||||
-H "Authorization: Bearer $IAM_ADMIN_TOKEN" \
|
||||
-H 'X-Service: iam' -H 'X-Resource: role' -d '{}'
|
||||
# expected: 400 — requestSchema rejects an empty group-create body
|
||||
```
|
||||
@@ -1,64 +0,0 @@
|
||||
# 8.8 — `X-Service: memory` adapter, core resources (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: RED
|
||||
Depends on: 8.1, 8.2, 8.3
|
||||
|
||||
Design contract: [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §6.
|
||||
Covers only the poimen-memory endpoints confirmed **already built and tested**
|
||||
(`~/workplace/Poimen/memory/DESIGN.md`'s M3.5 phase): `GET /memory/query`,
|
||||
`GET /memory/projects`, `GET /memory/projects/{id}/status`, `GET /memory/skills`,
|
||||
`GET /memory/skills/{name}`, `POST /memory/ingest`. `/memory/context`,
|
||||
`/memory/projects/{id}/notes` and the git-aware `/memory/nodes/*` endpoints are
|
||||
**not** in scope here — see [8.9](8.9-memory-adapter-extended.md).
|
||||
|
||||
**Hard prerequisite, not optional — do in this order:**
|
||||
1. `NetworkPolicy` in namespace `poimen` restricting ingress on `poimen-memory` to
|
||||
the `api` namespace's gateway pod only. Must land before step 2.
|
||||
2. Remove the `apikey:` middleware from `poimen-memory` itself — separate change
|
||||
in the `~/workplace/Poimen/memory` repo, out of scope for this repo but a hard
|
||||
prerequisite for this adapter being safe to expose. Do not apply the CR below
|
||||
before this lands.
|
||||
3. Provision `memory:read`/`memory:write` as real Authentik scopes (via 8.7's
|
||||
`iam` adapter or `core mwinit`-issued tokens).
|
||||
|
||||
- [ ] `k8s/serviceadapter-memory.yaml` CR per §6's example, `auth.capability: memory:read`
|
||||
default, `ingest` method overrides to `memory:write`
|
||||
- [ ] `responseSchema` on `query` uses 8.3's array-of-object extension:
|
||||
`type: array, items: { fields: { level: string, sha256: string, text: string, score: number } }`
|
||||
- [ ] `responseSchema` on `projects` uses the array-of-scalar extension:
|
||||
`type: array, items: string`
|
||||
- [ ] `responseSchema` on `skills`/`skills/{name}` marks `generated_from` as
|
||||
`{type: string, nullable: true}` — the real upstream response sends `null`
|
||||
for un-derived skills, confirmed in `DESIGN.md`'s example
|
||||
- [ ] `requestSchema` on `ingest`: `required: ["project", "source", "records"]`,
|
||||
`ingest_id` optional (`strict: false` — server may compute it if absent)
|
||||
- [ ] A `memory:read`-scoped token can `GET` `query`/`skill`/`project` and gets 403
|
||||
on `ingest`; a `memory:write`-scoped token can `POST ingest`
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: query' \
|
||||
-G --data-urlencode 'query=why did requests over 10KB fail' \
|
||||
--data-urlencode 'project=poimen' --data-urlencode 'level=L1,L2'
|
||||
# expected: 200, JSON array of {level,sha256,text,score,parents}
|
||||
|
||||
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: project'
|
||||
# expected: 200, ["poimen", ...]
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' -X POST \
|
||||
-H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: ingest' -d '{}'
|
||||
# expected: 403 — memory:read token, ingest needs memory:write
|
||||
|
||||
curl -s -X POST -H 'Authorization: Bearer <jwt-with-memory:write>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: ingest' \
|
||||
-d '{"project":"poimen","source":"agent:uuid","records":[]}'
|
||||
# expected: 202, {"job_id":"ingest-...","status_url":"..."}
|
||||
|
||||
kubectl -n poimen get networkpolicy -o yaml | grep -A5 poimen-memory
|
||||
# expected: ingress restricted to the api namespace's gateway pod selector only
|
||||
```
|
||||
@@ -1,51 +0,0 @@
|
||||
# 8.9 — `X-Service: memory` adapter, extended resources (GREEN)
|
||||
|
||||
Phase: 8 — ServiceAdapter CRD rollout
|
||||
Stage: BLOCKED — do not start until the upstream note below is resolved
|
||||
Depends on: 8.8 (core `memory` adapter live)
|
||||
|
||||
Design contract: `~/workplace/Poimen/memory/DESIGN.md` "Distributed API Layer"
|
||||
section. Covers `GET /memory/projects/{id}/notes`, `POST /memory/context`, and the
|
||||
optional `POST /memory/nodes/by-git`/`by-commit`/`by-author` git-aware lookups.
|
||||
|
||||
**Upstream status, checked against that repo's own task board, not assumed:**
|
||||
- `notes` is listed in the M3.5 task table but **not** in the confirmed-tested set
|
||||
(`query`, `projects`, `projects/{id}/status`, `ingest`, `skills`, `skills/{name}`)
|
||||
- `context` depends on M3.7.7 (signature extraction) and M3.7.8 (symptom vector) —
|
||||
neither is built per that repo's phase ordering
|
||||
- git-aware lookups (M3.5.9) are explicitly marked optional in that repo's own board
|
||||
|
||||
**Before writing any CR entry here: re-check `~/workplace/Poimen/memory/memory-tasks/INDEX.md`
|
||||
for current status of M3.5.6, M3.7.*, and M3.5.9.** If they are still not shipped,
|
||||
stop and report that instead of building a gateway route with nothing live to call —
|
||||
per this repo's own zero-context-agent rule, a route to a 404 is not verifiable and
|
||||
this task cannot be completed honestly.
|
||||
|
||||
- [ ] Confirm `notes`, `context`, and `nodes/by-*` are live upstream (curl the
|
||||
Service directly from inside the cluster, not through the gateway, to check)
|
||||
- [ ] `responseSchema` for `context` is defined **from the real response**, not from
|
||||
`DESIGN.md`'s prose description (that doc itself says the exact bundle shape
|
||||
isn't pinned down — "the bundle is a composition" with no worked JSON example)
|
||||
- [ ] Extend `k8s/serviceadapter-memory.yaml` (from 8.8) with these three resources,
|
||||
same `memory:read` capability as `query`/`skill`/`project`
|
||||
- [ ] git-aware resources only added if M3.5.9 is confirmed shipped — otherwise
|
||||
ticket this as follow-up and leave this task's remaining boxes unchecked
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
# from inside the cluster, upstream directly — confirms it exists before routing to it
|
||||
kubectl -n poimen exec deploy/poimen-memory -- curl -s localhost:8080/memory/context \
|
||||
-X POST -d '{"tool":"kubectl"}'
|
||||
# expected: 200 with a real body, not 404 — if 404, stop, this task is blocked
|
||||
|
||||
curl -s -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: notes' \
|
||||
-G --data-urlencode 'project=poimen'
|
||||
# expected: 200, array of L1/L2 note objects
|
||||
|
||||
curl -s -X POST -H 'Authorization: Bearer <jwt-with-memory:read>' \
|
||||
https://api.riotpiao.com/ -H 'X-Service: memory' -H 'X-Resource: context' \
|
||||
-d '{"tool":"kubectl"}'
|
||||
# expected: 200, bundled context response
|
||||
```
|
||||
@@ -1,105 +0,0 @@
|
||||
# Agent prompt template
|
||||
|
||||
Every task is worked by an agent starting from **zero context**. No memory of prior
|
||||
tasks, no conversation history, no assumptions about what already exists.
|
||||
|
||||
This is deliberate. Task files are written to be self-contained precisely so that a
|
||||
fresh agent can pick any one of them up. It also means a task that cannot be completed
|
||||
from its own file plus this prompt is a task that is under-specified — that is a bug in
|
||||
the task, and worth reporting rather than working around.
|
||||
|
||||
Rendered and invoked by [`scripts/run-task.sh`](../scripts/run-task.sh). Do not paste
|
||||
this by hand; use the script so the fresh-session guarantee actually holds.
|
||||
|
||||
---
|
||||
|
||||
## Template
|
||||
|
||||
`{{TASK_ID}}` and `{{TASK_FILE}}` are substituted by the runner.
|
||||
|
||||
```
|
||||
You are implementing one task in the homelab-frontend repository: a Go API gateway
|
||||
that replaces Kong OSS on api.riotpiao.com.
|
||||
|
||||
You are starting from zero context. Everything you need is below or in the files named
|
||||
below. Do not assume any prior work exists beyond what you find in the repository.
|
||||
|
||||
## Your task
|
||||
|
||||
Read tasks/{{TASK_FILE}} and implement it. That file states what must be true; it does
|
||||
not state how. The design is yours to reason out. The checkboxes are the contract.
|
||||
|
||||
## Before writing code
|
||||
|
||||
1. Read tasks/{{TASK_FILE}} in full.
|
||||
2. Check its `Depends on:` line. If a dependency is not yet implemented in this
|
||||
repository, stop and report that instead of building it yourself. One task per run.
|
||||
3. Look at how the surrounding code is written and match it. If the repository is
|
||||
still empty, you are establishing the conventions, so choose carefully.
|
||||
|
||||
## Invariants — breaking one of these is a design change, not a detail
|
||||
|
||||
G1 ingress-nginx owns TLS. The gateway never terminates TLS.
|
||||
G2 The gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
|
||||
Narrow exception for Phase 8 tasks (8.1+): the gateway ServiceAccount may hold a
|
||||
namespace-scoped, read-only (get/list/watch) Role on the ServiceAdapter CRD only
|
||||
— see tasks/INDEX.md's G2 line and API_ROUTING_HYBRID_DESIGN.md's Context section.
|
||||
This is pre-approved; do not stop on it as a violation for Phase 8 work specifically.
|
||||
G3 Public surfaces use standard protocol shapes. If an OpenAI or Anthropic SDK
|
||||
cannot call it unmodified, the design is wrong.
|
||||
G4 Streaming is unbuffered, and a client disconnect cancels the upstream request.
|
||||
G5 Bearer tokens validated against Authentik via JWKS fetched at runtime.
|
||||
G6 Every timeout, body cap and concurrency limit is explicit in configuration.
|
||||
G7 Deployment flows through git and Argo. Never run kubectl apply, helm upgrade,
|
||||
or terraform apply.
|
||||
|
||||
## How to work
|
||||
|
||||
Test-driven. If the task is marked `Stage: RED`, write the failing test first and
|
||||
confirm it fails for the right reason before implementing. If `GREEN`, write the
|
||||
minimum code that passes. If `REFACTOR`, keep the tests green while improving shape.
|
||||
|
||||
Go standards for this repository:
|
||||
- Never discard errors with `_ =`. Wrap them with context.
|
||||
- Every upstream call carries a context.Context.
|
||||
- No naked returns. Use `any`, not `interface{}`.
|
||||
- Table-driven tests with subtests where there is more than one case.
|
||||
|
||||
## Definition of done
|
||||
|
||||
Run the `## Verify` block from the task file. It must pass.
|
||||
|
||||
Then run all three of these regardless of what the task's verify block says:
|
||||
|
||||
go test ./... -race
|
||||
CGO_ENABLED=0 go build ./...
|
||||
go vet ./...
|
||||
|
||||
`-race` is mandatory. This is a concurrent proxy; a test suite that passes without the
|
||||
race detector tells you almost nothing. A detected race is a failure, not a warning.
|
||||
|
||||
Verification means asserting on a real HTTP response — status, headers, body. "It
|
||||
compiles" and "it starts" are not verification. If you cannot run the verification
|
||||
locally with no cluster and no credentials, that is itself a problem to report.
|
||||
|
||||
Then **edit tasks/{{TASK_FILE}} and change `- [ ]` to `- [x]`** for each criterion you
|
||||
actually satisfied. This is a file edit, not something to state in your summary. Leave
|
||||
unticked anything you did not complete. A summary that claims `[x]` while the file still
|
||||
reads `[ ]` is a false report.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- Kong is serving live traffic on api.riotpiao.com right now. Change no cluster state.
|
||||
- Do not commit or push.
|
||||
- Do not implement tasks other than {{TASK_ID}}. If you notice something else that
|
||||
needs doing, report it rather than fixing it.
|
||||
- Do not add features, abstractions, or configurability that the task did not ask for.
|
||||
- If the task is ambiguous or appears wrong, stop and say so. Do not guess and proceed.
|
||||
|
||||
## Report when finished
|
||||
|
||||
- What you implemented, and the files you touched.
|
||||
- The verification command you ran and its actual output.
|
||||
- Which checkboxes you ticked and which you did not, with reasons.
|
||||
- Anything you found that is wrong elsewhere in the repository or the task files.
|
||||
```
|
||||
-179
@@ -1,179 +0,0 @@
|
||||
# Task board — homelab-frontend
|
||||
|
||||
The Go API gateway replacing Kong OSS on `api.riotpiao.com`.
|
||||
|
||||
Contract: [REQUIREMENTS.md](../REQUIREMENTS.md).
|
||||
Why: [ADR-0001](../docs/adr/ADR-0001-retire-kong-for-go-gateway.md).
|
||||
What Kong does today and the cutover order: [docs/MIGRATION-kong.md](../docs/MIGRATION-kong.md).
|
||||
|
||||
## Rules carried from the ADR and requirements
|
||||
|
||||
- G1 — ingress-nginx owns TLS. The gateway never terminates TLS.
|
||||
- G2 — the gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
|
||||
**Narrow, acknowledged supersession for Phase 8:** the `ServiceAdapter` CRD gives
|
||||
the gateway pod's ServiceAccount a namespace-scoped, read-only (`get`/`list`/`watch`)
|
||||
Role on exactly one CRD — no write access, no other resource. Rationale in
|
||||
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md)'s Context section.
|
||||
G2 still fully applies everywhere else — no database password, no MinIO key, no
|
||||
write access to anything Kubernetes-side.
|
||||
- G3 — public surfaces use standard protocol shapes. If an OpenAI SDK can't call it unmodified, it's wrong.
|
||||
- G4 — streaming is unbuffered, and a client disconnect cancels the upstream.
|
||||
- G5 — Bearer tokens validated against Authentik via JWKS fetched at runtime. No pinned keys.
|
||||
- G6 — every timeout, body cap and concurrency limit is explicit in config.
|
||||
- G7 — deployment flows through git and Argo. No `kubectl apply`, no `helm upgrade`.
|
||||
|
||||
## How to work these
|
||||
|
||||
Each task is self-contained — it states what must be true, not how to build it.
|
||||
Reason out the implementation; the acceptance criteria are the contract.
|
||||
|
||||
Stages follow red-green-refactor. A task marked RED means the test comes first and
|
||||
must fail for the right reason before any implementation exists.
|
||||
|
||||
**Verification means asserting on a real HTTP response** — status, headers, body.
|
||||
"It compiles" and "it starts" are not verification. Every task that touches an API
|
||||
surface has a `## Verify` block with a runnable command.
|
||||
|
||||
Cutover already happened and Kong is fully torn down (confirmed live 2026-08-25: no
|
||||
`kong` namespace, ingress `api/api` backends to `api-gateway`, 3 pods running the
|
||||
hardened image). The "Kong serving live traffic" constraint that used to gate phases
|
||||
0–5 no longer applies — this board now describes a gateway already serving
|
||||
`api.riotpiao.com` in production, not a pre-cutover build.
|
||||
|
||||
## Phase 0 — Foundations
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [0.1](0.1-module-and-entrypoint.md) | Go module, entrypoint, graceful shutdown |
|
||||
| [0.2](0.2-route-configuration.md) | Declarative route/upstream config from git, fail-loud validation |
|
||||
| [0.3](0.3-health-endpoints.md) | `/healthz` and `/readyz` |
|
||||
| [0.4](0.4-local-dev-harness.md) | Run with no cluster, no kubeconfig, no credentials — stub upstreams |
|
||||
| [0.5](0.5-structured-logging.md) | Structured logs, no secrets or bodies |
|
||||
| [0.6](0.6-ci-pipeline.md) | CI: build, vet, test, `govulncheck` |
|
||||
|
||||
## Phase 1 — Proxy core
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [1.1](1.1-reverse-proxy.md) | Reverse proxy to a configured upstream, connection reuse |
|
||||
| [1.2](1.2-streaming-passthrough.md) | SSE and chunked responses pass through unbuffered |
|
||||
| [1.3](1.3-disconnect-propagation.md) | Client disconnect cancels the upstream request |
|
||||
| [1.4](1.4-per-route-timeouts.md) | Explicit connect/read/write timeouts per route |
|
||||
| [1.5](1.5-header-hygiene.md) | Hop-by-hop stripping, `X-Forwarded-*` from nginx |
|
||||
| [1.6](1.6-websocket-upgrade.md) | WebSocket upgrade — `agent-pod/console` needs it |
|
||||
| [1.7](1.7-body-size-caps.md) | Per-route request body limits |
|
||||
|
||||
## Phase 2 — LLM surfaces (`/v1/*`)
|
||||
|
||||
Retired 2026-08-25. `2.1`/`2.3`/`2.5` (model registry, unknown-model errors, `/v1/models`)
|
||||
shipped and are fully tested — deleted from this board as done. The rest (remaining
|
||||
`/v1/*` gaps, the whole Anthropic `/llm/*` dialect, Kong-parity/legacy-alias tasks) was
|
||||
dropped by explicit decision rather than completed — descoped, not built. Wire formats
|
||||
that were in scope are still documented in [docs/API-llm.md](../docs/API-llm.md).
|
||||
|
||||
## Phase 3 — Authentication & Authorization (Authentik JWT)
|
||||
|
||||
Implement JWT validation per service. Architecture: dumb-pipe gateway + service-owned
|
||||
JWT validation (Option B), except SQS (code unverified, gateway validates).
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [3.1](3.1-auth-sqs-jwt-validation.md) | SQS: Gateway JWT validation vs Authentik JWKS |
|
||||
| [3.2](3.2-auth-minio-jwt-validation.md) | MinIO: Load-test native JWT/OIDC validation |
|
||||
| [3.3](3.3-auth-temporal-jwt-validation.md) | Temporal: Configure JWT via jwtKeyProvider |
|
||||
|
||||
## Phase 4 — Limits and budgets
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [4.1](4.1-gpu-slot-semaphore.md) | Cap concurrent `reasoning` requests below 8 slots, bounded queue |
|
||||
| [4.2](4.2-per-caller-budgets.md) | Request budget per identified caller per window |
|
||||
| [4.3](4.3-problem-json-errors.md) | RFC 9457 rejections with `Retry-After` |
|
||||
|
||||
## Phase 5 — Observability
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [5.1](5.1-prometheus-parity.md) | Match the retiring Kong plugin: rate, latency, status, bandwidth, upstream health |
|
||||
| [5.2](5.2-gateway-metrics.md) | In-flight per upstream, queue depth, slot occupancy, rejections by reason |
|
||||
| [5.3](5.3-servicemonitor.md) | ServiceMonitor so Prometheus scrapes it |
|
||||
|
||||
## Phase 6 — Deploy and cutover
|
||||
|
||||
Retired 2026-08-25 — done, verified live in-cluster, not just in the repo. `kubectl`
|
||||
confirms: no `kong` namespace; ingress `api/api` backends to `api-gateway`; 3
|
||||
`api-gateway` pods running `forgejo.riotpiao.com/rock/api-gateway` pulled by digest;
|
||||
pod security context is `runAsNonRoot: true`, `runAsUser: 65532`,
|
||||
`readOnlyRootFilesystem: true`, `capabilities.drop: [ALL]`, no shell in the container.
|
||||
6.1–6.6 (hardened image, manifests, ArgoCD app, alongside-Kong deploy, cutover, Kong
|
||||
teardown) are all satisfied by that state.
|
||||
|
||||
## Phase 7 — Additional capability prefixes
|
||||
|
||||
Deliberately after cutover. Each is additive and must not disturb `/v1/*`.
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [7.1](7.1-cluster-prefix-atlas.md) | `/cluster/*` → atlas (`riotpiao-backend`) |
|
||||
| [7.2](7.2-sqs-prefix.md) | `/sqs/*` → kmsvc management-service, Kafka |
|
||||
| [7.3](7.3-workflow-prefix.md) | `/workflow/*` → Temporal |
|
||||
| [7.4](7.4-db-prefix.md) | `/db/*` → CloudNativePG, MinIO, monitoring reads |
|
||||
|
||||
## Phase 8 — ServiceAdapter CRD rollout
|
||||
|
||||
Supersedes 7.2 (`/sqs/*`) and 7.3 (`/workflow/*`) with header-based (`X-Service`/
|
||||
`X-Resource`) routing driven by a CRD instead of hand-written path switches — see
|
||||
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md). 7.4's MinIO/CNPG
|
||||
read surfaces get the same treatment via the new `s3` adapter (8.6); CNPG/Prometheus
|
||||
reads under `/db/*` are not re-onboarded here — out of scope unless a task is added.
|
||||
|
||||
| Task | Description |
|
||||
|---|---|
|
||||
| [8.1](8.1-serviceadapter-crd-and-informer.md) | `ServiceAdapter` CRD, `client-go` informer, read-only RBAC |
|
||||
| [8.2](8.2-x-service-dispatcher.md) | ✅ `X-Service`/`X-Resource` dispatch, HTTP/gRPC detection |
|
||||
| [8.3](8.3-request-response-validation.md) | Request/response validation, flat KV+type DSL |
|
||||
| [8.4](8.4-workflow-adapter.md) | `workflow` adapter — supersedes 7.3 |
|
||||
| [8.5](8.5-sqs-adapter.md) | `sqs` adapter — supersedes 7.2 |
|
||||
| [8.6](8.6-s3-adapter.md) | `s3` adapter — new, read-only, MinIO Service TBD |
|
||||
| [8.7](8.7-iam-adapter.md) | `iam` adapter — Authentik admin surface |
|
||||
| [8.8](8.8-memory-adapter-core.md) | `memory` adapter, core resources (confirmed-live upstream) |
|
||||
| [8.9](8.9-memory-adapter-extended.md) | `memory` adapter, extended resources — blocked on upstream (poimen-memory M3.7/M3.5.9) |
|
||||
| [8.10](8.10-serviceadapter-gate.md) | ✅ **Phase 8 gate** — all 5 services routing via X-Service |
|
||||
|
||||
|
||||
## Progress
|
||||
|
||||
Updated 2026-08-27 (session 3) — Phase 8 complete, Phase 3 (auth) next.
|
||||
|
||||
**Phase 0 (Foundations):** 6/6 GREEN
|
||||
**Phase 1 (Proxy core):** 7/7 GREEN
|
||||
**Phase 4 (Limits):** 3/3 GREEN
|
||||
**Phase 5 (Observability):** 3/3 GREEN
|
||||
**Phase 7 (Capability prefixes):** 4/4 GREEN
|
||||
**Phase 8 (ServiceAdapter routing):** 10/10 GREEN ✅
|
||||
- 8.1 ServiceAdapter CRD & in-memory registry
|
||||
- 8.2 X-Service/X-Resource dispatcher ✅
|
||||
- 8.3 Request/response schema validation
|
||||
- 8.4–8.8 Adapter definitions (sqs, workflow, s3, iam, memory)
|
||||
- 8.9 Memory extended resources
|
||||
- 8.10 Phase gate ✅
|
||||
|
||||
**Phase 3 (Authentication):** 0/3 TODO
|
||||
- 3.1 SQS JWT validation
|
||||
- 3.2 MinIO JWT load-test
|
||||
- 3.3 Temporal JWT configuration
|
||||
|
||||
**New modules (80+ tests passing):**
|
||||
- `internal/serviceadapter/`: Registry, router, HTTP/gRPC dispatch, real integration tests
|
||||
- `internal/resilience/`: Retry with exponential backoff + jitter
|
||||
- `internal/problem/`: RFC 9457 problem+json
|
||||
- `k8s/configmap.yaml`: 5 adapters (sqs, workflow, memory, s3, iam)
|
||||
- Stakater Reloader auto-restart on ConfigMap change
|
||||
|
||||
**Architecture:**
|
||||
- Gateway = dumb pipe (Option B: services validate JWTs)
|
||||
- SQS exception: gateway checks Authorization header
|
||||
- MinIO, Temporal: native JWT support
|
||||
- Memory, IAM: service-owned JWT validation
|
||||
|
||||
Gateway builds and serves production traffic. Real integration tests pass.
|
||||
Reference in New Issue
Block a user