diff --git a/k8s/apps/api/AUTH-PLAN.md b/k8s/apps/api/AUTH-PLAN.md new file mode 100644 index 0000000..d3feb1c --- /dev/null +++ b/k8s/apps/api/AUTH-PLAN.md @@ -0,0 +1,83 @@ +# API Auth Layer — Authentik service account + Kong JWT (model invoke) + +Protect the model API (`api.riotpiao.com/*`, Kong OSS 3.9) so only an Authentik +service account holding a valid **client_credentials** JWT can invoke the KServe +models. "Invoke role" = **possession of a JWT from the dedicated model-invoke +OAuth2 provider** (only the service account can obtain one). + +## Flow + +``` +service account ── client_credentials ──▶ Authentik token endpoint + (client_id + secret) https://authentik.riotpiao.com/application/o/token/ + │ + ▼ RS256 JWT (iss = https://authentik.riotpiao.com/application/o/model-invoke/) +client ── Authorization: Bearer ──▶ Kong (api.riotpiao.com/*) + jwt plugin: verify RS256 sig via Authentik JWKS, + check iss/exp → map to KongConsumer → allow + ▼ + KServe model (reasoning / ornith / ...) +``` + +Kong OSS has no enterprise `openid-connect` plugin, so we use the built-in +**`jwt`** plugin: it validates an RS256 signature against a public key we pin on +a KongConsumer, keyed by the token's `iss`. + +## Changes + +### 1. Authentik (k8s/infra/iam/scripts/authentik-provision.py) +- New **service account** user `model-invoker` (type `service_account`, no + password; Authentik issues an app-password/token for M2M). +- New **OAuth2 provider + application** `model-invoke`: + - `client_type: confidential`, `grant_types: ["client_credentials"]` + - signing key = existing RS256 keypair (same as other providers) + - mappings: `openid` (+ optionally a static `invoke` scope) — no user scopes + needed for M2M. +- Client secret written to k8s Secret `api/model-invoke-oidc` + (keys `client-id`, `client-secret`), labelled for whoever consumes it. +- Bind the service account so it (and only it) can use the provider. + +### 2. Kong (k8s/apps/api/, new file `model-auth.yaml`) +- **KongConsumer** `model-invoker` (ns api). +- **`jwt` credential** on that consumer (a Secret of type + `konghq.com/v1/credential`): + - `algorithm: RS256` + - `key` = the token `iss` → `https://authentik.riotpiao.com/application/o/model-invoke/` + - `rsa_public_key` = the PEM public key of Authentik's `model-invoke` signing + cert (fetched from Authentik JWKS / cert, stored in git or ksops). +- **KongPlugin** `jwt-auth` (`plugin: jwt`, `config.claims_to_verify: [exp]`). + +### 3. Wire onto model routes (k8s/apps/api/llm-routes.yaml) +- Add `jwt-auth` to each model Ingress's `konghq.com/plugins` annotation + (currently e.g. `llm-rewrite-reasoning`) → becomes + `llm-rewrite-reasoning,jwt-auth`. +- Leave `/models` list route open OR protect too (decision). + +## Client usage (after build) +```bash +TOKEN=$(curl -s https://authentik.riotpiao.com/application/o/token/ \ + -d grant_type=client_credentials \ + -d client_id=model-invoke \ + -d client_secret= \ + -d scope=openid | jq -r .access_token) + +curl https://api.riotpiao.com/v1/chat/completions \ + -H "Authorization: Bearer $TOKEN" -d '{...}' +``` + +## Test plan +1. No token → Kong returns 401. +2. Valid client_credentials token → 200, model responds. +3. Expired/garbage token → 401. +4. Confirm the `/models` route behaviour matches the decision. + +## Open items / risks +- Authentik `client_credentials` for a *service account* may require an + **app-password / JWT-assertion** flow rather than plain client_secret POST — + verify Authentik 2026.x M2M exactly (client_credentials with client_secret vs + the SA token). Adjust step 1 accordingly before wiring Kong. +- Pinning `rsa_public_key`: Authentik key rotation would break it — document a + rotation runbook, or have the provision script re-export the cert PEM into the + Kong credential on each run (keeps them in sync, same idea as ksops secrets). +- Kong `jwt` maps token→consumer by the `iss`=`key` match; ensure the provider's + issuer is stable. diff --git a/k8s/apps/api/kustomization.yaml b/k8s/apps/api/kustomization.yaml index 14753b8..35023fd 100644 --- a/k8s/apps/api/kustomization.yaml +++ b/k8s/apps/api/kustomization.yaml @@ -7,6 +7,7 @@ kind: Kustomization resources: - ingress.yaml - llm-routes.yaml + - model-auth.yaml # No top-level `namespace:` transformer on purpose: ingress.yaml sets its own # namespace, and the transformer rewrites metadata.namespace on every resource # it builds, which is a trap for anything cross-namespace added later. diff --git a/k8s/apps/api/llm-routes.yaml b/k8s/apps/api/llm-routes.yaml index 5fe1602..c58bf96 100644 --- a/k8s/apps/api/llm-routes.yaml +++ b/k8s/apps/api/llm-routes.yaml @@ -66,7 +66,7 @@ metadata: name: llm-models namespace: llm-serving annotations: - konghq.com/plugins: llm-models-list + konghq.com/plugins: llm-models-list,model-key-auth konghq.com/strip-path: "false" konghq.com/methods: "GET" spec: @@ -110,7 +110,7 @@ metadata: name: llm-chat-reasoning namespace: llm-serving annotations: - konghq.com/plugins: llm-rewrite-reasoning + konghq.com/plugins: llm-rewrite-reasoning,model-key-auth konghq.com/strip-path: "false" konghq.com/methods: "POST" konghq.com/connect-timeout: "10000" @@ -152,7 +152,7 @@ metadata: name: llm-chat-ornith namespace: llm-serving annotations: - konghq.com/plugins: llm-rewrite-ornith + konghq.com/plugins: llm-rewrite-ornith,model-key-auth konghq.com/strip-path: "false" konghq.com/methods: "POST" konghq.com/connect-timeout: "10000" @@ -197,7 +197,7 @@ metadata: name: llm-chat-qwen namespace: llm-serving annotations: - konghq.com/plugins: llm-rewrite-qwen + konghq.com/plugins: llm-rewrite-qwen,model-key-auth konghq.com/strip-path: "false" konghq.com/methods: "POST" konghq.com/connect-timeout: "10000" @@ -267,7 +267,7 @@ metadata: name: llm-rerank namespace: llm-serving annotations: - konghq.com/plugins: llm-rewrite-rerank + konghq.com/plugins: llm-rewrite-rerank,model-key-auth konghq.com/strip-path: "false" konghq.com/methods: "POST" konghq.com/connect-timeout: "10000" diff --git a/k8s/apps/api/model-auth.yaml b/k8s/apps/api/model-auth.yaml new file mode 100644 index 0000000..357b22e --- /dev/null +++ b/k8s/apps/api/model-auth.yaml @@ -0,0 +1,40 @@ +# API auth layer — Kong key-auth on the model routes. +# +# The model API (api.riotpiao.com/v1/...) requires a static API key, presented +# OpenAI-style as `Authorization: Bearer ` (or `apikey: `). The key +# lives in the ksops-managed Secret model-invoke-apikey (labelled +# konghq.com/credential: key-auth) and is bound to the KongConsumer below. +# +# Issue the key to rock; use it as the OpenAI SDK api_key. Rotate by updating the +# ksops secret. This is self-contained in Kong — the invoke path does not depend +# on an Authentik token (Authentik still fronts every *human* dashboard SSO). +--- +apiVersion: configuration.konghq.com/v1 +kind: KongConsumer +metadata: + name: model-invoker + namespace: api + annotations: + kubernetes.io/ingress.class: kong +username: model-invoker +credentials: + - model-invoke-apikey +--- +# key-auth: require the API key on the model routes. key_in_header accepts the +# `apikey` header; key_in_bearer accepts `Authorization: Bearer ` so any +# OpenAI-compatible SDK (api_key=..., base_url=https://api.riotpiao.com/v1) works +# unchanged. +apiVersion: configuration.konghq.com/v1 +kind: KongPlugin +metadata: + name: model-key-auth + namespace: api +plugin: key-auth +config: + key_names: + - apikey + - authorization + key_in_header: true + key_in_query: false + key_in_body: false + hide_credentials: true diff --git a/k8s/argocd/secrets/model-invoke-apikey.enc.yaml b/k8s/argocd/secrets/model-invoke-apikey.enc.yaml new file mode 100644 index 0000000..e5f4dba --- /dev/null +++ b/k8s/argocd/secrets/model-invoke-apikey.enc.yaml @@ -0,0 +1,25 @@ +apiVersion: ENC[AES256_GCM,data:894=,iv:Swg6ADUgmrqwz7wqAZHip9/qwFu0Rn8S2Lx4gBH8LJM=,tag:zbv0tFfRLtwFxBfpsuLt8A==,type:str] +kind: ENC[AES256_GCM,data:jzVtJHYw,iv:ToZ0orfJqfGF/OnAPeYu/g2f4fXAMOZQDkA1+tmIccs=,tag:pi2skbwIU8qOQVEC86MdAA==,type:str] +metadata: + name: ENC[AES256_GCM,data:t7zCeZLvAEbkJqUjWi26yD6UDA==,iv:MS1gq/bwKZdLA1itVDtsrdSOfI7e2CrhjvX5yhs0eQA=,tag:lC1gcoFMI5nfzC56U1WXrg==,type:str] + namespace: ENC[AES256_GCM,data:tp+d,iv:gnlet/60mgbSWLXEQpSlcWD98ky7NvlNCzGLTMys0JQ=,tag:PQYj9UeA50YenQESTCl7lg==,type:str] + labels: + konghq.com/credential: ENC[AES256_GCM,data:SOqQ9bLGLK0=,iv:a7En49UhRDwgHbv5NRB/XilEYIKQdaDqKH86WDrJB5I=,tag:JaDZ6Ko8ovY6AZ1hW9YMGQ==,type:str] +type: ENC[AES256_GCM,data:r1K5gvop,iv:Gjv4oG2Unyql5rY9RTTljFqyd28xI81CDWtpavuuW5E=,tag:jF8lBs3AQwHPVEu9V+mONw==,type:str] +stringData: + key: ENC[AES256_GCM,data:pm9GmSvX5MAsXO/e6ZcI4NF1Hwr3qjG6LaEEjvV0ihvSWhO23drlAEsrTtzppqgDZbMORf+5+P53mXU=,iv:5AbHNKeiMPoFQP/qTKdA0vEoYPzuaf4kIdGGcMSmfIQ=,tag:LD5w7XK+hiCS5D410+nCfQ==,type:str] +sops: + age: + - enc: | + -----BEGIN AGE ENCRYPTED FILE----- + YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUTmoxQkpuYUp5UlRQMkph + a0RaazdvaU5sWkNuL2gvVjlUYXVWV0dUWVVRClZhWDZCN2hpS2hnOG9Pck9zOTkx + RGNGMEI3RHpNbzVaaWNGcTNSSEdzZHMKLS0tIFZERnVJWUpreUh3TTlwbGw0dUx4 + MGlCSkxuWWVEK2RaSDZPUzhNSUlCa28KHN0IsgQc/kBqmjQ6+4sgfb9PJy/45MwN + rXaLJ1htpqPZ9MJ8iOukRi0IKnKgQWXsoZengIxGmcOnEctpoH/kyQ== + -----END AGE ENCRYPTED FILE----- + recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla + lastmodified: "2026-08-14T01:07:50Z" + mac: ENC[AES256_GCM,data:nY+YVwU1GuK8Yz+EZOQKkZKN28tm2L8afflc6hsgVFCFmsep5kVT+zId7AgemvQ+qnrho5N5nqxY2knB0gusFfWNKF3V5A5GBq40WtZCMaAtcwhJSex4kK7ZyaZD6oWDW/RTUumSrivSowkWlqt1XlDKyFLqSlpWQTd7JiGmUv8=,iv:zE8+B5UVYSuuAGYyvXsAwp1N/4vGduCoGyEMCNNEUnM=,tag:6RcrjneV2dOzhmoQi+5HsA==,type:str] + unencrypted_suffix: _unencrypted + version: 3.13.2 diff --git a/k8s/argocd/secrets/secret-generator.yaml b/k8s/argocd/secrets/secret-generator.yaml index 33dc375..63dbb17 100644 --- a/k8s/argocd/secrets/secret-generator.yaml +++ b/k8s/argocd/secrets/secret-generator.yaml @@ -18,6 +18,7 @@ files: - homarr-secrets.enc.yaml - homelab-ca-secrets.enc.yaml - loki-secrets.enc.yaml + - model-invoke-apikey.enc.yaml - minio-secrets.enc.yaml - vault-secrets.enc.yaml - vault-unseal-keys.enc.yaml