feat(api): add Kong key-auth for the model API — static API key (ksops-managed model-invoke-apikey) via KongConsumer model-invoker + key-auth plugin on all 5 model routes, OpenAI-SDK compatible (Authorization: Bearer <key>)

k8s/apps/api/AUTH-PLAN.md

@@ -0,0 +1,83 @@
+# API Auth Layer — Authentik service account + Kong JWT (model invoke)
+
+Protect the model API (`api.riotpiao.com/*`, Kong OSS 3.9) so only an Authentik
+service account holding a valid **client_credentials** JWT can invoke the KServe
+models. "Invoke role" = **possession of a JWT from the dedicated model-invoke
+OAuth2 provider** (only the service account can obtain one).
+
+## Flow
+
+```
+service account ── client_credentials ──▶ Authentik token endpoint
+                    (client_id + secret)   https://authentik.riotpiao.com/application/o/token/
+        │
+        ▼  RS256 JWT (iss = https://authentik.riotpiao.com/application/o/model-invoke/)
+client ── Authorization: Bearer <jwt> ──▶ Kong (api.riotpiao.com/*)
+                                           jwt plugin: verify RS256 sig via Authentik JWKS,
+                                           check iss/exp → map to KongConsumer → allow
+                                           ▼
+                                        KServe model (reasoning / ornith / ...)
+```
+
+Kong OSS has no enterprise `openid-connect` plugin, so we use the built-in
+**`jwt`** plugin: it validates an RS256 signature against a public key we pin on
+a KongConsumer, keyed by the token's `iss`.
+
+## Changes
+
+### 1. Authentik (k8s/infra/iam/scripts/authentik-provision.py)
+- New **service account** user `model-invoker` (type `service_account`, no
+  password; Authentik issues an app-password/token for M2M).
+- New **OAuth2 provider + application** `model-invoke`:
+  - `client_type: confidential`, `grant_types: ["client_credentials"]`
+  - signing key = existing RS256 keypair (same as other providers)
+  - mappings: `openid` (+ optionally a static `invoke` scope) — no user scopes
+    needed for M2M.
+- Client secret written to k8s Secret `api/model-invoke-oidc`
+  (keys `client-id`, `client-secret`), labelled for whoever consumes it.
+- Bind the service account so it (and only it) can use the provider.
+
+### 2. Kong (k8s/apps/api/, new file `model-auth.yaml`)
+- **KongConsumer** `model-invoker` (ns api).
+- **`jwt` credential** on that consumer (a Secret of type
+  `konghq.com/v1/credential`):
+  - `algorithm: RS256`
+  - `key`  = the token `iss` → `https://authentik.riotpiao.com/application/o/model-invoke/`
+  - `rsa_public_key` = the PEM public key of Authentik's `model-invoke` signing
+    cert (fetched from Authentik JWKS / cert, stored in git or ksops).
+- **KongPlugin** `jwt-auth` (`plugin: jwt`, `config.claims_to_verify: [exp]`).
+
+### 3. Wire onto model routes (k8s/apps/api/llm-routes.yaml)
+- Add `jwt-auth` to each model Ingress's `konghq.com/plugins` annotation
+  (currently e.g. `llm-rewrite-reasoning`) → becomes
+  `llm-rewrite-reasoning,jwt-auth`.
+- Leave `/models` list route open OR protect too (decision).
+
+## Client usage (after build)
+```bash
+TOKEN=$(curl -s https://authentik.riotpiao.com/application/o/token/ \
+  -d grant_type=client_credentials \
+  -d client_id=model-invoke \
+  -d client_secret=<secret> \
+  -d scope=openid | jq -r .access_token)
+
+curl https://api.riotpiao.com/v1/chat/completions \
+  -H "Authorization: Bearer $TOKEN" -d '{...}'
+```
+
+## Test plan
+1. No token → Kong returns 401.
+2. Valid client_credentials token → 200, model responds.
+3. Expired/garbage token → 401.
+4. Confirm the `/models` route behaviour matches the decision.
+
+## Open items / risks
+- Authentik `client_credentials` for a *service account* may require an
+  **app-password / JWT-assertion** flow rather than plain client_secret POST —
+  verify Authentik 2026.x M2M exactly (client_credentials with client_secret vs
+  the SA token). Adjust step 1 accordingly before wiring Kong.
+- Pinning `rsa_public_key`: Authentik key rotation would break it — document a
+  rotation runbook, or have the provision script re-export the cert PEM into the
+  Kong credential on each run (keeps them in sync, same idea as ksops secrets).
+- Kong `jwt` maps token→consumer by the `iss`=`key` match; ensure the provider's
+  issuer is stable.

k8s/apps/api/kustomization.yaml

@@ -7,6 +7,7 @@ kind: Kustomization
 resources:
   - ingress.yaml
   - llm-routes.yaml
+  - model-auth.yaml
 # No top-level `namespace:` transformer on purpose: ingress.yaml sets its own
 # namespace, and the transformer rewrites metadata.namespace on every resource
 # it builds, which is a trap for anything cross-namespace added later.

k8s/apps/api/llm-routes.yaml

@@ -66,7 +66,7 @@ metadata:
   name: llm-models
   namespace: llm-serving
   annotations:
-    konghq.com/plugins: llm-models-list
+    konghq.com/plugins: llm-models-list,model-key-auth
     konghq.com/strip-path: "false"
     konghq.com/methods: "GET"
 spec:
@@ -110,7 +110,7 @@ metadata:
   name: llm-chat-reasoning
   namespace: llm-serving
   annotations:
-    konghq.com/plugins: llm-rewrite-reasoning
+    konghq.com/plugins: llm-rewrite-reasoning,model-key-auth
     konghq.com/strip-path: "false"
     konghq.com/methods: "POST"
     konghq.com/connect-timeout: "10000"
@@ -152,7 +152,7 @@ metadata:
   name: llm-chat-ornith
   namespace: llm-serving
   annotations:
-    konghq.com/plugins: llm-rewrite-ornith
+    konghq.com/plugins: llm-rewrite-ornith,model-key-auth
     konghq.com/strip-path: "false"
     konghq.com/methods: "POST"
     konghq.com/connect-timeout: "10000"
@@ -197,7 +197,7 @@ metadata:
   name: llm-chat-qwen
   namespace: llm-serving
   annotations:
-    konghq.com/plugins: llm-rewrite-qwen
+    konghq.com/plugins: llm-rewrite-qwen,model-key-auth
     konghq.com/strip-path: "false"
     konghq.com/methods: "POST"
     konghq.com/connect-timeout: "10000"
@@ -267,7 +267,7 @@ metadata:
   name: llm-rerank
   namespace: llm-serving
   annotations:
-    konghq.com/plugins: llm-rewrite-rerank
+    konghq.com/plugins: llm-rewrite-rerank,model-key-auth
     konghq.com/strip-path: "false"
     konghq.com/methods: "POST"
     konghq.com/connect-timeout: "10000"

k8s/apps/api/model-auth.yaml

@@ -0,0 +1,40 @@
+# API auth layer — Kong key-auth on the model routes.
+#
+# The model API (api.riotpiao.com/v1/...) requires a static API key, presented
+# OpenAI-style as `Authorization: Bearer <key>` (or `apikey: <key>`). The key
+# lives in the ksops-managed Secret model-invoke-apikey (labelled
+# konghq.com/credential: key-auth) and is bound to the KongConsumer below.
+#
+# Issue the key to rock; use it as the OpenAI SDK api_key. Rotate by updating the
+# ksops secret. This is self-contained in Kong — the invoke path does not depend
+# on an Authentik token (Authentik still fronts every *human* dashboard SSO).
+---
+apiVersion: configuration.konghq.com/v1
+kind: KongConsumer
+metadata:
+  name: model-invoker
+  namespace: api
+  annotations:
+    kubernetes.io/ingress.class: kong
+username: model-invoker
+credentials:
+  - model-invoke-apikey
+---
+# key-auth: require the API key on the model routes. key_in_header accepts the
+# `apikey` header; key_in_bearer accepts `Authorization: Bearer <key>` so any
+# OpenAI-compatible SDK (api_key=..., base_url=https://api.riotpiao.com/v1) works
+# unchanged.
+apiVersion: configuration.konghq.com/v1
+kind: KongPlugin
+metadata:
+  name: model-key-auth
+  namespace: api
+plugin: key-auth
+config:
+  key_names:
+    - apikey
+    - authorization
+  key_in_header: true
+  key_in_query: false
+  key_in_body: false
+  hide_credentials: true

k8s/argocd/secrets/model-invoke-apikey.enc.yaml

@@ -0,0 +1,25 @@
+apiVersion: ENC[AES256_GCM,data:894=,iv:Swg6ADUgmrqwz7wqAZHip9/qwFu0Rn8S2Lx4gBH8LJM=,tag:zbv0tFfRLtwFxBfpsuLt8A==,type:str]
+kind: ENC[AES256_GCM,data:jzVtJHYw,iv:ToZ0orfJqfGF/OnAPeYu/g2f4fXAMOZQDkA1+tmIccs=,tag:pi2skbwIU8qOQVEC86MdAA==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:t7zCeZLvAEbkJqUjWi26yD6UDA==,iv:MS1gq/bwKZdLA1itVDtsrdSOfI7e2CrhjvX5yhs0eQA=,tag:lC1gcoFMI5nfzC56U1WXrg==,type:str]
+    namespace: ENC[AES256_GCM,data:tp+d,iv:gnlet/60mgbSWLXEQpSlcWD98ky7NvlNCzGLTMys0JQ=,tag:PQYj9UeA50YenQESTCl7lg==,type:str]
+    labels:
+        konghq.com/credential: ENC[AES256_GCM,data:SOqQ9bLGLK0=,iv:a7En49UhRDwgHbv5NRB/XilEYIKQdaDqKH86WDrJB5I=,tag:JaDZ6Ko8ovY6AZ1hW9YMGQ==,type:str]
+type: ENC[AES256_GCM,data:r1K5gvop,iv:Gjv4oG2Unyql5rY9RTTljFqyd28xI81CDWtpavuuW5E=,tag:jF8lBs3AQwHPVEu9V+mONw==,type:str]
+stringData:
+    key: ENC[AES256_GCM,data:pm9GmSvX5MAsXO/e6ZcI4NF1Hwr3qjG6LaEEjvV0ihvSWhO23drlAEsrTtzppqgDZbMORf+5+P53mXU=,iv:5AbHNKeiMPoFQP/qTKdA0vEoYPzuaf4kIdGGcMSmfIQ=,tag:LD5w7XK+hiCS5D410+nCfQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUTmoxQkpuYUp5UlRQMkph
+            a0RaazdvaU5sWkNuL2gvVjlUYXVWV0dUWVVRClZhWDZCN2hpS2hnOG9Pck9zOTkx
+            RGNGMEI3RHpNbzVaaWNGcTNSSEdzZHMKLS0tIFZERnVJWUpreUh3TTlwbGw0dUx4
+            MGlCSkxuWWVEK2RaSDZPUzhNSUlCa28KHN0IsgQc/kBqmjQ6+4sgfb9PJy/45MwN
+            rXaLJ1htpqPZ9MJ8iOukRi0IKnKgQWXsoZengIxGmcOnEctpoH/kyQ==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-08-14T01:07:50Z"
+    mac: ENC[AES256_GCM,data:nY+YVwU1GuK8Yz+EZOQKkZKN28tm2L8afflc6hsgVFCFmsep5kVT+zId7AgemvQ+qnrho5N5nqxY2knB0gusFfWNKF3V5A5GBq40WtZCMaAtcwhJSex4kK7ZyaZD6oWDW/RTUumSrivSowkWlqt1XlDKyFLqSlpWQTd7JiGmUv8=,iv:zE8+B5UVYSuuAGYyvXsAwp1N/4vGduCoGyEMCNNEUnM=,tag:6RcrjneV2dOzhmoQi+5HsA==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2

k8s/argocd/secrets/secret-generator.yaml

@@ -18,6 +18,7 @@ files:
   - homarr-secrets.enc.yaml
   - homelab-ca-secrets.enc.yaml
   - loki-secrets.enc.yaml
+  - model-invoke-apikey.enc.yaml
   - minio-secrets.enc.yaml
   - vault-secrets.enc.yaml
   - vault-unseal-keys.enc.yaml