# Poimen Memory: Authentik JWT + SOPS Encryption Setup ## Overview The Poimen Memory service uses: 1. **Authentik service account** for OAuth2 client credentials flow 2. **SOPS + Age encryption** to encrypt secrets in git 3. **JWT tokens** for authentication to LLM gateway, S3, and other services ## Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ Kubernetes (poimen) │ ├─────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────┐ ┌─────────────────┐ │ │ │ ConfigMap │ │ Secret (SOPS) │ │ │ │ (unencrypted)│ │ (age-encrypted)│ │ │ └──────┬───────┘ └────────┬────────┘ │ │ │ │ │ │ ├─────────┬───────────┤ │ │ │ │ │ │ │ ┌────▼─────────▼───────────▼────┐ │ │ │ poimen-memory Pod │ │ │ │ Environment Variables: │ │ │ │ - LLM_ENDPOINT │ │ │ │ - AUTHENTIK_ISSUER │ │ │ │ - AUTHENTIK_CLIENT_ID │ │ │ │ - AUTHENTIK_CLIENT_SECRET │ │ │ │ - S3_ACCESS_KEY │ │ │ │ - S3_SECRET_KEY │ │ │ └────┬────────────────┬──────────┘ │ │ │ │ │ │ ┌──────▼──┐ ┌──────────▼──────┐ │ │ │ Authentik│ │ LLM Endpoint │ │ │ │ (JWT) │ │ (api.riotpiao) │ │ │ └──────────┘ └─────────────────┘ │ │ │ │ ┌─────────────────────────────────────┐ │ │ │ Entity Extraction Pipeline │ │ │ │ ┌────────────────────────────┐ │ │ │ │ │ 1. WikiLink fallback │ │ │ │ │ │ 2. LLM extraction (JWT auth)│ │ │ │ │ │ 3. Reflection verification │ │ │ │ │ │ 4. Contradiction detection │ │ │ │ │ └────────────────────────────┘ │ │ │ └──────────────┬──────────────────────┘ │ │ │ │ │ ┌───────▼────────┐ │ │ │ PostgreSQL │ │ │ │ (entities DB) │ │ │ └────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────┘ ``` ## Step 1: Create Authentik Service Account ### In Authentik Admin Panel: 1. Navigate: **Settings** → **Applications** → **Create Application** 2. Name: `poimen-memory` 3. Slug: `poimen-memory` 4. Provider: Create a new OAuth2 Provider - Name: `poimen-memory` - Client type: `confidential` - Client ID: `` - Client secret: `` 5. Save and note the **Client ID** and **Client Secret** ### Verify OAuth2 Token Endpoint: ```bash curl -X POST https://authentik.riotpiao.com/application/o/token/ \ -d "grant_type=client_credentials" \ -d "client_id=" \ -d "client_secret=" # Response: # { # "access_token": "eyJ0eXAi...", # "token_type": "Bearer", # "expires_in": 3600 # } ``` ## Step 2: Create Encrypted Secrets File ### 2.1 Ensure SOPS is configured: ```bash # Load SOPS_AGE_KEY_FILE export SOPS_AGE_KEY_FILE=~/.sops/key.txt # Verify key exists ls -la ~/.sops/key.txt ``` ### 2.2 Create unencrypted secrets template: ```yaml # k8s/app/poimen-memory-secrets.yaml apiVersion: v1 kind: Secret metadata: name: poimen-memory-secrets namespace: poimen type: Opaque stringData: # Authentik OAuth2 Credentials AUTHENTIK_ISSUER: "https://authentik.riotpiao.com/application/o/memory" AUTHENTIK_AUDIENCE: "poimen-memory" AUTHENTIK_CLIENT_ID: "" AUTHENTIK_CLIENT_SECRET: "" # LLM Gateway API Key (optional fallback) LLM_API_KEY: "" # S3/Minio Credentials S3_ACCESS_KEY: "" S3_SECRET_KEY: "" ``` ### 2.3 Encrypt with SOPS: ```bash export SOPS_AGE_KEY_FILE=~/.sops/key.txt cd ~/workplace/Poimen/memory sops -e k8s/app/poimen-memory-secrets.yaml > k8s/app/poimen-memory-secrets.enc.yaml # Verify encryption worked sops -d k8s/app/poimen-memory-secrets.enc.yaml | head -20 ``` ### 2.4 Commit encrypted file only: ```bash git add k8s/app/poimen-memory-secrets.enc.yaml git add .sops.yaml git rm k8s/app/poimen-memory-secrets.yaml # Remove plaintext git commit -m "feat: add SOPS-encrypted Authentik secrets" ``` ## Step 3: Deploy to Kubernetes ### 3.1 Install KSOPS plugin (if using ArgoCD): ```bash # ArgoCD Helm values kustomization: plugins: - name: Kustomize image: ghcr.io/viaduct-ai/kustomize-sops:v4.1.1 ``` ### 3.2 Apply secrets manifest: ```bash # With KSOPS: ArgoCD auto-decrypts and applies # Without KSOPS: Manual decryption before apply export SOPS_AGE_KEY_FILE=~/.sops/key.txt sops -d k8s/app/poimen-memory-secrets.enc.yaml | kubectl apply -f - # Verify secret created kubectl -n poimen get secret poimen-memory-secrets kubectl -n poimen describe secret poimen-memory-secrets ``` ### 3.3 Update deployment envFrom: ```yaml # k8s/app/deployment.yaml spec: template: spec: containers: - name: poimen-memory envFrom: - configMapRef: name: poimen-memory-config - secretRef: name: poimen-memory-secrets # <-- Add this ``` ## Step 4: Entity Extractor JWT Flow ### Code: `crates/mem-ingest/src/entity_extractor.rs` ```rust // Initialization pub struct LlmEntityExtractor { jwt_issuer: Option>>, } impl LlmEntityExtractor { pub fn new(model_name: &str) -> Self { let jwt_issuer = AuthentikJwtIssuer::from_env().ok(); Self { jwt_issuer: jwt_issuer.map(|iss| Arc::new(Mutex::new(iss))), } } } // LLM call with JWT async fn call_llm_endpoint(&self, prompt: &str) -> Result { // Get JWT token from Authentik (cached, auto-refreshed) let auth_header = if let Some(jwt_issuer) = &self.jwt_issuer { let issuer = jwt_issuer.lock().await; let token = issuer.get_access_token().await?; format!("Bearer {}", token) } else { format!("Bearer {}", fallback_api_key) }; // POST to LLM endpoint with JWT client .post(&endpoint) .header("Authorization", auth_header) .json(&payload) .send() .await? } ``` ## Step 5: Runtime Verification ### 5.1 Check JWT token exchange in logs: ```bash kubectl -n poimen logs deployment/poimen-memory | grep -i "authentik\|jwt" # Expected output: # [2026-01-09T20:30:15Z] Obtained Authentik JWT token (expires in 3600 seconds) # [2026-01-09T20:30:15Z] LLM response (via Authentik JWT): {...} ``` ### 5.2 Test entity extraction end-to-end: ```bash # Port-forward to service kubectl -n poimen port-forward svc/poimen-memory 8080:8080 & # Ingest a record curl -X POST http://localhost:8080/memory/ingest \ -H "Content-Type: application/json" \ -d '{ "project": "homelab", "source": "test://jwt", "ingest_id": "jwt-test-001", "records": [{ "role": "architect", "text": "[[Kubernetes]] uses [[Docker]]. [[ArgoCD]] manages deployments.", "timestamp": "2026-01-09T20:30:00Z", "source_position": 0 }] }' # Check logs for JWT usage kubectl -n poimen logs deployment/poimen-memory | tail -20 ``` ## Step 6: Monitoring & Maintenance ### Token Expiry Handling: - JWT tokens are cached with auto-refresh - If token expires during use, new token is fetched automatically - No manual token rotation required ### Credential Rotation: - Rotate Authentik client secret periodically - Update SOPS secret file and re-encrypt - Redeploy pod to pick up new secret ### SOPS Key Rotation (Yearly): ```bash # Generate new age key age-keygen -o ~/.sops/key.txt.new # Re-encrypt all secrets with new key for file in k8s/**/*.enc.yaml; do sops -r $file done # Update ArgoCD to use new key # Commit changes git add k8s/**/*.enc.yaml git commit -m "chore: rotate SOPS encryption keys" ``` ## Troubleshooting ### Issue: "AUTHENTIK_ISSUER not set" **Cause**: Secret not mounted properly **Solution**: `kubectl -n poimen get secret poimen-memory-secrets` ### Issue: "JWT token request failed: 401" **Cause**: Invalid client credentials **Solution**: Verify Client ID/Secret in Authentik, check SOPS decryption ### Issue: "error loading config: no matching creation rules found" **Cause**: SOPS .sops.yaml not configured correctly **Solution**: Use `.sops.yaml` with explicit age key instead of config-based rules ### Issue: "LLM API error: 403 Forbidden" **Cause**: JWT token doesn't have permission to LLM gateway **Solution**: Add RBAC role "LLM User" to service account in Authentik --- ## Files Modified - ✅ `crates/mem-ingest/src/authentik_jwt.rs` — JWT token exchange module - ✅ `crates/mem-ingest/src/entity_extractor.rs` — LLM calls with JWT - ✅ `crates/mem-ingest/src/lib.rs` — Module export - ✅ `k8s/app/poimen-memory-secrets.yaml` — Secret template (plaintext, not committed) - ✅ `k8s/app/poimen-memory-secrets.enc.yaml` — Secret encrypted with SOPS - ✅ `k8s/app/deployment.yaml` — Updated envFrom for secrets - ✅ `k8s/app/config.yaml` — LLM endpoint configuration - ✅ `k8s/.sops.yaml` — SOPS encryption rules