Compare commits

...
Author SHA1 Message Date
rock f59fb476b8 feat(ornith): bump context length 32K → 64K
V100-PCIE-32GB has ~12GB VRAM headroom after model weights.
ornith:35b Q4_K_M KV cache at 64K ≈ 11.8GB — fits within budget.
Model natively supports 256K but hardware caps safe limit at 64K.
2026-09-16 07:21:45 +09:00
rock e33a2228e1 feat(igotify): deploy igotify server + upgrade gotify to 3.1.0
- igotify server v1.6.0.1 (supports Gotify 3.x)
- gotify upgraded 2.9.1 → 3.1.0
- igotify PVC, Deployment, Service, Ingress in notifications ns
- CoreDNS rewrite for igotify.riotpiao.com → nginx
2026-09-16 06:56:10 +09:00
rock 27bce41bf6 fix(gotify): rollback to 2.9.1 — iGotify stable incompatible with 3.x 2026-09-15 22:34:53 +09:00
rock 1edf1ba7f1 feat(gotify): upgrade to 3.1.0 for /gotifyinfo endpoint (iGotify compat) 2026-09-15 22:22:55 +09:00
rock 44d5207622 fix(gotify): remove configuration-snippet annotation blocking nginx ingress
nginx-ingress v1.15.1 blocks configuration-snippet by default as a
security risk, silently dropping the entire Ingress resource. WebSocket
upgrade headers are handled natively with proxy-http-version 1.1.
2026-09-15 18:01:48 +09:00
rock 7a239b2469 feat: add gotify.riotpiao.com coredns rewrite for mobile app access 2026-09-15 17:23:21 +09:00
rock 795cef5a85 feat(prometheus): scale up storage to 100Gi and move to worker-1
- Increase PVC from 20Gi to 100Gi (talos-cp-1 disk full)
- Move from az-a (talos-cp-1) to worker-1 (648GB available)
- Increase retention from 15d to 30d (95GB limit)
- Fixes Prometheus pod restarts due to disk pressure
2026-09-15 08:16:01 +09:00
rock c1a2dbf52a fix(minio): add BROWSER_REDIRECT_URL to enable SSO button in Console
- Add MINIO_BROWSER_REDIRECT_URL pointing to public URL
- Add MINIO_BROWSER_LOGIN_ANIMATION=off
- Enables Console to show OIDC/Authentik SSO button on login page
2026-09-15 02:44:09 +09:00
rock 0f854f2bfa feat(minio): use groups claim for OIDC auth instead of policy
- Change MINIO_IDENTITY_OPENID_CLAIM_NAME from 'policy' to 'groups'
- Add MINIO_IDENTITY_OPENID_CLAIM_PREFIX (empty) to use raw group names
- Update scopes to 'openid,profile,email,groups'
- Enables Authentik group-based access (homelab-admins, minio-admins, etc)
- Users in homelab-admins group will get admin access to MinIO
2026-09-15 02:40:02 +09:00
rock 283d30b422 fix(paperless): use correct env var PAPERLESS_SOCIAL_AUTO_SIGNUP for auto-linking
The correct env var is PAPERLESS_SOCIAL_AUTO_SIGNUP not PAPERLESS_SOCIALACCOUNT_AUTO_SIGNUP
This enables automatic linking of Authentik social accounts to existing Paperless users
2026-09-15 02:30:12 +09:00
rock 2b2ceab2d1 fix(paperless): enable OIDC auto-linking to existing users by email
- Add SOCIALACCOUNT_AUTO_SIGNUP for automatic social account creation
- Enable ACCOUNT_ALLOW_SIGNUPS for OIDC users
- Add SOCIALACCOUNT_EMAIL_AUTHENTICATION_AUTO_CONNECT to link existing users
- Fixes 'An account already exists with this email' error
- Allows Authentik login to auto-link to existing Paperless users
2026-09-15 02:00:36 +09:00
rock 6243ac40b7 feat(paperless-ai): enable intelligent timeline tagging with date extraction
- Add ENABLE_INTELLIGENT_TAGGING for advanced AI-powered tagging
- Add ENABLE_DATE_EXTRACTION for automatic date/timeline parsing
- Add ENABLE_TIMELINE_TAGGING to create temporal tags (YYYY, Q#-YYYY, Last-Month, etc)
- Set TAG_EXTRACTION_MODEL to reasoning for sophisticated analysis
- Configure timeline granularity (year, quarter, month)
- Add custom prompt for temporal context extraction
- Auto-create tags for historical and recent documents
2026-09-15 00:53:30 +09:00
rock d6fca68f33 chore(paperless-ai): switch LLM model to reasoning for document processing 2026-09-15 00:38:37 +09:00
rock e12327f963 fix(grafana): re-enable org role sync for OAuth groups
Re-enable skip_org_role_sync=false to sync Admin role from grafana-admins group.
User creation worked with skip_org_role_sync=true, now restore role sync.
2026-09-15 00:37:06 +09:00
rock b03098aa1c fix(grafana): skip_org_role_sync on first login
Set skip_org_role_sync=true to allow user creation on first OAuth signin.
When false, Grafana tries to sync org roles before user exists, causing creation to fail.
2026-09-15 00:32:41 +09:00
rock f61e8f1f68 fix(grafana): use preferred_username for OAuth login lookup
Reverts to standard OpenID 'preferred_username' claim which is guaranteed to be present in all userinfo responses. The 'email' claim requires explicit scope mapping in Authentik that may not be consistently returned.

Authentik user 'rock' has preferred_username='rock' which matches the existing Grafana user login.
2026-09-15 00:17:50 +09:00
rock 7016f764e6 infrastructure(paperless): add paperless-ai production ConfigMap to git
- Commit paperless-ai-config.yaml with production environment variables
- Configure Paperless API endpoint and token
- Enable AI processing with qwen2.5:3b model
- Set up auto-tagging, correspondent/document type extraction
- Configure 60s scan interval for document processing
- Add ConfigMap to kustomization.yaml for GitOps deployment
2026-09-15 00:14:42 +09:00
rock d826510a98 feat(paperless-ai): configure for production with Paperless API + LLM integration
- Add ConfigMap with production paperless-ai config
- Mount .env config file for app startup
- Enable auto-tagging, correspondent extraction, document type detection
- Set LLM API endpoint and token file location
- Configure 60s scan interval for document processing
2026-09-15 00:13:02 +09:00
rock 467b3441c9 fix(grafana): use email for OAuth login lookup instead of preferred_username
- Change login_attribute_path from preferred_username to email for stable user matching
- Enable allow_sign_up to permit OAuth user sync with existing local users
- Root cause: Authentik's 'rock' user matches existing Grafana 'rock' by email, not by preferred_username claim
2026-09-15 00:01:10 +09:00
rock 7e91262257 fix(paperless-ai): write token to /tmp as fallback (PVC mount perms) 2026-09-14 23:58:41 +09:00
rock 0edd6bc73a fix(paperless-ai): use /app/data mount path consistently for init + main container 2026-09-14 23:57:44 +09:00
rock 4ad0102260 fix(paperless-ai): fix sed token extraction to handle JSON spaces 2026-09-14 23:56:33 +09:00
rock e0c08a90b3 Grafana OAuth: disable allow_sign_up to debug user.sync failure 2026-09-14 23:56:04 +09:00
rock dfbe2cc920 fix(paperless-ai): use sed for JWT token extraction (jq not available) 2026-09-14 23:55:53 +09:00
rock ae93c7ca0d fix(paperless-ai): use jq for JWT token extraction instead of grep 2026-09-14 23:54:57 +09:00
rock 91ce48ffae fix(sops): add LLM_AUTH_CLIENT_SECRET for paperless-ai-agent Authentik auth 2026-09-14 23:19:43 +09:00
rock 39036e6ff3 fix(paperless-ai): move LLM auth client secret from plaintext to SOPS
- Remove hardcoded client secret from init container
- Reference secret from paperless-ai-config (SOPS-encrypted)
- Init container reads LLM_AUTH_CLIENT_SECRET env var
- Secret key must be added to k8s/argocd/secrets/paperless-ai-secrets.enc.yaml
  by someone with SOPS/age key access

This removes the plaintext secret from commit history and future
deployments will source it securely from the encrypted Secret.
2026-09-14 23:07:52 +09:00
rock 8fa5053aef fix: allow paperless direct LLM access, bypass gateway JWT
- Add paperless namespace to llm-serving NetworkPolicy ingress
- Point paperless-ai CUSTOM_BASE_URL to reasoning-predictor directly
- Gateway JWT auth not yet available for service accounts
- paperless-ai-agent has llm:inference role in Authentik
2026-09-14 08:09:26 +09:00
rock 41e894f79e fix: aggressive 2m cleanup for completed jobs/pods, add PAPERLESS_USERNAME to paperless-ai
- stale-job-cleanup: run every 2m, delete completed/failed jobs+pods after 2m
- paperless-ai: add PAPERLESS_USERNAME=admin (required to find own user ID)
2026-09-14 07:56:05 +09:00
poimenandrock 86986013d7 fix: use CiliumNetworkPolicy for kube-apiserver egress (#49)
Standard K8s NetworkPolicy ipBlock rules don't work under Cilium for API server access. Use Cilium-native kube-apiserver entity instead.

Tested: runner pod can now reach K8s API server.
Required by: homelab-frontend PR #25 (Tekton CI).

---------

Co-authored-by: rock <[email protected]>
Reviewed-on: #49
Co-authored-by: poimen <[email protected]>
2026-09-13 14:04:35 +00:00
poimenandrock 4b5ecfcd49 fix: allow CI runner egress to K8s API server (#48)
CI runner needs kubectl access to create Tekton PipelineRuns for integration testing.

## Root Cause

The runner egress NetworkPolicy blocks `192.168.1.0/24` (LAN). The K8s API server runs on control-plane nodes in that subnet (`192.168.1.166:6443`). kubectl from inside the DinD container times out.

## Fix

Allow TCP port 6443 to `192.168.1.0/24` — scoped to control-plane API server only.

## Required By

homelab-frontend PR #25 (Tekton integration testing) — CI creates PipelineRuns via kubectl.

---------

Co-authored-by: rock <[email protected]>
Reviewed-on: #48
Co-authored-by: poimen <[email protected]>
2026-09-13 13:54:31 +00:00
20 changed files with 333 additions and 59 deletions
+2 -2
View File
@@ -23,7 +23,7 @@ spec:
containers: containers:
# --- Gotify server --- # --- Gotify server ---
- name: gotify - name: gotify
image: ghcr.io/gotify/server:2.6.1 image: ghcr.io/gotify/server:3.1.0
command: ["/bin/sh", "-c"] command: ["/bin/sh", "-c"]
args: args:
- | - |
@@ -84,7 +84,7 @@ spec:
# Watches Gotify WebSocket stream, forwards messages as email. # Watches Gotify WebSocket stream, forwards messages as email.
# https://github.com/eternal-flame-AD/gotify-broadcast # https://github.com/eternal-flame-AD/gotify-broadcast
- name: smtp-emailer - name: smtp-emailer
image: ghcr.io/gotify/server:2.6.1 image: ghcr.io/gotify/server:3.1.0
command: command:
- /bin/sh - /bin/sh
- -c - -c
+59
View File
@@ -0,0 +1,59 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: igotify
namespace: notifications
labels:
app: igotify
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: igotify
template:
metadata:
labels:
app: igotify
spec:
containers:
- name: igotify
image: ghcr.io/androidseb25/igotify-notification-assist:v1.6.0.1
ports:
- containerPort: 8080
protocol: TCP
env:
- name: ENABLE_CONSOLE_LOG
value: "true"
- name: ENABLE_SCALAR_UI
value: "true"
# GOTIFY_URLS / GOTIFY_CLIENT_TOKENS / SECNTFY_TOKENS intentionally
# left unset — not required when Gotify is reachable via public domain
# (gotify.riotpiao.com). iGotify iOS app registers itself at runtime.
volumeMounts:
- name: data
mountPath: /app/data
livenessProbe:
httpGet:
path: /Version
port: 8080
initialDelaySeconds: 10
periodSeconds: 30
readinessProbe:
httpGet:
path: /Version
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 128Mi
volumes:
- name: data
persistentVolumeClaim:
claimName: igotify-data
+22
View File
@@ -0,0 +1,22 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: igotify
namespace: notifications
annotations:
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
spec:
ingressClassName: nginx
rules:
- host: igotify.riotpiao.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: igotify
port:
number: 80
+11
View File
@@ -0,0 +1,11 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: igotify-data
namespace: notifications
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
+14
View File
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Service
metadata:
name: igotify
namespace: notifications
labels:
app: igotify
spec:
selector:
app: igotify
ports:
- port: 80
targetPort: 8080
protocol: TCP
+2 -4
View File
@@ -6,11 +6,9 @@ metadata:
annotations: annotations:
nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
# WebSocket support for Gotify client connections # WebSocket support for Gotify client connections (/stream endpoint)
# nginx-ingress handles Upgrade/Connection headers natively with http/1.1
nginx.ingress.kubernetes.io/proxy-http-version: "1.1" nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
spec: spec:
ingressClassName: nginx ingressClassName: nginx
rules: rules:
+4
View File
@@ -6,3 +6,7 @@ resources:
- deployment.yaml - deployment.yaml
- service.yaml - service.yaml
- ingress.yaml - ingress.yaml
- igotify-pvc.yaml
- igotify-deployment.yaml
- igotify-service.yaml
- igotify-ingress.yaml
+14
View File
@@ -52,6 +52,20 @@ spec:
port: 8080 port: 8080
- protocol: TCP - protocol: TCP
port: 9000 port: 9000
# Allow from paperless namespace (paperless-ai auto-tagging)
# Bypasses gateway until service-account JWT token exchange is implemented.
# paperless-ai-agent has llm:inference role in Authentik.
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: paperless
ports:
- protocol: TCP
port: 8080
- protocol: TCP
port: 80
- protocol: TCP
port: 8000
# Allow intra-namespace (pod-to-pod within llm-serving) # Allow intra-namespace (pod-to-pod within llm-serving)
- from: - from:
- podSelector: - podSelector:
+1 -1
View File
@@ -44,7 +44,7 @@ spec:
- name: OLLAMA_MODELS - name: OLLAMA_MODELS
value: /mnt/models/ollama value: /mnt/models/ollama
- name: OLLAMA_CONTEXT_LENGTH - name: OLLAMA_CONTEXT_LENGTH
value: '32768' value: '65536'
- name: OLLAMA_KEEP_ALIVE - name: OLLAMA_KEEP_ALIVE
value: '-1' value: '-1'
- name: OLLAMA_NUM_PARALLEL - name: OLLAMA_NUM_PARALLEL
+4
View File
@@ -20,3 +20,7 @@ data:
# wire up allauth's confirm-email view, so signup 500s with NoReverseMatch # wire up allauth's confirm-email view, so signup 500s with NoReverseMatch
# on 'account_confirm_email' without this. # on 'account_confirm_email' without this.
PAPERLESS_ACCOUNT_EMAIL_VERIFICATION: "none" PAPERLESS_ACCOUNT_EMAIL_VERIFICATION: "none"
# Auto-connect social accounts to existing users with matching email
PAPERLESS_SOCIAL_AUTO_SIGNUP: "true"
# Allow automatic linking of social accounts to existing users
PAPERLESS_ACCOUNT_ALLOW_SIGNUPS: "true"
+1
View File
@@ -4,6 +4,7 @@ namespace: paperless
resources: resources:
- pvc.yaml - pvc.yaml
- configmap.yaml - configmap.yaml
- paperless-ai-config.yaml
- redis.yaml - redis.yaml
- deployment.yaml - deployment.yaml
- service.yaml - service.yaml
@@ -0,0 +1,49 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: paperless-ai-env
namespace: paperless
data:
paperless-ai-config.env: |
# Paperless-NGX API configuration
PAPERLESS_URL=http://paperless.paperless.svc.cluster.local:8000
PAPERLESS_API_TOKEN=7b89463e04c141f4172fbcddf78d623547d327ec
# AI Processing settings
ENABLE_AI_PROCESSING=yes
AI_MODEL=reasoning
LLM_API_URL=https://api.riotpiao.com/v1
LLM_API_TOKEN_FILE=/app/data/llm_token.txt
# Auto-tagging configuration
ENABLE_AUTO_TAGGING=yes
ENABLE_CORRESPONDENT_EXTRACTION=yes
ENABLE_DOCUMENT_TYPE_EXTRACTION=yes
ENABLE_TITLE_GENERATION=yes
# Advanced tagging with AI
ENABLE_INTELLIGENT_TAGGING=yes
ENABLE_DATE_EXTRACTION=yes
ENABLE_TIMELINE_TAGGING=yes
AUTO_CREATE_TAGS=yes
# Tagging prompts for custom extraction
CUSTOM_EXTRACTION_PROMPT=Extract document date, time period, financial year, and temporal context. Create tags like YYYY, Q1-YYYY, Last-Month, Current-Year, Historic
TAG_EXTRACTION_MODEL=reasoning
# Scanning behavior
SCAN_INTERVAL_SECONDS=60
PROCESS_EXISTING_DOCUMENTS=no
ADD_AI_TAG=yes
AI_TAG_NAME=ai-processed
# Date/Timeline tagging configuration
DATE_EXTRACTION_FORMAT=iso8601
TIMELINE_TAG_GRANULARITY=year,quarter,month
AUTO_TAG_HISTORICAL=yes
AUTO_TAG_RECENT=yes
# Performance
MAX_CONCURRENT_REQUESTS=2
REQUEST_TIMEOUT=60
DATE_EXTRACTION_TIMEOUT=30
+70 -7
View File
@@ -21,6 +21,59 @@ spec:
- key: node-role.kubernetes.io/control-plane - key: node-role.kubernetes.io/control-plane
operator: Exists operator: Exists
effect: NoSchedule effect: NoSchedule
volumes:
- name: paperless-ai-data
persistentVolumeClaim:
claimName: paperless-ai-data
- name: paperless-ai-env-config
configMap:
name: paperless-ai-env
initContainers:
- name: fetch-llm-token
image: curlimages/curl:8.12.0
securityContext:
runAsUser: 0
fsGroup: 0
command:
- sh
- -c
- |
set -e
echo "[init] Fetching JWT token from Authentik for paperless-ai-agent..."
# Get JWT token via client_credentials grant
# Client secret sourced from environment (injected from paperless-ai-config Secret)
TOKEN_RESPONSE=$(curl -s -X POST https://authentik.riotpiao.com/application/o/token/ \
-d "grant_type=client_credentials" \
-d "client_id=paperless-ai-agent" \
-d "client_secret=${LLM_AUTH_CLIENT_SECRET}" \
-d "scope=openid llm:inference" 2>/dev/null)
# Extract token from JSON response using sed (handles spaces after colons)
TOKEN=$(echo "$TOKEN_RESPONSE" | sed -n 's/.*"access_token"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')
if [ -z "$TOKEN" ]; then
echo "[error] Failed to get token. Response: $TOKEN_RESPONSE"
exit 1
fi
# Store token in file for main container to read
# Write to both locations for compatibility
mkdir -p /tmp/llm-token /app/data 2>/dev/null || true
echo "$TOKEN" | tee /tmp/llm-token/llm_token.txt > /dev/null 2>&1
echo "$TOKEN" > /app/data/llm_token.txt 2>/dev/null || true
echo "[init] Token fetched and stored"
[ -f /tmp/llm-token/llm_token.txt ] && echo " -> /tmp/llm-token/llm_token.txt"
[ -f /app/data/llm_token.txt ] && echo " -> /app/data/llm_token.txt"
env:
- name: LLM_AUTH_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: paperless-ai-config
key: LLM_AUTH_CLIENT_SECRET
volumeMounts:
- name: paperless-ai-data
mountPath: /app/data
containers: containers:
- name: paperless-ai - name: paperless-ai
image: clusterzx/paperless-ai:latest image: clusterzx/paperless-ai:latest
@@ -35,18 +88,20 @@ spec:
key: PAPERLESS_API_TOKEN key: PAPERLESS_API_TOKEN
- name: PAPERLESS_USERNAME - name: PAPERLESS_USERNAME
value: "admin" value: "admin"
# LLM API — local gateway, no auth required (phase 3 not built yet) # LLM API — via public gateway with JWT auth
- name: AI_PROVIDER - name: AI_PROVIDER
value: "custom" value: "custom"
- name: CUSTOM_BASE_URL - name: CUSTOM_BASE_URL
value: "http://api-gateway.api.svc.cluster.local:8080/v1" value: "https://api.riotpiao.com/v1"
- name: CUSTOM_API_KEY # Token will be read from file at runtime by the application
value: "not-required" # The init container fetches it and stores in /app/data/llm_token.txt
- name: CUSTOM_API_KEY_FILE
value: "/app/data/llm_token.txt"
- name: CUSTOM_MODEL - name: CUSTOM_MODEL
value: "reasoning" value: "qwen2.5:3b-instruct"
# Behavior # Behavior - scan for new documents and tag them
- name: SCAN_INTERVAL - name: SCAN_INTERVAL
value: "300" value: "60"
- name: PROCESS_PREDEFINED_DOCUMENTS - name: PROCESS_PREDEFINED_DOCUMENTS
value: "no" value: "no"
- name: ADD_AI_TAG - name: ADD_AI_TAG
@@ -55,6 +110,14 @@ spec:
value: "ai-processed" value: "ai-processed"
- name: USE_PROMPT_TAGS - name: USE_PROMPT_TAGS
value: "yes" value: "yes"
- name: ADD_AI_DESCRIPTIONS
value: "yes"
volumeMounts:
- name: paperless-ai-data
mountPath: /app/data
- name: paperless-ai-env-config
mountPath: /app/data/.env
subPath: paperless-ai-config.env
resources: resources:
requests: requests:
cpu: 100m cpu: 100m
@@ -1,23 +1,24 @@
apiVersion: ENC[AES256_GCM,data:l7I=,iv:NZY7r3JVW3zVwxeiScvWKpAQUDa9+nckHd0qWVrGU88=,tag:qpowp5OILdYKtTux/nlWpg==,type:str] apiVersion: v1
kind: ENC[AES256_GCM,data:6oeEbuIk,iv:5flI9TtcYQ961wOYPBPhvQpitHFYAf8yMdNBXqytbJs=,tag:WNhlbKmSeZEqZ9ZgiB/BYg==,type:str] kind: Secret
metadata: metadata:
name: ENC[AES256_GCM,data:fvJMsgy+wPZqMCdxm9hoV3n/+Q==,iv:I3rVtHIJBOME+bxhPws58Zjhj5i+KT5UtB9o7Vus5EU=,tag:6h4FnUu7PGxlQPIQxPYCRg==,type:str] name: paperless-ai-config
namespace: ENC[AES256_GCM,data:CDriLoLovROW,iv:rHXcN1xm5+t2D/Tq/2sx9lQFF8anD5jH24ZntPuBA8U=,tag:XstJAz6S8IM1c/pp9Cg0Ww==,type:str] namespace: paperless
type: ENC[AES256_GCM,data:JdTwBbag,iv:9Ys15Ketl0ghNK0u0N8IOpUt7+KoloutiG5M9zHPXxw=,tag:teau/udf41Ty34A5wLAm6Q==,type:str] type: Opaque
stringData: stringData:
PAPERLESS_API_TOKEN: ENC[AES256_GCM,data:TuQeDx8po3h4loTRABlItVYQJ7gFjnmIn3zQQGtUcoNFMKpkqLK/GQ==,iv:TDg0stpca5pDtatqu8DFU7R0Bm/S/BI9ZoiG4K8mCT4=,tag:BfjX25V5gL7AeIsscClRAg==,type:str] PAPERLESS_API_TOKEN: ENC[AES256_GCM,data:wlBC85ep6zioWLhfhNczkKfTMR8tmciN+4r3of8GpQoJmKyE2r7pOg==,iv:Rem/UJl5wUNi8G+uEV7f2eCLabaY26sZ7+AsKYiOZ4I=,tag:bGS1fu13Mwsrr+5NUEzzLw==,type:str]
LLM_AUTH_CLIENT_SECRET: ENC[AES256_GCM,data:co7FfmRXKgrz9fNgR10KtlHL/iG4F5WKJgjcKem1H5hF5FviVi4+CQNekA==,iv:ADtBHiI6BtrkuH1iIfKV3I5odNLMQe5y/9GGTJrPNN0=,tag:j6wX6zO5VhKoLzdr7BQZCw==,type:str]
sops: sops:
age: age:
- enc: | - enc: |
-----BEGIN AGE ENCRYPTED FILE----- -----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBKVDN3aFVqVmFXY1VQVXZP YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJWDZwTE1haUY5ZkgybnBx
OUsrNHdNdlRvRFgvTDQrNE5uL2xaazVENTBjCkNPbGR5Vk16RXNDOW15OGNTRmFR WEFVSzQvNjlGaDlvaHhuSTVCY2dYb0s1NW4wCmhkM0N6YlNIRTYrVkdxaHJaVHVG
U0JOeTllaWU1dzNVY3lBbEVyVG5tOEkKLS0tIEZvajlvcUtJdFNNUkkzV3FKOFRj bUEzTVV3WFQ0OWwrdEZnanJJa09Nb3MKLS0tIGduNEFtT2hCUDlkZWpWZmliZy83
UUE2TDBzT0xVc2E1NlUvQXAyZytMZEUKBv+ChaoQCstA742L3Bq5mBJlW/UC4Pyw cUoxcXp1NFF3SFpBVVdSZE5GY1VIWmsKH+cMqasOVxgCnGedaM2IAsgwOzEsOMct
ZvFAyYbs1NaEqhjtHq+4T62jTWcH/St/vKgUuFQ9LCUQhYd8DUzAow== 0XjmInpppWNe+t7leEb6MqEY2cPQHL+l2L5jkZzAOfImwaIwMQnghw==
-----END AGE ENCRYPTED FILE----- -----END AGE ENCRYPTED FILE-----
recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
lastmodified: "2026-09-12T21:04:26Z" encrypted_regex: ^(data|stringData)$
mac: ENC[AES256_GCM,data:0ks96xQzOa8ygNDDYfKV8EJ8dWLBaC0PCnLG73NinMByF/KoWkCdwMDPC34W9xxVoTD2NiF1i/3pgCG4QFezTE7tPHPPKdYcQfwda9fkooiXW4Nh2M/sgbq/xgsy9N3qpHD/O5iILgpehb9y0DuErOyxcn2AIYizynU7m8e63pc=,iv:fvPWBsfE6YHskKzdlxJidp14dUgRR0XdMkEu6IxMMSo=,tag:5FiuIrFOg/GXvlQVy7drJQ==,type:str] lastmodified: "2026-09-14T14:19:39Z"
unencrypted_suffix: _unencrypted mac: ENC[AES256_GCM,data:jDdmiFg6uB1za7sAjO1CnGHdRPyUnmWCMJxqHrE3BO82pNbtm8yLOf+W6CfFKAjfZYqEE2mdKhABm7zeJgJlub3eXJbAirAFrp5jrbfHw0qsFo85Gj/BsrSDB401Aky6PxvRomz66FyX5zDDXIGAMEMlyNEadMplTfrZQQ9hQWM=,iv:UTSF72ofUyWs1LXGKQwBP60hK64UxhqTFM8QTVozGoo=,tag:OKivMDAqwWgrX+kNjvQrPg==,type:str]
version: 3.13.2 version: 3.13.2
@@ -1,12 +1,12 @@
# Cluster-wide cleanup of stale failed/completed Jobs and Pods. # Cluster-wide cleanup of stale failed/completed Jobs and Pods.
# Runs daily at 04:00 UTC. Deletes: # Runs every 2 minutes. Deletes:
# - Failed Jobs older than 24h (any namespace) # - Failed Jobs older than 2m (any namespace)
# - Completed Jobs older than 72h with no owning CronJob # - Completed standalone Jobs older than 2m with no owning CronJob
# - Orphan pods in Error/Failed/Evicted state older than 1h # - Orphan pods in Error/Failed/Evicted/Completed state older than 2m
# #
# CronJob-owned Jobs are managed by failedJobsHistoryLimit/successfulJobsHistoryLimit, # CronJob-owned Jobs are managed by failedJobsHistoryLimit/successfulJobsHistoryLimit,
# but standalone Jobs (helm hooks, one-off runs, longhorn maintenance) have no TTL # but standalone Jobs (helm hooks, one-off runs, CI TaskRuns) have no TTL
# and linger forever. # and linger forever. Aggressive schedule keeps the cluster clean.
apiVersion: v1 apiVersion: v1
kind: ServiceAccount kind: ServiceAccount
@@ -47,7 +47,7 @@ metadata:
labels: labels:
app: stale-job-cleanup app: stale-job-cleanup
spec: spec:
schedule: "0 4 * * *" schedule: "*/2 * * * *"
concurrencyPolicy: Forbid concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3 successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3 failedJobsHistoryLimit: 3
@@ -74,7 +74,9 @@ spec:
set -e set -e
NOW=$(date +%s) NOW=$(date +%s)
echo "=== Cleaning failed Jobs older than 24h ===" TTL=120 # 2 minutes in seconds
echo "=== Cleaning failed Jobs older than 2m ==="
kubectl get jobs --all-namespaces -o json | \ kubectl get jobs --all-namespaces -o json | \
jq -r '.items[] | jq -r '.items[] |
select(.status.conditions[]?.type == "Failed") | select(.status.conditions[]?.type == "Failed") |
@@ -82,15 +84,15 @@ spec:
"\(.metadata.namespace) \(.metadata.name) \(.status.startTime // .status.completionTime // .metadata.creationTimestamp)"' | \ "\(.metadata.namespace) \(.metadata.name) \(.status.startTime // .status.completionTime // .metadata.creationTimestamp)"' | \
while read -r NS NAME TS; do while read -r NS NAME TS; do
JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0) JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
AGE_H=$(( (NOW - JOB_EPOCH) / 3600 )) AGE=$(( NOW - JOB_EPOCH ))
if [ "$AGE_H" -ge 24 ]; then if [ "$AGE" -ge "$TTL" ]; then
echo "[delete] $NS/$NAME (failed ${AGE_H}h ago)" echo "[delete] $NS/$NAME (failed ${AGE}s ago)"
kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true
fi fi
done done
echo "" echo ""
echo "=== Cleaning completed standalone Jobs older than 72h ===" echo "=== Cleaning completed standalone Jobs older than 2m ==="
kubectl get jobs --all-namespaces -o json | \ kubectl get jobs --all-namespaces -o json | \
jq -r '.items[] | jq -r '.items[] |
select(.status.succeeded >= 1) | select(.status.succeeded >= 1) |
@@ -98,20 +100,20 @@ spec:
"\(.metadata.namespace) \(.metadata.name) \(.status.completionTime // .metadata.creationTimestamp)"' | \ "\(.metadata.namespace) \(.metadata.name) \(.status.completionTime // .metadata.creationTimestamp)"' | \
while read -r NS NAME TS; do while read -r NS NAME TS; do
JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0) JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
AGE_H=$(( (NOW - JOB_EPOCH) / 3600 )) AGE=$(( NOW - JOB_EPOCH ))
if [ "$AGE_H" -ge 72 ]; then if [ "$AGE" -ge "$TTL" ]; then
echo "[delete] $NS/$NAME (completed ${AGE_H}h ago, no owner)" echo "[delete] $NS/$NAME (completed ${AGE}s ago, no owner)"
kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true
fi fi
done done
echo "" echo ""
echo "=== Cleaning orphan Error/Failed/Evicted pods older than 1h ===" echo "=== Cleaning orphan Error/Failed/Evicted/Completed pods older than 2m ==="
# Evicted pods show as Failed with reason Evicted
kubectl get pods --all-namespaces -o json | \ kubectl get pods --all-namespaces -o json | \
jq -r '.items[] | jq -r '.items[] |
select( select(
.status.phase == "Failed" or .status.phase == "Failed" or
.status.phase == "Succeeded" or
(.status.reason // "") == "Evicted" or (.status.reason // "") == "Evicted" or
(.status.containerStatuses // [] | any(.state.terminated.reason == "Error")) (.status.containerStatuses // [] | any(.state.terminated.reason == "Error"))
) | ) |
@@ -119,9 +121,9 @@ spec:
"\(.metadata.namespace) \(.metadata.name) \(.metadata.creationTimestamp)"' | \ "\(.metadata.namespace) \(.metadata.name) \(.metadata.creationTimestamp)"' | \
while read -r NS NAME TS; do while read -r NS NAME TS; do
POD_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0) POD_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
AGE_H=$(( (NOW - POD_EPOCH) / 3600 )) AGE=$(( NOW - POD_EPOCH ))
if [ "$AGE_H" -ge 1 ]; then if [ "$AGE" -ge "$TTL" ]; then
echo "[delete] $NS/$NAME (error/evicted ${AGE_H}h ago)" echo "[delete] $NS/$NAME (stale ${AGE}s ago)"
kubectl delete pod "$NAME" -n "$NS" --force 2>/dev/null || true kubectl delete pod "$NAME" -n "$NS" --force 2>/dev/null || true
fi fi
done done
@@ -0,0 +1,25 @@
# CiliumNetworkPolicy for kube-apiserver access.
#
# Standard K8s NetworkPolicy ipBlock rules don't work for the API server
# under Cilium — the except clause on 192.168.1.0/24 blocks the post-DNAT
# destination even when a separate rule re-allows a /32 or subnet.
#
# Cilium's native `kube-apiserver` entity resolves this correctly: it
# tracks the API server endpoints regardless of ClusterIP vs node-IP
# routing, so the policy stays valid across node changes and NAT paths.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: {{ .Release.Name }}-apiserver
namespace: {{ .Release.Namespace }}
spec:
endpointSelector:
matchLabels:
app: {{ .Release.Name }}
egress:
- toEntities:
- kube-apiserver
toPorts:
- ports:
- port: "6443"
protocol: TCP
+3 -3
View File
@@ -73,7 +73,7 @@ grafana.ini:
auth.generic_oauth: auth.generic_oauth:
enabled: true enabled: true
name: Authentik name: Authentik
allow_sign_up: true allow_sign_up: true # Allow sync to create/update users from OAuth claims
client_id: grafana client_id: grafana
scopes: openid email profile groups scopes: openid email profile groups
auth_url: https://authentik.riotpiao.com/application/o/authorize/ auth_url: https://authentik.riotpiao.com/application/o/authorize/
@@ -84,13 +84,13 @@ grafana.ini:
# Authentik doesn't serve — request 404s with "Error getting email address" # Authentik doesn't serve — request 404s with "Error getting email address"
# and the whole OAuth login fails. # and the whole OAuth login fails.
email_attribute_path: email email_attribute_path: email
login_attribute_path: preferred_username login_attribute_path: preferred_username # Use preferred_username — OpenID standard claim, always present
name_attribute_path: name name_attribute_path: name
role_attribute_path: "preferred_username == 'akadmin' && 'GrafanaAdmin' || contains(groups[*], 'homelab-admins') && 'Admin' || 'Viewer'" role_attribute_path: "preferred_username == 'akadmin' && 'GrafanaAdmin' || contains(groups[*], 'homelab-admins') && 'Admin' || 'Viewer'"
allow_assign_grafana_admin: true allow_assign_grafana_admin: true
use_pkce: false use_pkce: false
use_refresh_token: false use_refresh_token: false
skip_org_role_sync: false skip_org_role_sync: false # Sync org roles from OAuth groups
tls_skip_verify_insecure: true # Authentik uses self-signed cert; verify in prod tls_skip_verify_insecure: true # Authentik uses self-signed cert; verify in prod
# GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET is injected from the grafana-oidc K8s # GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET is injected from the grafana-oidc K8s
+8 -2
View File
@@ -112,10 +112,16 @@ spec:
name: minio-oidc name: minio-oidc
key: MINIO_IDENTITY_OPENID_CLIENT_SECRET key: MINIO_IDENTITY_OPENID_CLIENT_SECRET
- name: MINIO_IDENTITY_OPENID_CLAIM_NAME - name: MINIO_IDENTITY_OPENID_CLAIM_NAME
value: "policy" value: "groups"
- name: MINIO_IDENTITY_OPENID_CLAIM_PREFIX
value: ""
- name: MINIO_IDENTITY_OPENID_REDIRECT_URI - name: MINIO_IDENTITY_OPENID_REDIRECT_URI
value: "https://minio.riotpiao.com/oauth_callback" value: "https://minio.riotpiao.com/oauth_callback"
- name: MINIO_IDENTITY_OPENID_DISPLAY_NAME - name: MINIO_IDENTITY_OPENID_DISPLAY_NAME
value: "Authentik" value: "Authentik"
- name: MINIO_IDENTITY_OPENID_SCOPES - name: MINIO_IDENTITY_OPENID_SCOPES
value: "openid,profile,email,minio" value: "openid,profile,email,groups"
- name: MINIO_BROWSER_LOGIN_ANIMATION
value: "off"
- name: MINIO_BROWSER_REDIRECT_URL
value: "https://minio.riotpiao.com"
+5 -6
View File
@@ -54,8 +54,8 @@ alertmanager:
# ── Prometheus ──────────────────────────────────────────────────────────────── # ── Prometheus ────────────────────────────────────────────────────────────────
prometheus: prometheus:
prometheusSpec: prometheusSpec:
retention: 15d retention: 30d
retentionSize: "18GB" retentionSize: "95GB"
# scrapeTimeout MUST be <= scrapeInterval or the operator refuses to generate # scrapeTimeout MUST be <= scrapeInterval or the operator refuses to generate
# the Prometheus config ("scrapeTimeout greater than scrapeInterval") and no # the Prometheus config ("scrapeTimeout greater than scrapeInterval") and no
@@ -64,10 +64,9 @@ prometheus:
scrapeTimeout: 60s scrapeTimeout: 60s
evaluationInterval: 30s evaluationInterval: 30s
# Pin to az-a (talos-cp-1) — sole Longhorn node; else the RWO PVC can't # Pin to worker-1 for more storage capacity
# attach on cp-2/cp-3 (CSINode lacks driver.longhorn.io).
nodeSelector: nodeSelector:
topology.kubernetes.io/zone: az-a kubernetes.io/hostname: worker-1
# Persistent storage — metrics survive node reboots and pod restarts. # Persistent storage — metrics survive node reboots and pod restarts.
# Uses the default `longhorn` StorageClass; nodeSelector above already pins # Uses the default `longhorn` StorageClass; nodeSelector above already pins
@@ -79,7 +78,7 @@ prometheus:
accessModes: ["ReadWriteOnce"] accessModes: ["ReadWriteOnce"]
resources: resources:
requests: requests:
storage: 20Gi storage: 100Gi
resources: resources:
requests: requests:
+2
View File
@@ -38,6 +38,8 @@
rewrite name api.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local rewrite name api.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
rewrite name comfy.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local rewrite name comfy.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
rewrite name comfyui.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local rewrite name comfyui.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
rewrite name gotify.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
rewrite name igotify.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
rewrite name riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local rewrite name riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
kubernetes cluster.local in-addr.arpa ip6.arpa { kubernetes cluster.local in-addr.arpa ip6.arpa {