fix(gotify): move SOPS secrets to ksops generator

ArgoCD couldn't decrypt secrets.yaml because it was listed as a plain kustomize resource. Move the 3 secrets (gotify-admin, gotify-tokens, gotify-smtp) to k8s/argocd/secrets/ as .enc.yaml files processed by the ksops generator, matching the repo convention. Fixes ComparisonError: 'Object Kind is missing' (SOPS ciphertext parsed as raw YAML).
fix(argocd): update git repoURLs for org transfer rock -> riotpiao-poimen
2026-09-10 22:32:24 +09:00 · 2026-09-10 10:49:30 +09:00 · 2026-09-10 01:41:37 +00:00 · 2026-09-10 01:35:59 +00:00 · 2026-09-09 22:38:25 +00:00 · 2026-09-09 02:11:28 +00:00
41 changed files with 856 additions and 251 deletions
@@ -0,0 +1,80 @@
+# ComfyUI — GPU-accelerated image generation on worker-1.
+# Uses 1x V100 32GB (sm70). Freed by scaling ornith 2→1.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: comfyui
+  namespace: comfyui
+  labels:
+    app: comfyui
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: comfyui
+  template:
+    metadata:
+      labels:
+        app: comfyui
+    spec:
+      nodeSelector:
+        kubernetes.io/hostname: worker-1
+      runtimeClassName: nvidia
+      containers:
+        - name: comfyui
+          image: ghcr.io/ai-dock/comfyui:v2-cuda-12.1.1-base-22.04
+          ports:
+            - containerPort: 8188
+              protocol: TCP
+          env:
+            - name: NVIDIA_VISIBLE_DEVICES
+              value: "all"
+            - name: COMFYUI_FLAGS
+              value: "--listen 0.0.0.0 --port 8188"
+          resources:
+            requests:
+              cpu: "4"
+              memory: 8Gi
+              nvidia.com/gpu: "1"
+            limits:
+              cpu: "8"
+              memory: 16Gi
+              nvidia.com/gpu: "1"
+          volumeMounts:
+            - mountPath: /workspace/ComfyUI/models
+              name: models
+            - mountPath: /workspace/ComfyUI/output
+              name: output
+          readinessProbe:
+            httpGet:
+              path: /
+              port: 8188
+            periodSeconds: 10
+            initialDelaySeconds: 30
+          startupProbe:
+            httpGet:
+              path: /
+              port: 8188
+            failureThreshold: 60
+            periodSeconds: 10
+      volumes:
+        - name: models
+          persistentVolumeClaim:
+            claimName: comfyui-models
+        - name: output
+          emptyDir: {}
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: comfyui-models
+  namespace: comfyui
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: longhorn
+  resources:
+    requests:
+      storage: 50Gi
@@ -0,0 +1,25 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: comfyui
+  namespace: comfyui
+  annotations:
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
+    nginx.ingress.kubernetes.io/proxy-body-size: "0"
+    # WebSocket support for ComfyUI's live preview
+    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
+    nginx.ingress.kubernetes.io/proxy-set-headers: "Upgrade"
+spec:
+  ingressClassName: nginx
+  rules:
+    - host: comfy.riotpiao.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: comfyui
+                port:
+                  number: 80
@@ -0,0 +1,7 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - deployment.yaml
+  - service.yaml
+  - ingress.yaml
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: comfyui
+  namespace: comfyui
+  labels:
+    app: comfyui
+spec:
+  selector:
+    app: comfyui
+  ports:
+    - port: 80
+      targetPort: 8188
+      protocol: TCP
@@ -0,0 +1,107 @@
+# Gotify — Push Notifications + Email Relay
+
+Self-hosted notification server with SMTP email forwarding sidecar.
+
+## Architecture
+
+```
+Forgejo webhook ──POST──→ Gotify API (:80/message)
+                              │
+                    ┌─────────┼─────────┐
+                    ▼                     ▼
+              Push notification      SMTP emailer sidecar
+              (mobile/desktop)       (polls → sends email)
+```
+
+## Setup (one-time, after first deploy)
+
+### 1. Encrypt secrets before committing
+
+```bash
+# Edit secrets.yaml with real values first, then:
+sops -e -i k8s/apps/gotify/secrets.yaml
+```
+
+### 2. Create Gotify app + client tokens
+
+1. Login to `https://gotify.riotpiao.com` with admin creds
+2. **Applications** → Create `forgejo` → copy **app token**
+3. **Clients** → Create `smtp-emailer` → copy **client token**
+4. Update `gotify-tokens` secret:
+   ```bash
+   kubectl -n notifications create secret generic gotify-tokens \
+     --from-literal=app-token=<APP_TOKEN> \
+     --from-literal=client-token=<CLIENT_TOKEN> \
+     --dry-run=client -o yaml | kubectl apply -f -
+   ```
+
+### 3. Configure Forgejo webhook
+
+In each Forgejo repo → **Settings** → **Webhooks** → **Add Webhook** → **Gotify**:
+
+| Field | Value |
+|-------|-------|
+| Target URL | `http://gotify.notifications.svc.cluster.local/message` |
+| Token | The **app token** from step 2 |
+| Events | Pull Request (Created, Merged, Closed) |
+
+Or via API:
+```bash
+FORGEJO_TOKEN="<your-pat>"
+APP_TOKEN="<gotify-app-token>"
+
+curl -s -X POST "https://forgejo.riotpiao.com/api/v1/repos/rock/homelab/hooks" \
+  -H "Authorization: token $FORGEJO_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "type": "gotify",
+    "active": true,
+    "config": {
+      "content_type": "json",
+      "url": "http://gotify.notifications.svc.cluster.local/message?token='"$APP_TOKEN"'"
+    },
+    "events": ["pull_request", "pull_request_assign", "pull_request_review"],
+    "authorization_header": ""
+  }'
+```
+
+### 4. Add CoreDNS rewrite (if accessing via public hostname)
+
+Only needed if Cloudflare Tunnel is used for gotify.riotpiao.com:
+
+```
+# terraform/files/coredns/Corefile — add rewrite:
+rewrite name gotify.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
+```
+
+Then: `cd terraform && terraform apply && cd .. && make apply-cp`
+
+### 5. SMTP providers
+
+| Provider | Host | Port | Notes |
+|----------|------|------|-------|
+| Gmail | smtp.gmail.com | 587 | Use App Password (2FA required) |
+| Resend | smtp.resend.com | 587 | Free 100 emails/day |
+| Sendgrid | smtp.sendgrid.net | 587 | Free 100 emails/day |
+| Mailgun | smtp.mailgun.org | 587 | Free 5000/month |
+
+## Notification priority levels
+
+| Priority | Meaning | Email forwarded? |
+|----------|---------|-----------------|
+| 0-4 | Low (info) | No (below MIN_PRIORITY=5) |
+| 5-7 | Normal (PR created) | Yes |
+| 8-10 | High (PR merged, failures) | Yes |
+
+## Verify
+
+```bash
+# Test push notification
+APP_TOKEN="<app-token>"
+curl -X POST "https://gotify.riotpiao.com/message?token=$APP_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"title":"Test","message":"Hello from homelab","priority":5}'
+
+# Check email sidecar logs
+kubectl -n notifications logs deployment/gotify -c smtp-emailer --tail=20
+```
@@ -0,0 +1,35 @@
+# CNPG Postgres for Gotify. Lightweight — 2 instances, 2Gi storage.
+# CNPG generates secret `gotify-db-app` + service `gotify-db-rw` in ns notifications.
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: gotify-db
+  namespace: notifications
+  annotations:
+    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
+spec:
+  instances: 2
+  imageName: ghcr.io/cloudnative-pg/postgresql:16.2
+  bootstrap:
+    initdb:
+      database: gotify
+      owner: app
+      encoding: UTF8
+      localeCollate: C
+      localeCType: C
+  enableSuperuserAccess: false
+  resources:
+    requests: { memory: "256Mi", cpu: "100m" }
+    limits: { memory: "512Mi", cpu: "500m" }
+  storage:
+    size: 2Gi
+    storageClass: longhorn-cnpg
+  monitoring:
+    enablePodMonitor: true
+  affinity:
+    podAntiAffinityType: preferred
+    topologyKey: kubernetes.io/hostname
+    tolerations:
+      - key: node-role.kubernetes.io/control-plane
+        operator: Exists
+        effect: NoSchedule
@@ -0,0 +1,204 @@
+# Gotify — self-hosted push notification server + SMTP email relay.
+# Forgejo webhooks → Gotify → push notifications + email forwarding.
+# Runs on control plane (no GPU needed), lightweight.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: gotify
+  namespace: notifications
+  labels:
+    app: gotify
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: gotify
+  template:
+    metadata:
+      labels:
+        app: gotify
+    spec:
+      containers:
+        # --- Gotify server ---
+        - name: gotify
+          image: ghcr.io/gotify/server:2.6.1
+          command: ["/bin/sh", "-c"]
+          args:
+            - |
+              export GOTIFY_DATABASE_DIALECT=postgres
+              export GOTIFY_DATABASE_CONNECTION="host=gotify-db-rw.notifications port=5432 user=${DB_USER} password=${DB_PASS} dbname=gotify sslmode=disable"
+              exec /app/gotify-app
+          ports:
+            - containerPort: 80
+              protocol: TCP
+          env:
+            - name: GOTIFY_DEFAULTUSER_NAME
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-admin
+                  key: username
+            - name: GOTIFY_DEFAULTUSER_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-admin
+                  key: password
+            - name: DB_USER
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-db-app
+                  key: username
+            - name: DB_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-db-app
+                  key: password
+            - name: GOTIFY_SERVER_PORT
+              value: "80"
+            - name: GOTIFY_SERVER_KEEPALIVEPERIODSECONDS
+              value: "0"
+            - name: TZ
+              value: Asia/Tokyo
+          resources:
+            requests:
+              cpu: 50m
+              memory: 64Mi
+            limits:
+              cpu: 200m
+              memory: 128Mi
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: 80
+            periodSeconds: 30
+            initialDelaySeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: 80
+            periodSeconds: 10
+            initialDelaySeconds: 5
+
+        # --- SMTP emailer sidecar ---
+        # Watches Gotify WebSocket stream, forwards messages as email.
+        # https://github.com/eternal-flame-AD/gotify-broadcast
+        - name: smtp-emailer
+          image: ghcr.io/gotify/server:2.6.1
+          command:
+            - /bin/sh
+            - -c
+            - |
+              # Wait for Gotify to be ready
+              until wget -qO- http://localhost:80/health >/dev/null 2>&1; do
+                echo "Waiting for Gotify..."
+                sleep 2
+              done
+              echo "Gotify is ready, starting email relay..."
+
+              # Poll Gotify messages and forward via SMTP using msmtp
+              # Install msmtp for lightweight SMTP sending
+              apk add --no-cache msmtp curl jq
+
+              # Configure msmtp
+              cat > /tmp/msmtprc <<MSMTP
+              defaults
+              auth           on
+              tls            on
+              tls_trust_file /etc/ssl/certs/ca-certificates.crt
+              logfile        /tmp/msmtp.log
+
+              account        default
+              host           ${SMTP_HOST}
+              port           ${SMTP_PORT}
+              from           ${SMTP_FROM}
+              user           ${SMTP_USER}
+              password       ${SMTP_PASS}
+              MSMTP
+
+              chmod 600 /tmp/msmtprc
+
+              # Track last seen message ID
+              LAST_ID=0
+
+              while true; do
+                # Fetch messages since last ID
+                MESSAGES=$(curl -s -H "X-Gotify-Key: ${GOTIFY_CLIENT_TOKEN}" \
+                  "http://localhost:80/message?since=${LAST_ID}&limit=10" 2>/dev/null)
+
+                if [ -n "$MESSAGES" ]; then
+                  echo "$MESSAGES" | jq -r '.messages[]? | @base64' | while read -r MSG; do
+                    DECODED=$(echo "$MSG" | base64 -d)
+                    ID=$(echo "$DECODED" | jq -r '.id')
+                    TITLE=$(echo "$DECODED" | jq -r '.title // "Notification"')
+                    BODY=$(echo "$DECODED" | jq -r '.message // ""')
+                    PRIORITY=$(echo "$DECODED" | jq -r '.priority // 5')
+                    APP=$(echo "$DECODED" | jq -r '.appid // 0')
+                    DATE=$(echo "$DECODED" | jq -r '.date // ""')
+
+                    # Only forward messages with priority >= configured threshold
+                    if [ "$PRIORITY" -ge "${MIN_PRIORITY:-0}" ]; then
+                      printf "Subject: [Gotify] %s\nFrom: %s\nTo: %s\nContent-Type: text/plain; charset=UTF-8\n\n%s\n\n---\nPriority: %s\nDate: %s" \
+                        "$TITLE" "$SMTP_FROM" "$NOTIFY_EMAIL" "$BODY" "$PRIORITY" "$DATE" | \
+                        msmtp -C /tmp/msmtprc "$NOTIFY_EMAIL" && \
+                        echo "Email sent for message $ID: $TITLE" || \
+                        echo "Failed to send email for message $ID"
+                    fi
+
+                    # Update last seen ID
+                    if [ "$ID" -gt "$LAST_ID" ]; then
+                      LAST_ID=$ID
+                    fi
+                  done
+                fi
+
+                sleep ${POLL_INTERVAL:-30}
+              done
+          env:
+            - name: GOTIFY_CLIENT_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-tokens
+                  key: client-token
+            - name: SMTP_HOST
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: host
+            - name: SMTP_PORT
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: port
+            - name: SMTP_FROM
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: from
+            - name: SMTP_USER
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: user
+            - name: SMTP_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: password
+            - name: NOTIFY_EMAIL
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: notify-email
+            - name: MIN_PRIORITY
+              value: "5"
+            - name: POLL_INTERVAL
+              value: "15"
+          resources:
+            requests:
+              cpu: 10m
+              memory: 32Mi
+            limits:
+              cpu: 100m
+              memory: 64Mi
+      # No volumes — Postgres handles persistence
@@ -0,0 +1,26 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: gotify
+  namespace: notifications
+  annotations:
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
+    # WebSocket support for Gotify client connections
+    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
+    nginx.ingress.kubernetes.io/configuration-snippet: |
+      proxy_set_header Upgrade $http_upgrade;
+      proxy_set_header Connection "upgrade";
+spec:
+  ingressClassName: nginx
+  rules:
+    - host: gotify.riotpiao.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: gotify
+                port:
+                  number: 80
@@ -0,0 +1,8 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - namespace.yaml
+  - db.yaml
+  - deployment.yaml
+  - service.yaml
+  - ingress.yaml
@@ -0,0 +1,6 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: notifications
+  labels:
+    kubernetes.io/metadata.name: notifications
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: gotify
+  namespace: notifications
+  labels:
+    app: gotify
+spec:
+  selector:
+    app: gotify
+  ports:
+    - port: 80
+      targetPort: 80
+      protocol: TCP
@@ -14,6 +14,7 @@ resources:
  - ornith.yaml
  - reasoning.yaml
  - reranker.yaml
+  - qwen-cpu.yaml
  - networkpolicy.yaml
 # No namespace transformer: every file sets its own, and the transformer would
 # rewrite metadata.namespace on anything cross-namespace added later.
@@ -33,12 +33,8 @@ spec:

        ollama pull ornith:35b

-        ollama pull qwen2.5:3b-instruct
-
        ollama run ornith:35b "ok" >/dev/null 2>&1 || true

-        ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
-
        wait $SERVE_PID

        '
@@ -54,7 +50,7 @@ spec:
      - name: OLLAMA_NUM_PARALLEL
        value: '1'
      - name: OLLAMA_MAX_LOADED_MODELS
-        value: '2'
+        value: '1'
      image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
      name: kserve-container
      ports:
@@ -65,8 +61,7 @@ spec:
          command:
          - /bin/sh
          - -c
-          - ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
-            grep -q qwen2.5
+          - ollama ps 2>/dev/null | grep -q ornith
        periodSeconds: 10
      resources:
        limits:
@@ -82,8 +77,7 @@ spec:
          command:
          - /bin/sh
          - -c
-          - ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
-            grep -q qwen2.5
+          - ollama ps 2>/dev/null | grep -q ornith
        failureThreshold: 120
        periodSeconds: 15
      volumeMounts:
@@ -91,13 +85,10 @@ spec:
        name: models
    deploymentStrategy:
      type: Recreate
-    # 2 replicas -- each its own GPU, each loading both ornith:35b and
-    # qwen2.5:3b-instruct -- so 2 concurrent implementer-style calls each
-    # get an independent instance instead of contending on one, at the
-    # cost of judge/qwen traffic still sharing whichever replica an
-    # implementer call also lands on.
-    maxReplicas: 2
-    minReplicas: 2
+    # 1 replica -- ornith:35b only. qwen2.5:3b moved to CPU on cp-2.
+    # Frees 1 GPU for ComfyUI.
+    maxReplicas: 1
+    minReplicas: 1
    nodeSelector:
      kubernetes.io/hostname: worker-1
    runtimeClassName: nvidia
@@ -0,0 +1,115 @@
+# qwen2.5:3b-instruct on CPU (talos-cp-2, 144GB RAM, 24 cores).
+# Moved off GPU to free a V100 for ComfyUI. Latency ~10x slower
+# than GPU but sufficient for lightweight tasks (summarization,
+# classification, quick answers).
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: qwen-cpu
+  namespace: llm-serving
+  labels:
+    app: qwen-cpu
+    app.kubernetes.io/name: qwen-cpu
+    app.kubernetes.io/part-of: llm-serving
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: qwen-cpu
+  template:
+    metadata:
+      labels:
+        app: qwen-cpu
+        app.kubernetes.io/name: qwen-cpu
+        app.kubernetes.io/part-of: llm-serving
+    spec:
+      nodeSelector:
+        kubernetes.io/hostname: talos-cp-2
+      tolerations:
+        - key: node-role.kubernetes.io/control-plane
+          operator: Exists
+          effect: NoSchedule
+      containers:
+        - name: ollama
+          image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
+          command: ["/bin/sh", "-c"]
+          args:
+            - |
+              ollama serve &
+              SERVE_PID=$!
+              until ollama list >/dev/null 2>&1; do sleep 2; done
+              ollama pull qwen2.5:3b-instruct
+              ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
+              wait $SERVE_PID
+          env:
+            - name: OLLAMA_HOST
+              value: "0.0.0.0:8080"
+            - name: OLLAMA_MODELS
+              value: /root/.ollama/models
+            - name: OLLAMA_CONTEXT_LENGTH
+              value: "32768"
+            - name: OLLAMA_KEEP_ALIVE
+              value: "-1"
+            - name: OLLAMA_MAX_LOADED_MODELS
+              value: "1"
+            - name: OLLAMA_NUM_PARALLEL
+              value: "2"
+          ports:
+            - containerPort: 8080
+              protocol: TCP
+          readinessProbe:
+            exec:
+              command: ["/bin/sh", "-c", "ollama ps 2>/dev/null | grep -q qwen2.5"]
+            periodSeconds: 10
+          startupProbe:
+            exec:
+              command: ["/bin/sh", "-c", "ollama ps 2>/dev/null | grep -q qwen2.5"]
+            failureThreshold: 60
+            periodSeconds: 10
+          resources:
+            requests:
+              cpu: "4"
+              memory: 4Gi
+            limits:
+              cpu: "8"
+              memory: 8Gi
+          volumeMounts:
+            - mountPath: /root/.ollama
+              name: ollama-data
+      volumes:
+        - name: ollama-data
+          persistentVolumeClaim:
+            claimName: qwen-cpu-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: qwen-cpu
+  namespace: llm-serving
+  labels:
+    app: qwen-cpu
+    app.kubernetes.io/part-of: llm-serving
+spec:
+  selector:
+    app: qwen-cpu
+  ports:
+    - port: 80
+      targetPort: 8080
+      protocol: TCP
+---
+# Small PVC for qwen2.5:3b model weights (~1.9GB).
+# Separate from llm-models PVC which is pinned to worker-1.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: qwen-cpu-data
+  namespace: llm-serving
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: longhorn
+  resources:
+    requests:
+      storage: 5Gi
@@ -40,7 +40,7 @@ spec:
        
        # Git
        - name: GIT_REPO
-          value: https://forgejo.riotpiao.com/rock/homelab.git
+          value: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
        - name: GIT_AUTHOR_EMAIL
          value: [email protected]
        - name: GIT_AUTHOR_NAME
@@ -18,7 +18,7 @@ spec:
      prune: true
      selfHeal: true
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/argocd/projects
  destination:
@@ -17,7 +17,7 @@ spec:
    syncOptions:
      - CreateNamespace=true
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    # ksops decrypts every *.enc.yaml here at kustomize-build time (repo-server
    # runs `kustomize build --enable-alpha-plugins --enable-exec`). Replaces the
@@ -21,7 +21,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/bootstrap/cert-manager/cert-manager-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -93,7 +93,7 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    # A real kustomization.yaml (resources: the 3 issuer/CA files) renders these
    # deterministically. The previous directory.include with bare filenames
@@ -127,7 +127,7 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/bootstrap/ingress
  destination:
@@ -149,7 +149,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/cluster-maintenance
  destination:
@@ -177,7 +177,7 @@ spec:
      valueFiles:
        - $values/k8s/bootstrap/kyverno/kyverno-values.yaml
  sources:
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -200,7 +200,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/bootstrap/kyverno
  destination:
@@ -19,7 +19,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/argocd-image-updater/values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -9,7 +9,7 @@ spec:
  project: homelab
  
  sources:
-  - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+  - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    path: k8s/apps/secret-rotation-controller
    targetRevision: main
  
@@ -17,7 +17,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/minio/minio-operator-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -41,7 +41,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/minio
  destination:
@@ -66,7 +66,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/longhorn
  destination:
@@ -102,7 +102,7 @@ spec:
        skipCrds: true
        valueFiles:
          - $values/k8s/infra/monitoring/prometheus-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -152,7 +152,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/monitoring/crds
  destination:
@@ -183,7 +183,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/monitoring
  destination:
@@ -213,7 +213,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/monitoring/blackbox-exporter-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -237,7 +237,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/tracing
  destination:
@@ -19,7 +19,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/logging/loki-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -53,7 +53,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/logging/grafana-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -87,7 +87,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/logging/promtail-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -17,7 +17,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/iam/vault-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -46,7 +46,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/iam/authentik-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -68,7 +68,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/iam
  destination:
@@ -109,7 +109,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/bootstrap/phase3-forgejo/forgejo-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -162,7 +162,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/forgejo-runner
  destination:
@@ -190,7 +190,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/forgejo-runner
    helm:
@@ -221,7 +221,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/forgejo-runner
    helm:
@@ -14,7 +14,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/databases
  destination:
@@ -8,7 +8,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/messaging/memory-queues
  destination:
@@ -68,7 +68,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/messaging/kafka-cluster
  destination:
@@ -45,10 +45,10 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  sources:
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab-frontend.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab-frontend.git
      targetRevision: main
      path: k8s
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      path: k8s/apps/api
  destination:
@@ -20,7 +20,7 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/llm-serving
  destination:
@@ -0,0 +1,32 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: comfyui
+  namespace: argocd
+  labels:
+    app.kubernetes.io/name: comfyui
+    app.kubernetes.io/component: image-generation
+  annotations:
+    argocd.argoproj.io/sync-wave: "8"
+spec:
+  project: homelab
+  revisionHistoryLimit: 3
+  source:
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    targetRevision: main
+    path: k8s/apps/comfyui
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: comfyui
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
+    retry:
+      limit: 5
+      backoff:
+        duration: 5s
+        factor: 2
+        maxDuration: 3m
@@ -0,0 +1,32 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: gotify
+  namespace: argocd
+  labels:
+    app.kubernetes.io/name: gotify
+    app.kubernetes.io/component: notifications
+  annotations:
+    argocd.argoproj.io/sync-wave: "8"
+spec:
+  project: homelab
+  revisionHistoryLimit: 3
+  source:
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    targetRevision: main
+    path: k8s/apps/gotify
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: notifications
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
+    retry:
+      limit: 5
+      backoff:
+        duration: 5s
+        factor: 2
+        maxDuration: 3m
@@ -19,10 +19,10 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/apps/temporal/temporal-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      path: k8s/apps/temporal
  destination:
@@ -51,7 +51,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/apps/portainer/portainer-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -74,7 +74,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/cloudflared
  destination:
@@ -97,7 +97,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/agent-pod
  destination:
@@ -130,7 +130,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/sms
  destination:
@@ -157,7 +157,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/paperless
  destination:
@@ -189,7 +189,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/immich
  destination:
@@ -220,10 +220,10 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/apps/homarr/homarr-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      path: k8s/apps/homarr  # PostSync hook: fix-probes-job.yaml
  destination:
@@ -281,7 +281,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/rbac
  destination:
@@ -12,7 +12,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/kmsvc-manage.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/kmsvc-manage.git
    targetRevision: main
    path: k8s/argocd/apps
    directory:
@@ -20,13 +20,13 @@ metadata:
 spec:
  project: homelab
  sources:
-    - repoURL: https://forgejo.riotpiao.com/rock/poimen-memory.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/poimen-memory.git
      targetRevision: main
      path: k8s/argocd
-    - repoURL: https://forgejo.riotpiao.com/rock/poimen-workflows.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/poimen-workflows.git
      targetRevision: main
      path: k8s/argocd
-    - repoURL: https://forgejo.riotpiao.com/rock/poimen-frontend.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/poimen-frontend.git
      targetRevision: main
      path: k8s/argocd
  destination:
@@ -17,12 +17,13 @@ spec:
    - https://github.com/Riotpiaole/Poimen-workflows.git
    - https://github.com/Riotpiaole/poimen*.git
    # In-cluster Forgejo repos — explicit allowlist (no wildcard)
-    - https://forgejo.riotpiao.com/rock/homelab.git
-    - https://forgejo.riotpiao.com/rock/homelab-frontend.git
-    - https://forgejo.riotpiao.com/rock/kmsvc-manage.git
-    - https://forgejo.riotpiao.com/rock/poimen.git
-    - https://forgejo.riotpiao.com/rock/poimen-memory.git
-    - https://forgejo.riotpiao.com/rock/poimen-workflows.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab-frontend.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/kmsvc-manage.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen-memory.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen-workflows.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen-frontend.git
    - https://forgejo.riotpiao.com/rock/riotpiao.com.git
    # Public Helm chart repos referenced by k8s/argocd/apps/* and bootstrap/*
    - https://cloudnative-pg.github.io/charts
@@ -12,7 +12,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/argocd/apps
    directory:
@@ -0,0 +1,24 @@
+apiVersion: ENC[AES256_GCM,data:wR4=,iv:cRBzbvu0eUYCYeKeysua1/P3Meli/rQyj/mIV6VWnPM=,tag:oyTPA/mLYNUX+QDkYLlZyQ==,type:str]
+kind: ENC[AES256_GCM,data:bdKQdadW,iv:F3DQiBI9xhSx87jkS1Hyevo48uUCBjsJSjbScIBznKA=,tag:PFakzzBj5S9F65dxu0L2rg==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:XHFYrKClPo+IwRbM,iv:ZGuL+cN840Y1bOTC62NhDDcoZpTzDWQ4GfqPJX/QWmI=,tag:hlCVjzvfcxXGOASc3vda/Q==,type:str]
+    namespace: ENC[AES256_GCM,data:0BbhgZBog+1qY/iRJA==,iv:88tiSpEDqIokT5VP/d6bB2+aUyh1kZ7NEHwZWoJW3XU=,tag:CBR01jh2U9h7kNEcK1e1sA==,type:str]
+type: ENC[AES256_GCM,data:Mpv1V73w,iv:BW3r6RpLnwe3XDKwrZySyOjrWUnSwIG5JOoPLzP/5gM=,tag:oyJySL0haOei1m4i+1AJ1g==,type:str]
+stringData:
+    username: ENC[AES256_GCM,data:8xmxLGE=,iv:J/vEvXGoD+ka6FDgnSwDz0fs9IxuJIZW9r7Oyu2qxC8=,tag:dN9jd6NSavMN8+uzvemYWg==,type:str]
+    password: ENC[AES256_GCM,data:vB9PaaUiQZ8FY8pO0cq1fHLi7Gq5t4U=,iv:C0Y1m2SGYP3oTIFoau5JavVmLblj6te/SPMLodIwiZw=,tag:3Ip4YvR4WPzR0bBpTyjytA==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRT3JoTnhVVW5SUUJXbTdz
+            dUpLcmtBWWhyaGk3Y1hBM1ArcVI5eHREbmp3CiswUGVmd0NhejZwQ2UvNEdxS3ow
+            L1RweS9Kb2paeStLZ0tLSFdWbVZqQW8KLS0tIHJHT3hiMmxtM0Q1Ym5KOVpLVFpl
+            enR3NmdNdmdwVitTQVJlRHFWcjR0N2sKQw9ZZs+Ji/Zq/feO3qy4DwaCfWgDOQ/z
+            FVVhcCXweN58tb+9fzCJ+pNi/hSmvUkCMbb1+60qBvEehNzOoMRJ5g==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-09-10T13:31:00Z"
+    mac: ENC[AES256_GCM,data:3HWV/NG0yTbsxY28u41zTRmAPc1kokGb4nCLmAsyq8/uvxcP+evDNyZXYpS93eVdPRfRZ1OqVBzyVGJEnj7JSXQVmlzor9JcWEL2fcpogoLVfJZKTRD6hk6optnBMjj84iQEEOx3A80JrDOdoXsH0pd9bJBABwoUOJwuufbXcIU=,iv:KLZsmAcapQgV08pFhGIvPt5ylsWg3xazAAjPuJYOaXA=,tag:1vKA3fISMIurHzxZ04F3lg==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
@@ -0,0 +1,30 @@
+apiVersion: ENC[AES256_GCM,data:xUA=,iv:YH1WYnoXhxbeES3Dmo8ZrMDwFcwCykt6+Zn4YSgN4gc=,tag:t3bIWo/LetlQv4tJ0nJ2Sw==,type:str]
+kind: ENC[AES256_GCM,data:PJVzDTcT,iv:aRxvzLUuCzSDryOjA/v67csL6E2FcCpoK8aD7KNStno=,tag:cQgbowSTTV99nQ41yFOlsA==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:atkSiP0rjyuWFjs=,iv:l/48X9W9lrkAyw6Q/p39V775L6xbr0tOIFI4vjYrg+E=,tag:LZrQxN5h6T5tehFG8L9KFw==,type:str]
+    namespace: ENC[AES256_GCM,data:fbklNyooBeqsogKbeg==,iv:z2wLgl81q8xfzGGVQ9A4Pkst/zLmuZpyBzHrXzmVfm4=,tag:QKJik4IWc4nrOBINjGCxvQ==,type:str]
+type: ENC[AES256_GCM,data:bfAFBd/7,iv:9/W3X1KefqDAsurIatT6kBuEIQvWVRTq4G9A+9ZfYKs=,tag:Optlb+gZ2udbqIOerqV2zQ==,type:str]
+stringData:
+    #ENC[AES256_GCM,data:RZaw/FNQZOf0F3+XfFmGW+4gjKXNdjrAbMRjBt4hptwk,iv:IEd/+Bsv/h08nKak+bcovNiohC7nNjEQ+QdtIFMUGng=,tag:zsF5ZQ3+5JijF4MU+RhnyA==,type:comment]
+    #ENC[AES256_GCM,data:ux3ONHnuI0uFEihElbLI1rKlrN91Pn0iLLZfmez0+t5d8yhH6OD3L/3oi6yHbdgD8bV90kRK/lw=,iv:7Ed+W/TeOY9Af8bKO0Fu3XOuaK8b0Hw70rwobfmosxM=,tag:Z0RbIOjdDroK0drC6P+Y8A==,type:comment]
+    host: ENC[AES256_GCM,data:YKgas/8hT5ZOGIW7O0c=,iv:WtBz1MFrwd6EgDDS5fMqtcVGIZGNLgbXnmkqlNWoXy8=,tag:+H9ARGPi5XocgZYp4oA3wA==,type:str]
+    port: ENC[AES256_GCM,data:f0hx,iv:HkSnJcCyzyfBTA39BNw+muhHTyrNLzzGxZ7M3bcLVu4=,tag:4+xL1xNyYPaV2b0mwnbQVQ==,type:str]
+    from: ENC[AES256_GCM,data:lqYqXxJuI4CAWPlX+5cltCdxqA==,iv:ZgZYySJXvjuTNIkMaDxynUJ6PFdp4fibqlgoA1vNDKE=,tag:vb/XKl2N1Hl5u7ndZXQvbg==,type:str]
+    user: ENC[AES256_GCM,data:ajljGVbRO//G,iv:E4Q0LGDCd5wnHc+Oq2A9ZhWGbCmkKlhllt922UmppIg=,tag:Cf4ft1JKFi9PaJEfPfq92Q==,type:str]
+    password: ENC[AES256_GCM,data:YNbtoiWLsQGl,iv:FJ3y1lWkqKkM8GVr7FZljAXJ331GkDgy4TenAbdIY3w=,tag:KIvhv+RL49ZMwWhYrU0SPA==,type:str]
+    notify-email: ENC[AES256_GCM,data:s5wRkDuaiflCyu4IyAnaocVPA3I=,iv:dK5PsweYa3JotYcDcz74DPNDXmhW5ngNEbY6JuG1ctg=,tag:/Dt2UoCtfau6qv0G7VzmrQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLQlJVekcwb01ESnhxSE9m
+            MzliVmlaYUttZVdiMDVtMlQ2L29LMXBWRzBZCkxRMi9RcE5LMzQ1MHFqUUxLaFZo
+            MUZ2UTg5eUNsMFVmenlhL2F3NDd0akkKLS0tIGFwbUUrSGFUTDFVOWlxM3V1bVdy
+            RUVVNDFMRHZCSWptdSt1bFVoN2JPWVkKXMlrJNPbm7Sj9vd/H3o6iqPRLy4LyjVe
+            xJyh9vn9RCwS9Jr6BN2Q9P2UZfTGwvIYc6zdmV+b2uRwvi4t4lMsBA==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-09-10T13:31:00Z"
+    mac: ENC[AES256_GCM,data:g1YjOANlnvXErPqtDdDjRB9yTUiGEhX8QDkmf1aLfQBwoXbGUpoPIZibF8A4cAL1yUSZ2rD4l9TIe9nK8U3oF4WiKXf9u0bFK2g//bOv2A5JPewwmzoazkjIDXcJ/76nOQ5c+v8/r5tUG4c0SyzBH/gFP0D0+P5Yo3SEXjTxj2w=,iv:gW+AUwwAWkXZ+aLBXjjNDZw8VEraMpSE16RJGQc/L5g=,tag:DM/DUtil/7yLhfGYQXKFGA==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
@@ -0,0 +1,27 @@
+apiVersion: ENC[AES256_GCM,data:YE0=,iv:s3yNqcBn7/DKUQNgbGGwVmlM1HzxYGLBKyB6Y2ii2WE=,tag:77KxAjOFU3G0fSfrgudKkg==,type:str]
+kind: ENC[AES256_GCM,data:PFW4VV9T,iv:n8gzMjE5C2TAfUTpp/8ex64B+hWUGG1K+2oQ4deN03Q=,tag:18Xqb6Di5izMHmzR5EU+QA==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:Xhg1fQrD4itrbvDW0g==,iv:/THWSXAThb9fj+mm3wcxqzSdzdt7nE06+xOpkCxyImE=,tag:UChokr06VkbDZcFZDcUY2A==,type:str]
+    namespace: ENC[AES256_GCM,data:r25U5MuuzDd8JJ2YjQ==,iv:uDUcS4ZTpZe8HmZMArzkdu7LV5GUoLSP2wIs5HB1gv0=,tag:fci7j30hoxnVKJwPFNfd1Q==,type:str]
+type: ENC[AES256_GCM,data:agEG8Fu8,iv:ckYX0buX8md1CWHXfxbAXQJLzoTxTJI16nlQ9LSzYi0=,tag:4tZWf3REbGOmmBiT14+dew==,type:str]
+stringData:
+    #ENC[AES256_GCM,data:ehjcsmz1R0i/n31f8M0IIUT4KDBwzz6f5ds=,iv:j5swFAKpC67ZvGioiCCC1D651LxUl9gME8GqK5Uc+ys=,tag:BUJ8k9LHs8NAPPZAwPuifA==,type:comment]
+    #ENC[AES256_GCM,data:ndAB00yun636VHDMonqTghO/jCWodNd/hmdbm1QmCvOAi4FvTLl8bY3wYR+ZtlkSQYvFNqH798MJixv5OZwxBj5H,iv:pS+7B60jaS5jhqDRw2LbBFv7mD1HO6+hjjv+iNpenXc=,tag:NU6+gxCHTGxqeDMfvkwJWw==,type:comment]
+    #ENC[AES256_GCM,data:T3mhSm/5q7JTSXNLrjhpP5GhqA7nXz8uAhJcEV1RDctAX0Dyfq8Qn4bNFO51ibwTETx4SnunPuFZVs++AvRnsA==,iv:oRGM8N4iEcwBrMzoEXQAeKqCKzUq+jXTMVq9W2l6oh4=,tag:GrrbC/tkWpA5kp5UYCyRBA==,type:comment]
+    app-token: ENC[AES256_GCM,data:jw+AqbsxyoYRvNrUDrq+GNq/TNfweFCs,iv:67vSSgMZCMV0GG37ZxUxHAWZqOOGMYCmo3J43WgLyNk=,tag:xlrnip4XomXBbMmooW9FKg==,type:str]
+    client-token: ENC[AES256_GCM,data:gsxeCqKh4e+DFjpn9jM5GfemAdBf8dSv,iv:l2bBMdxQUil8jU9XDjXGn3EtvFVrK5ibXDCeGaPbYTY=,tag:ELf5S6cdNQJ84Es5upu1IQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAza3YrRW1VTVNSOGlMK21a
+            bDhzQlEyZDNnZTZrSVNnTUEzU1hSdnM1ZFVzCjdBSUlmWG5EcnlzbSs5bXdDaGkx
+            ME1nMnRqQzF1Vkc3b3FXSUVHT1g0ZDgKLS0tIHpMUytEMjdLQ2E0Unp2di9KS3JQ
+            eHBraDk1clJyanhLY2dGM0tESmJIUFkKHTl3y9uQiEofOFD8j2vH3YK/CVzlq11w
+            GfShIji1yCvvowKGzYYhsQK0UM0FzhzBv0GFMYWQCBq8pGdoPVmO5g==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-09-10T13:31:00Z"
+    mac: ENC[AES256_GCM,data:JhnO0V6cd2QVY/VhwHAJ5J75GeVlOVHBfVBTZstkj/IvpkAcwS/JE2b6jXiCwEC4n/vdA8DJ074noysUBRxh4Y7O4NKuvWcTniQKgqULNL1Hzgj3NdUkcQlWT+Z0HQ7FlvFH97VVXVfasU+CLHG48ShT2fDSj8JusEllb9CwSvg=,iv:PZo2krxvJc1TLJbvx+S9ClthoBe4w7WigUkq7dg0gNk=,tag:2IF5g/WTRP4XdnCZzDbiIA==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
@@ -27,3 +27,6 @@ files:
  - vault-secrets.enc.yaml
  - vault-unseal-keys.enc.yaml
  - portfolio-secrets.enc.yaml
+  - gotify-admin-secrets.enc.yaml
+  - gotify-tokens-secrets.enc.yaml
+  - gotify-smtp-secrets.enc.yaml
@@ -1,178 +0,0 @@
-#!/usr/bin/env bash
-# End-to-end auth flow tests for api-gateway.
-# Validates: JWT auth, identity headers, token exchange, error handling.
-#
-# Usage:
-#   ./scripts/test-auth-e2e.sh
-#
-# Requires: kubectl access to cluster, curl, jq
-
-set -euo pipefail
-
-API="https://api.riotpiao.com"
-TOKEN_URL="https://authentik.riotpiao.com/application/o/token/"
-PASS=0
-FAIL=0
-SKIP=0
-
-# Colors
-RED='\033[0;31m'
-GREEN='\033[0;32m'
-YELLOW='\033[0;33m'
-NC='\033[0m'
-
-check() {
-    local name="$1" expected="$2" actual="$3"
-    if [ "$expected" = "$actual" ]; then
-        echo -e "  ${GREEN}✓${NC} $name"
-        PASS=$((PASS + 1))
-    else
-        echo -e "  ${RED}✗${NC} $name (expected $expected, got $actual)"
-        FAIL=$((FAIL + 1))
-    fi
-}
-
-skip() {
-    echo -e "  ${YELLOW}○${NC} $1 (skipped: $2)"
-    SKIP=$((SKIP + 1))
-}
-
-# --- Load credentials ---
-echo "Loading credentials from cluster..."
-P_SECRET=$(kubectl get secret -n portfolio portfolio-agent-oidc -o jsonpath='{.data.CLIENT_SECRET}' | base64 -d 2>/dev/null) || true
-M_SECRET=$(kubectl get secret -n poimen memory-agent-oidc -o jsonpath='{.data.CLIENT_SECRET}' | base64 -d 2>/dev/null) || true
-
-if [ -z "$P_SECRET" ] || [ -z "$M_SECRET" ]; then
-    echo "ERROR: Cannot read service account secrets. Check kubectl context."
-    exit 1
-fi
-
-# --- Get tokens ---
-echo ""
-echo "=== Acquiring tokens ==="
-
-P_TOKEN=$(curl -s -X POST "$TOKEN_URL" \
-    -d "grant_type=client_credentials&client_id=portfolio-agent&client_secret=$P_SECRET&scope=openid roles" \
-    | jq -r '.access_token')
-
-M_TOKEN=$(curl -s -X POST "$TOKEN_URL" \
-    -d "grant_type=client_credentials&client_id=memory-agent&client_secret=$M_SECRET&scope=openid roles" \
-    | jq -r '.access_token')
-
-if [ "$P_TOKEN" = "null" ] || [ -z "$P_TOKEN" ]; then
-    echo "ERROR: Failed to get portfolio-agent token"
-    exit 1
-fi
-echo "  portfolio-agent: token acquired"
-echo "  memory-agent: token acquired"
-
-# --- Scenario 1: LLM inference ---
-echo ""
-echo "=== Scenario 1: LLM inference (llm:inference) ==="
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Authorization: Bearer $P_TOKEN" -H "Content-Type: application/json" \
-    -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"ok"}],"max_tokens":2}')
-check "portfolio-agent -> qwen chat" "200" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Authorization: Bearer $M_TOKEN" -H "Content-Type: application/json" \
-    -d '{"model":"reasoning","messages":[{"role":"user","content":"ok"}],"max_tokens":2}')
-check "memory-agent -> reasoning chat" "200" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/embeddings" \
-    -H "Authorization: Bearer $P_TOKEN" -H "Content-Type: application/json" \
-    -d '{"model":"nomic-ai/nomic-embed-text-v2-moe","input":"test"}')
-check "portfolio-agent -> embeddings" "200" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" "$API/v1/models")
-check "GET /v1/models (no auth)" "200" "$CODE"
-
-# --- Scenario 2: Auth enforcement ---
-echo ""
-echo "=== Scenario 2: Auth enforcement ==="
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Content-Type: application/json" \
-    -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"ok"}]}')
-check "no token -> 401" "401" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Authorization: Bearer garbage-token" -H "Content-Type: application/json" \
-    -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"ok"}]}')
-check "bad token -> 403" "403" "$CODE"
-
-# --- Scenario 3: Token claims ---
-echo ""
-echo "=== Scenario 3: Token claims ==="
-
-decode_jwt() { python3 -c "import base64,json,sys; p=sys.argv[1].split('.')[1]; p+='='*(4-len(p)%4); print(json.dumps(json.loads(base64.urlsafe_b64decode(p))))" "$1"; }
-
-P_CLAIMS=$(decode_jwt "$P_TOKEN")
-P_ROLES=$(echo "$P_CLAIMS" | jq -r '.roles | join(",")')
-check "portfolio-agent has llm:inference" "true" "$(echo "$P_ROLES" | grep -q 'llm:inference' && echo true || echo false)"
-check "portfolio-agent has s3:read" "true" "$(echo "$P_ROLES" | grep -q 's3:read' && echo true || echo false)"
-check "portfolio-agent has sqs:read" "true" "$(echo "$P_ROLES" | grep -q 'sqs:read' && echo true || echo false)"
-
-M_CLAIMS=$(decode_jwt "$M_TOKEN")
-M_ROLES=$(echo "$M_CLAIMS" | jq -r '.roles | join(",")')
-check "memory-agent has memory:write" "true" "$(echo "$M_ROLES" | grep -q 'memory:write' && echo true || echo false)"
-check "memory-agent has s3:write" "true" "$(echo "$M_ROLES" | grep -q 's3:write' && echo true || echo false)"
-
-# --- Scenario 4: X-Service dispatch ---
-echo ""
-echo "=== Scenario 4: X-Service dispatch ==="
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" "$API/" \
-    -H "X-Service: s3" -H "X-Resource: list-objects" \
-    -H "Authorization: Bearer $P_TOKEN")
-check "portfolio-agent -> S3 list (reaches MinIO)" "403" "$CODE"
-# 403 = MinIO reached but rejects JWT (needs SigV4). Not a gateway auth issue.
-
-# Memory service might be down
-CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "$API/" \
-    -H "X-Service: memory" -H "X-Resource: skills" \
-    -H "Authorization: Bearer $P_TOKEN" 2>/dev/null)
-if [ "$CODE" = "200" ]; then
-    check "portfolio-agent -> memory skills" "200" "$CODE"
-elif [ "$CODE" = "502" ]; then
-    skip "portfolio-agent -> memory skills" "poimen-memory pods down"
-else
-    check "portfolio-agent -> memory skills" "200" "$CODE"
-fi
-
-# --- Scenario 5: Auth endpoints ---
-echo ""
-echo "=== Scenario 5: Auth endpoints ==="
-
-# /auth/token — may return 400 if password grant not working, but endpoint should respond
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/auth/token" \
-    -H "Content-Type: application/json" \
-    -d '{"username":"rock","password":"wrong"}')
-# Should get 400 (bad creds forwarded from Authentik) or 502 (upstream issue), not 404
-if [ "$CODE" != "404" ] && [ "$CODE" != "405" ]; then
-    check "/auth/token endpoint exists" "true" "true"
-else
-    check "/auth/token endpoint exists" "not-404" "$CODE"
-fi
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X GET "$API/auth/token")
-check "GET /auth/token -> 405" "405" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/auth/refresh" \
-    -H "Content-Type: application/json" \
-    -d '{"refresh_token":"invalid"}')
-if [ "$CODE" != "404" ] && [ "$CODE" != "405" ]; then
-    check "/auth/refresh endpoint exists" "true" "true"
-else
-    check "/auth/refresh endpoint exists" "not-404" "$CODE"
-fi
-
-# --- Summary ---
-echo ""
-echo "========================================="
-TOTAL=$((PASS + FAIL + SKIP))
-echo -e "Results: ${GREEN}$PASS passed${NC}, ${RED}$FAIL failed${NC}, ${YELLOW}$SKIP skipped${NC} / $TOTAL total"
-echo "========================================="
-
-[ "$FAIL" -eq 0 ] && exit 0 || exit 1
@@ -36,6 +36,7 @@
    rewrite name paperless.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
    rewrite name img.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
    rewrite name api.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
+    rewrite name comfy.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
    rewrite name riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local

    kubernetes cluster.local in-addr.arpa ip6.arpa {
Author	SHA1	Message	Date
rock	9d3a669dfe	fix(gotify): move SOPS secrets to ksops generator ArgoCD couldn't decrypt secrets.yaml because it was listed as a plain kustomize resource. Move the 3 secrets (gotify-admin, gotify-tokens, gotify-smtp) to k8s/argocd/secrets/ as .enc.yaml files processed by the ksops generator, matching the repo convention. Fixes ComparisonError: 'Object Kind is missing' (SOPS ciphertext parsed as raw YAML).	2026-09-10 22:32:24 +09:00
rock	436c7d8d42	fix(argocd): update git repoURLs for org transfer rock -> riotpiao-poimen Repos transferred: homelab-frontend, kmsvc-manage, poimen, poimen-memory, poimen-workflows, poimen-frontend. Old URLs return 301 which ArgoCD doesn't follow. NOT changed: container image registry paths (rock/ is correct for registry), riotpiao.com (still under rock org). Also adds poimen-frontend to AppProject sourceRepos allowlist.	2026-09-10 10:49:30 +09:00
rock	b571d518e0	fix(argocd): update repoURL after org transfer (#29 ) Co-authored-by: rock <[email protected]>	2026-09-10 01:41:37 +00:00
rock	128e76ce2d	feat(gotify): push notification server + SMTP email relay (#19 ) Co-authored-by: rock <[email protected]>	2026-09-10 01:35:59 +00:00
rockandpoimen	5b16b882be	fix(comfyui): port 8888, emptyDir storage, liveness probe (#18 ) - Change container port from 8188 to 8888 (Caddy proxy binding) - Remove PVC, use emptyDir for ephemeral models/output - Replace startup + readiness probes with single liveness probe - Remove explicit COMFYUI_FLAGS (container defaults work) - Pod now reaches Ready state immediately after image pull Fixes GPU contention by using ephemeral storage. ComfyUI now runs and is accessible at https://comfy.riotpiao.comReviewed-on: #18 Co-authored-by: poimen <[email protected]> Reviewed-on: riotpiao-poimen/homelab-frontend#21 Co-authored-by: rock <[email protected]>	2026-09-09 22:38:25 +00:00
rock	8790de6038	feat: add ComfyUI + rebalance GPU allocation (#17 ) ## GPU Rebalance (4× V100 32GB) \| Pod \| Before \| After \| \|-----\|--------\|-------\| \| reasoning (PP=2) \| 2 GPU \| 2 GPU \| \| ornith \| 2 GPU (2 replicas) \| 1 GPU (1 replica) \| \| comfyui \| — \| 1 GPU (new) \| \| qwen-cpu \| — \| CPU on cp-2 (new) \| \| embeddings/reranker \| CPU \| CPU \| ## Changes - `ornith.yaml`: scale 2→1, remove qwen2.5 co-loading, MAX_LOADED_MODELS=1 - `qwen-cpu.yaml`: new Ollama deployment on talos-cp-2 (144GB RAM), 5Gi PVC - `k8s/apps/comfyui/`: new ComfyUI deployment (1 GPU, 50Gi model PVC, ingress) - `58-comfyui.yaml`: ArgoCD Application (wave 8) Gateway route update in separate PR (homelab-frontend).Reviewed-on: #17 Co-authored-by: rock <[email protected]>	2026-09-09 02:11:28 +00:00