fix(argocd): update repoURL from rock/homelab to riotpiao-poimen/homelab

Repo transferred to riotpiao-poimen org. Old URL returns 301 redirect which ArgoCD doesn't follow — all 26 apps stopped syncing. Live cluster already patched via kubectl. This commit persists the fix.
feat(gotify): push notification server + SMTP email relay (#19 )
2026-09-10 10:41:04 +09:00 · 2026-09-10 01:35:59 +00:00 · 2026-09-09 22:38:25 +00:00 · 2026-09-09 02:11:28 +00:00
35 changed files with 853 additions and 240 deletions
@@ -0,0 +1,80 @@
+# ComfyUI — GPU-accelerated image generation on worker-1.
+# Uses 1x V100 32GB (sm70). Freed by scaling ornith 2→1.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: comfyui
+  namespace: comfyui
+  labels:
+    app: comfyui
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: comfyui
+  template:
+    metadata:
+      labels:
+        app: comfyui
+    spec:
+      nodeSelector:
+        kubernetes.io/hostname: worker-1
+      runtimeClassName: nvidia
+      containers:
+        - name: comfyui
+          image: ghcr.io/ai-dock/comfyui:v2-cuda-12.1.1-base-22.04
+          ports:
+            - containerPort: 8188
+              protocol: TCP
+          env:
+            - name: NVIDIA_VISIBLE_DEVICES
+              value: "all"
+            - name: COMFYUI_FLAGS
+              value: "--listen 0.0.0.0 --port 8188"
+          resources:
+            requests:
+              cpu: "4"
+              memory: 8Gi
+              nvidia.com/gpu: "1"
+            limits:
+              cpu: "8"
+              memory: 16Gi
+              nvidia.com/gpu: "1"
+          volumeMounts:
+            - mountPath: /workspace/ComfyUI/models
+              name: models
+            - mountPath: /workspace/ComfyUI/output
+              name: output
+          readinessProbe:
+            httpGet:
+              path: /
+              port: 8188
+            periodSeconds: 10
+            initialDelaySeconds: 30
+          startupProbe:
+            httpGet:
+              path: /
+              port: 8188
+            failureThreshold: 60
+            periodSeconds: 10
+      volumes:
+        - name: models
+          persistentVolumeClaim:
+            claimName: comfyui-models
+        - name: output
+          emptyDir: {}
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: comfyui-models
+  namespace: comfyui
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: longhorn
+  resources:
+    requests:
+      storage: 50Gi
@@ -0,0 +1,25 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: comfyui
+  namespace: comfyui
+  annotations:
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
+    nginx.ingress.kubernetes.io/proxy-body-size: "0"
+    # WebSocket support for ComfyUI's live preview
+    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
+    nginx.ingress.kubernetes.io/proxy-set-headers: "Upgrade"
+spec:
+  ingressClassName: nginx
+  rules:
+    - host: comfy.riotpiao.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: comfyui
+                port:
+                  number: 80
@@ -0,0 +1,7 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - deployment.yaml
+  - service.yaml
+  - ingress.yaml
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: comfyui
+  namespace: comfyui
+  labels:
+    app: comfyui
+spec:
+  selector:
+    app: comfyui
+  ports:
+    - port: 80
+      targetPort: 8188
+      protocol: TCP
@@ -0,0 +1,107 @@
+# Gotify — Push Notifications + Email Relay
+
+Self-hosted notification server with SMTP email forwarding sidecar.
+
+## Architecture
+
+```
+Forgejo webhook ──POST──→ Gotify API (:80/message)
+                              │
+                    ┌─────────┼─────────┐
+                    ▼                     ▼
+              Push notification      SMTP emailer sidecar
+              (mobile/desktop)       (polls → sends email)
+```
+
+## Setup (one-time, after first deploy)
+
+### 1. Encrypt secrets before committing
+
+```bash
+# Edit secrets.yaml with real values first, then:
+sops -e -i k8s/apps/gotify/secrets.yaml
+```
+
+### 2. Create Gotify app + client tokens
+
+1. Login to `https://gotify.riotpiao.com` with admin creds
+2. **Applications** → Create `forgejo` → copy **app token**
+3. **Clients** → Create `smtp-emailer` → copy **client token**
+4. Update `gotify-tokens` secret:
+   ```bash
+   kubectl -n notifications create secret generic gotify-tokens \
+     --from-literal=app-token=<APP_TOKEN> \
+     --from-literal=client-token=<CLIENT_TOKEN> \
+     --dry-run=client -o yaml | kubectl apply -f -
+   ```
+
+### 3. Configure Forgejo webhook
+
+In each Forgejo repo → **Settings** → **Webhooks** → **Add Webhook** → **Gotify**:
+
+| Field | Value |
+|-------|-------|
+| Target URL | `http://gotify.notifications.svc.cluster.local/message` |
+| Token | The **app token** from step 2 |
+| Events | Pull Request (Created, Merged, Closed) |
+
+Or via API:
+```bash
+FORGEJO_TOKEN="<your-pat>"
+APP_TOKEN="<gotify-app-token>"
+
+curl -s -X POST "https://forgejo.riotpiao.com/api/v1/repos/rock/homelab/hooks" \
+  -H "Authorization: token $FORGEJO_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "type": "gotify",
+    "active": true,
+    "config": {
+      "content_type": "json",
+      "url": "http://gotify.notifications.svc.cluster.local/message?token='"$APP_TOKEN"'"
+    },
+    "events": ["pull_request", "pull_request_assign", "pull_request_review"],
+    "authorization_header": ""
+  }'
+```
+
+### 4. Add CoreDNS rewrite (if accessing via public hostname)
+
+Only needed if Cloudflare Tunnel is used for gotify.riotpiao.com:
+
+```
+# terraform/files/coredns/Corefile — add rewrite:
+rewrite name gotify.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
+```
+
+Then: `cd terraform && terraform apply && cd .. && make apply-cp`
+
+### 5. SMTP providers
+
+| Provider | Host | Port | Notes |
+|----------|------|------|-------|
+| Gmail | smtp.gmail.com | 587 | Use App Password (2FA required) |
+| Resend | smtp.resend.com | 587 | Free 100 emails/day |
+| Sendgrid | smtp.sendgrid.net | 587 | Free 100 emails/day |
+| Mailgun | smtp.mailgun.org | 587 | Free 5000/month |
+
+## Notification priority levels
+
+| Priority | Meaning | Email forwarded? |
+|----------|---------|-----------------|
+| 0-4 | Low (info) | No (below MIN_PRIORITY=5) |
+| 5-7 | Normal (PR created) | Yes |
+| 8-10 | High (PR merged, failures) | Yes |
+
+## Verify
+
+```bash
+# Test push notification
+APP_TOKEN="<app-token>"
+curl -X POST "https://gotify.riotpiao.com/message?token=$APP_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"title":"Test","message":"Hello from homelab","priority":5}'
+
+# Check email sidecar logs
+kubectl -n notifications logs deployment/gotify -c smtp-emailer --tail=20
+```
@@ -0,0 +1,35 @@
+# CNPG Postgres for Gotify. Lightweight — 2 instances, 2Gi storage.
+# CNPG generates secret `gotify-db-app` + service `gotify-db-rw` in ns notifications.
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: gotify-db
+  namespace: notifications
+  annotations:
+    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
+spec:
+  instances: 2
+  imageName: ghcr.io/cloudnative-pg/postgresql:16.2
+  bootstrap:
+    initdb:
+      database: gotify
+      owner: app
+      encoding: UTF8
+      localeCollate: C
+      localeCType: C
+  enableSuperuserAccess: false
+  resources:
+    requests: { memory: "256Mi", cpu: "100m" }
+    limits: { memory: "512Mi", cpu: "500m" }
+  storage:
+    size: 2Gi
+    storageClass: longhorn-cnpg
+  monitoring:
+    enablePodMonitor: true
+  affinity:
+    podAntiAffinityType: preferred
+    topologyKey: kubernetes.io/hostname
+    tolerations:
+      - key: node-role.kubernetes.io/control-plane
+        operator: Exists
+        effect: NoSchedule
@@ -0,0 +1,204 @@
+# Gotify — self-hosted push notification server + SMTP email relay.
+# Forgejo webhooks → Gotify → push notifications + email forwarding.
+# Runs on control plane (no GPU needed), lightweight.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: gotify
+  namespace: notifications
+  labels:
+    app: gotify
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: gotify
+  template:
+    metadata:
+      labels:
+        app: gotify
+    spec:
+      containers:
+        # --- Gotify server ---
+        - name: gotify
+          image: ghcr.io/gotify/server:2.6.1
+          command: ["/bin/sh", "-c"]
+          args:
+            - |
+              export GOTIFY_DATABASE_DIALECT=postgres
+              export GOTIFY_DATABASE_CONNECTION="host=gotify-db-rw.notifications port=5432 user=${DB_USER} password=${DB_PASS} dbname=gotify sslmode=disable"
+              exec /app/gotify-app
+          ports:
+            - containerPort: 80
+              protocol: TCP
+          env:
+            - name: GOTIFY_DEFAULTUSER_NAME
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-admin
+                  key: username
+            - name: GOTIFY_DEFAULTUSER_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-admin
+                  key: password
+            - name: DB_USER
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-db-app
+                  key: username
+            - name: DB_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-db-app
+                  key: password
+            - name: GOTIFY_SERVER_PORT
+              value: "80"
+            - name: GOTIFY_SERVER_KEEPALIVEPERIODSECONDS
+              value: "0"
+            - name: TZ
+              value: Asia/Tokyo
+          resources:
+            requests:
+              cpu: 50m
+              memory: 64Mi
+            limits:
+              cpu: 200m
+              memory: 128Mi
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: 80
+            periodSeconds: 30
+            initialDelaySeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: 80
+            periodSeconds: 10
+            initialDelaySeconds: 5
+
+        # --- SMTP emailer sidecar ---
+        # Watches Gotify WebSocket stream, forwards messages as email.
+        # https://github.com/eternal-flame-AD/gotify-broadcast
+        - name: smtp-emailer
+          image: ghcr.io/gotify/server:2.6.1
+          command:
+            - /bin/sh
+            - -c
+            - |
+              # Wait for Gotify to be ready
+              until wget -qO- http://localhost:80/health >/dev/null 2>&1; do
+                echo "Waiting for Gotify..."
+                sleep 2
+              done
+              echo "Gotify is ready, starting email relay..."
+
+              # Poll Gotify messages and forward via SMTP using msmtp
+              # Install msmtp for lightweight SMTP sending
+              apk add --no-cache msmtp curl jq
+
+              # Configure msmtp
+              cat > /tmp/msmtprc <<MSMTP
+              defaults
+              auth           on
+              tls            on
+              tls_trust_file /etc/ssl/certs/ca-certificates.crt
+              logfile        /tmp/msmtp.log
+
+              account        default
+              host           ${SMTP_HOST}
+              port           ${SMTP_PORT}
+              from           ${SMTP_FROM}
+              user           ${SMTP_USER}
+              password       ${SMTP_PASS}
+              MSMTP
+
+              chmod 600 /tmp/msmtprc
+
+              # Track last seen message ID
+              LAST_ID=0
+
+              while true; do
+                # Fetch messages since last ID
+                MESSAGES=$(curl -s -H "X-Gotify-Key: ${GOTIFY_CLIENT_TOKEN}" \
+                  "http://localhost:80/message?since=${LAST_ID}&limit=10" 2>/dev/null)
+
+                if [ -n "$MESSAGES" ]; then
+                  echo "$MESSAGES" | jq -r '.messages[]? | @base64' | while read -r MSG; do
+                    DECODED=$(echo "$MSG" | base64 -d)
+                    ID=$(echo "$DECODED" | jq -r '.id')
+                    TITLE=$(echo "$DECODED" | jq -r '.title // "Notification"')
+                    BODY=$(echo "$DECODED" | jq -r '.message // ""')
+                    PRIORITY=$(echo "$DECODED" | jq -r '.priority // 5')
+                    APP=$(echo "$DECODED" | jq -r '.appid // 0')
+                    DATE=$(echo "$DECODED" | jq -r '.date // ""')
+
+                    # Only forward messages with priority >= configured threshold
+                    if [ "$PRIORITY" -ge "${MIN_PRIORITY:-0}" ]; then
+                      printf "Subject: [Gotify] %s\nFrom: %s\nTo: %s\nContent-Type: text/plain; charset=UTF-8\n\n%s\n\n---\nPriority: %s\nDate: %s" \
+                        "$TITLE" "$SMTP_FROM" "$NOTIFY_EMAIL" "$BODY" "$PRIORITY" "$DATE" | \
+                        msmtp -C /tmp/msmtprc "$NOTIFY_EMAIL" && \
+                        echo "Email sent for message $ID: $TITLE" || \
+                        echo "Failed to send email for message $ID"
+                    fi
+
+                    # Update last seen ID
+                    if [ "$ID" -gt "$LAST_ID" ]; then
+                      LAST_ID=$ID
+                    fi
+                  done
+                fi
+
+                sleep ${POLL_INTERVAL:-30}
+              done
+          env:
+            - name: GOTIFY_CLIENT_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-tokens
+                  key: client-token
+            - name: SMTP_HOST
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: host
+            - name: SMTP_PORT
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: port
+            - name: SMTP_FROM
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: from
+            - name: SMTP_USER
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: user
+            - name: SMTP_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: password
+            - name: NOTIFY_EMAIL
+              valueFrom:
+                secretKeyRef:
+                  name: gotify-smtp
+                  key: notify-email
+            - name: MIN_PRIORITY
+              value: "5"
+            - name: POLL_INTERVAL
+              value: "15"
+          resources:
+            requests:
+              cpu: 10m
+              memory: 32Mi
+            limits:
+              cpu: 100m
+              memory: 64Mi
+      # No volumes — Postgres handles persistence
@@ -0,0 +1,26 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: gotify
+  namespace: notifications
+  annotations:
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
+    # WebSocket support for Gotify client connections
+    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
+    nginx.ingress.kubernetes.io/configuration-snippet: |
+      proxy_set_header Upgrade $http_upgrade;
+      proxy_set_header Connection "upgrade";
+spec:
+  ingressClassName: nginx
+  rules:
+    - host: gotify.riotpiao.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: gotify
+                port:
+                  number: 80
@@ -0,0 +1,9 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - namespace.yaml
+  - db.yaml
+  - secrets.yaml
+  - deployment.yaml
+  - service.yaml
+  - ingress.yaml
@@ -0,0 +1,6 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: notifications
+  labels:
+    kubernetes.io/metadata.name: notifications
@@ -0,0 +1,92 @@
+#ENC[AES256_GCM,data:zdor03HqWha5KnsrNNfM+NT96bZafKhVJ1vfARlqbXzd56QTMq6tScippCbJWCtqLhK5,iv:shItz1uc3RYNBo9Xmq4M+0/Yjw6PbpF4x1tfZZQOSUM=,tag:8Bcwz5XkgLtjuMrDxFXOZQ==,type:comment]
+#ENC[AES256_GCM,data:QuQeqD21UgcVOUjlJTzQOAXioYWmnc4b38UTPAClL3WIf7/MqDZ2SQ==,iv:PHw7QplRXB4XBQh3fEolZpdvMMxLucaZx/aY8K5LE20=,tag:Vp3BUb2JPOBffwZW0jPfOg==,type:comment]
+#
+#ENC[AES256_GCM,data:3JOwvqkN0IsF4ogdR6SK9xZKCs7I,iv:uB3qIR9r2AZ7pewPzV+5anulFHnnmUmlBhfhUdM8A2U=,tag:w4g3k9bBcqO6o0pGfCRjuA==,type:comment]
+#ENC[AES256_GCM,data:mMQ0VWb11cOPA5QgdUqNGgeeLyzbFRKU7YOhFHmUx7Zb3NmePRXZ2gT6Y9vuieiMxCL/GeNXeg==,iv:URCI0VXAlcT5QI01L4yBB+P3+VT7lEfbmQiMQhVTUYk=,tag:CiyVNUzFZBmydrK9I861Og==,type:comment]
+#ENC[AES256_GCM,data:AX/p8bnjaddZFwR3hkvZNsm8op3nx6b8jnWEj6mNvPryq/MGpoS9SkNvlyACQ5KYMIYG6QtltG9buv4n5CVJzyYX,iv:YKdhnl+YtvvOFjpNXp3Ap3mnDPaDa0YoWupviRwPjwE=,tag:etGle/aGv7RemlmUeKEkCg==,type:comment]
+#ENC[AES256_GCM,data:Rs33VKbpeUCDrUnblHfBXiVjZa7tlxFV+QWiMYMv/uLzmXBS7NCpU7fIgNKG,iv:ZrZug41B6AysyVylQ6T6Z+Txc2nBUvblX8pRP2r12iQ=,tag:SvrNa0N0mprMq/Wqf77tZg==,type:comment]
+#ENC[AES256_GCM,data:sBzMx0RT9FAdtE4pEyWUkNX/nKDgHoEHk4IITSKpAl0/3i2uWgWs9NO4Sp1USAQaTtIa+Os=,iv:Zqj2GAtfw3UsIDgrTZ8erug+KgjP2ckNcas7NaZei+E=,tag:6xRTvzspIVLVhQJMQinPOQ==,type:comment]
+#ENC[AES256_GCM,data:bD+GzYKonJecM6Pym1powOYISWJDPIEVT8V1Hu8AM1U/wCdmq70TNq6DRu1FDSG+,iv:a/p1nM+rkie+mZ7E9A1Zg3qQloGTdtRWSIUJPiFNk24=,tag:ymyVypKvIUWmcZeHc5BQFQ==,type:comment]
+apiVersion: ENC[AES256_GCM,data:4vs=,iv:JAbEScwusLVWdsz4AuuYsBQDuusynhsbstrEyihrZ/k=,tag:b1IY2bxT6TL7djcFCiyMlg==,type:str]
+kind: ENC[AES256_GCM,data:1+fCn5DR,iv:HvI/Id59qQC//Uh3Q0P3oJo8ZSWZnhTXv3LTox3BuEU=,tag:i9aPbsqch9YKekzbf5G4cg==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:4528BwgvzAtwiHyK,iv:/vz1jcRbLJmAoqDvgTmDbxvE5gYxpInSEGBP7IUFvlI=,tag:yZPEJNrHht9IVrlzsAUqFA==,type:str]
+    namespace: ENC[AES256_GCM,data:FZMvrE3BV3iD3As59A==,iv:PfOFjLpZspP6JeTdNwa1IwjRl4JE1XKeAOab+8S8imk=,tag:yg3bWjkPngilwirv1CscDw==,type:str]
+type: ENC[AES256_GCM,data:QnoC8qce,iv:8paBbJZtmEbKSKlkwhy+3lLiRLoy1I+lB0xMBPdcNMg=,tag:p4/aUaSZHBzeU1CRqJ5rLw==,type:str]
+stringData:
+    username: ENC[AES256_GCM,data:ANiWzQM=,iv:I3LrmBV5HEtjO+aysXuZEOmbvhyOfuOJCqZHx1ykWyU=,tag:zEV4gu2125L0bHGUkgGdNQ==,type:str]
+    password: ENC[AES256_GCM,data:E5OW/ydSmskcnR8UYZsi9SZM5kpbiAE=,iv:NTgtzAK8P0jeczb8Egrrkmlf1Ejulabxh2/rLWzwNTk=,tag:kF2luernkCvnNl8o3EEvfA==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3b3VUNi9oRFB3ZjFyeVQr
+            a2dTR3AvWnFDNi9YZ0QvaVNLU3dUSnlEMVdZCklhZjAzNURsZDBNWEI1MzZ1LzFl
+            cUNhUWJUcVhNSjZqSG43MWdudStaRUkKLS0tIEpuc2kwR2hpUzRiSXZyb1I0V3lE
+            YWJPcHFZU1lBOFFrQ3ErdzF4RzRqYzgK7q0N+ZcF/plIHR7HeRTF2qRmE4Za+eO+
+            mMCo01fxd3ybf8gXnC9lrYmIK4oCkS8cg/B8mfIIBqts7XDzHiCaGA==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-09-10T01:34:10Z"
+    mac: ENC[AES256_GCM,data:SURFYfHkesBFpdzeoMSdASSduGefetZ9sPY+0KgeUP4lMRX70kH6vnZdt5UyXe4hMgRCDA38sF+UgITW1pxhngqp8tL8QK5GBj+wTX9/zGmbtYl0zPvjvvXLx1vXGvUtfcEyxHW90EfDWae3JxQHCNlUcEH61hNj1ccZX+ceufM=,iv:mCQp3110nD3tg1eaf6P4e9XiMPbO+bhchC0zZ5+vqX8=,tag:vWGl3iaEi3KEoW87JBl3/g==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
+---
+apiVersion: ENC[AES256_GCM,data:too=,iv:DFFOApoqaZDGmJe7NcGBrvE69kp85YpLwJUsImKSxmk=,tag:sKWVbV1U3oI656Sb2+iWew==,type:str]
+kind: ENC[AES256_GCM,data:Lp1+y5tL,iv:pHbNnGCX6qsAIMSjkdHNyuHZct63jcDatnaek6Z5rjg=,tag:k/0Y7noIi/nAAVLpCv9GTw==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:QHAhhTHIxeyBXCfE2g==,iv:Q/2HuRPlbEC5AMONhi6POcCDdCJv3vHWMvmm6ZXGrr0=,tag:B+3Jhut9I7NzfpGtF969dQ==,type:str]
+    namespace: ENC[AES256_GCM,data:VnXAy9fR8ixXanheUw==,iv:Uvkcm7zqgLCWlzcx9fO2/ccytzM7MlD0Zl1X8WNaz7Q=,tag:JsWelai4T+aNg1UabZPMTg==,type:str]
+type: ENC[AES256_GCM,data:I3UeAwER,iv:mtIMz/kuzbHx0HMSPVaqzHMLvhCGzvkdNLRfr7xNKBM=,tag:QCgh1anTe7oN2j3kkUM2/g==,type:str]
+stringData:
+    #ENC[AES256_GCM,data:1XpJNypHreS+gUaRgEZe+CaJykvp3JJgI6Q=,iv:g/loGxxl6hvVvODmqwtip6om92df0tWVVg/msZvDOrg=,tag:ATCy0nG4PpGNoPQDRkXRcA==,type:comment]
+    #ENC[AES256_GCM,data:EmitK30ZlOZz7MruQ/ItDim2LjU3uUZlOQhfr2KO65t2lcAw3xcGZAdaQpQfyZ0RDUQucUgnl+g8ckhH6rZ7+/N9,iv:4655+CO98ak8cG/dZI0H2RCykLS29AWAZ4LjrH3Tg2M=,tag:eqhcnuY6rbyy98ABZrtPrw==,type:comment]
+    #ENC[AES256_GCM,data:iBIgf2Ru/cwE8D1L7uTdCmcii3lJxny1JE84jntuhROkCK1bqZU2cyf8eC0rq1zONiWwPuTokqFv18v+pdmbzQ==,iv:cnOCtS1qGy/w7ZkG2zRjqpgSeLU0FgvuV4tuBJ2SQ5U=,tag:3uxrBDiH/zMpLvFxZVfn1Q==,type:comment]
+    app-token: ENC[AES256_GCM,data:NNpIl+DOZ9hjP5TY0bmHWRfSC88GTy1O,iv:NDnz4i/rYbbUxpxUSHFyXc31iI4PQSasjEcHV3u9E70=,tag:8mBEOwhXxVPmoX85fAVKdg==,type:str]
+    client-token: ENC[AES256_GCM,data:nMSuoalWILpFijzVcB7sfA+d4EDb7HzL,iv:cgqkBnBUDqYMBXpQVCMzIPvpVvMu3BKl1CDqtY04xrk=,tag:Eql+ZAqPdvzm2M19HOoOHA==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3b3VUNi9oRFB3ZjFyeVQr
+            a2dTR3AvWnFDNi9YZ0QvaVNLU3dUSnlEMVdZCklhZjAzNURsZDBNWEI1MzZ1LzFl
+            cUNhUWJUcVhNSjZqSG43MWdudStaRUkKLS0tIEpuc2kwR2hpUzRiSXZyb1I0V3lE
+            YWJPcHFZU1lBOFFrQ3ErdzF4RzRqYzgK7q0N+ZcF/plIHR7HeRTF2qRmE4Za+eO+
+            mMCo01fxd3ybf8gXnC9lrYmIK4oCkS8cg/B8mfIIBqts7XDzHiCaGA==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-09-10T01:34:10Z"
+    mac: ENC[AES256_GCM,data:SURFYfHkesBFpdzeoMSdASSduGefetZ9sPY+0KgeUP4lMRX70kH6vnZdt5UyXe4hMgRCDA38sF+UgITW1pxhngqp8tL8QK5GBj+wTX9/zGmbtYl0zPvjvvXLx1vXGvUtfcEyxHW90EfDWae3JxQHCNlUcEH61hNj1ccZX+ceufM=,iv:mCQp3110nD3tg1eaf6P4e9XiMPbO+bhchC0zZ5+vqX8=,tag:vWGl3iaEi3KEoW87JBl3/g==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
+---
+apiVersion: ENC[AES256_GCM,data:Ub4=,iv:t0/4neL1XNR9sc2a0rZ7c4fOnZjRw0ACOY2Pp2A61IY=,tag:X8W/L0wfQLw4JF5aQ1dteg==,type:str]
+kind: ENC[AES256_GCM,data:54r8Je+L,iv:feX1TVQNZ7VrYIFiYKvopB0hYEIl9iw0ly2A6u/j+1Y=,tag:gD7iCoOCxIUT5uj1bGfikQ==,type:str]
+metadata:
+    name: ENC[AES256_GCM,data:ub9FC+uwioKPMXg=,iv:1YbPJQle7t60Onnh77R8ffc8M87DNU1cmBQbVA72B28=,tag:3DsCpXMHcTIYN3uBh6Ztug==,type:str]
+    namespace: ENC[AES256_GCM,data:dLIZlOoIo9YF2VyHew==,iv:YxQFWoMhVMG3jUJqmAtAUwzRr7MkSRjiWlq92rnymIE=,tag:iw5AI7kKlp+wKcgg2GGMxw==,type:str]
+type: ENC[AES256_GCM,data:4+CN/8f0,iv:Gmnh4TP0oCAiJ2VTrBT/Q1PWkPZks6XnzWu3io32hvQ=,tag:m5SNbovIU7pjwGACc4Q50g==,type:str]
+stringData:
+    #ENC[AES256_GCM,data:bGrL1ia7o4ADA4J7soyOLCMVq3Y7kUU0BHhdz9ID45/q,iv:fwMHTJ68cYgBrzeAQ/CNdpfH14bAv7Exu7H6/QUk1ok=,tag:Lwc+FnbCd48oErLp5qEmcw==,type:comment]
+    #ENC[AES256_GCM,data:bQdj5OgHL60gffBB5DZPOL6+RFWxrrBxa8lPkXbYR7viOf0v0xb9iBqjS/o5kIOjjcdeJibgGhQ=,iv:DJQBppj4w3KqFOC1MTEYwoc1LT0Qz17fKb611hgH+AE=,tag:D+l7BE+FEgxZFTIDLpKsiA==,type:comment]
+    host: ENC[AES256_GCM,data:kaPQN225/tgQLLmejLQ=,iv:Xn69DoAa7Zop6EjZwIRgO/zVPdxvARgKkoacIE4Fcwc=,tag:hlCapggLwTsXr4DeejAYkQ==,type:str]
+    port: ENC[AES256_GCM,data:vkBH,iv:uCA8fhnMZDg/qiSQYRYmhuc7dnteoavcuJxfSCE8GU0=,tag:ujosmW6GjrEa30mBCEFXgw==,type:str]
+    from: ENC[AES256_GCM,data:kiWkZ7zmIT6gOHCH6hHI8fiJcQ==,iv:tqpYbLohLYbQ6kotOC5vDw6cZEFvNK1UEiKIa+nFzuQ=,tag:POZAxMpiGItr1lRY1yhW6w==,type:str]
+    user: ENC[AES256_GCM,data:Vdew4lqPsOD2,iv:1zJE/hEDbchie4crugcjCfvACklXoeSzqopXbPulD/E=,tag:QdJvWQRi9Tc7pgMBigKsBg==,type:str]
+    password: ENC[AES256_GCM,data:JcjnnXFkAXmI,iv:VgpYDrTn6o3l7tAk0lGMB2rpu+kB1IuAFu2VZMyJkX4=,tag:QR+En9nv/30yyt09GTcyqg==,type:str]
+    notify-email: ENC[AES256_GCM,data:zMvA9KXy6rK2HFRrjAmZd5c/ay8=,iv:BtCXUsHneVc1QzvPKRwBjmbYc39I3Xy/zFoMg4l22Ks=,tag:NTZKDoPaLARJoTIX9u6oGQ==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB3b3VUNi9oRFB3ZjFyeVQr
+            a2dTR3AvWnFDNi9YZ0QvaVNLU3dUSnlEMVdZCklhZjAzNURsZDBNWEI1MzZ1LzFl
+            cUNhUWJUcVhNSjZqSG43MWdudStaRUkKLS0tIEpuc2kwR2hpUzRiSXZyb1I0V3lE
+            YWJPcHFZU1lBOFFrQ3ErdzF4RzRqYzgK7q0N+ZcF/plIHR7HeRTF2qRmE4Za+eO+
+            mMCo01fxd3ybf8gXnC9lrYmIK4oCkS8cg/B8mfIIBqts7XDzHiCaGA==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+    lastmodified: "2026-09-10T01:34:10Z"
+    mac: ENC[AES256_GCM,data:SURFYfHkesBFpdzeoMSdASSduGefetZ9sPY+0KgeUP4lMRX70kH6vnZdt5UyXe4hMgRCDA38sF+UgITW1pxhngqp8tL8QK5GBj+wTX9/zGmbtYl0zPvjvvXLx1vXGvUtfcEyxHW90EfDWae3JxQHCNlUcEH61hNj1ccZX+ceufM=,iv:mCQp3110nD3tg1eaf6P4e9XiMPbO+bhchC0zZ5+vqX8=,tag:vWGl3iaEi3KEoW87JBl3/g==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: gotify
+  namespace: notifications
+  labels:
+    app: gotify
+spec:
+  selector:
+    app: gotify
+  ports:
+    - port: 80
+      targetPort: 80
+      protocol: TCP
@@ -14,6 +14,7 @@ resources:
  - ornith.yaml
  - reasoning.yaml
  - reranker.yaml
+  - qwen-cpu.yaml
  - networkpolicy.yaml
 # No namespace transformer: every file sets its own, and the transformer would
 # rewrite metadata.namespace on anything cross-namespace added later.
@@ -33,12 +33,8 @@ spec:

        ollama pull ornith:35b

-        ollama pull qwen2.5:3b-instruct
-
        ollama run ornith:35b "ok" >/dev/null 2>&1 || true

-        ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
-
        wait $SERVE_PID

        '
@@ -54,7 +50,7 @@ spec:
      - name: OLLAMA_NUM_PARALLEL
        value: '1'
      - name: OLLAMA_MAX_LOADED_MODELS
-        value: '2'
+        value: '1'
      image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
      name: kserve-container
      ports:
@@ -65,8 +61,7 @@ spec:
          command:
          - /bin/sh
          - -c
-          - ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
-            grep -q qwen2.5
+          - ollama ps 2>/dev/null | grep -q ornith
        periodSeconds: 10
      resources:
        limits:
@@ -82,8 +77,7 @@ spec:
          command:
          - /bin/sh
          - -c
-          - ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
-            grep -q qwen2.5
+          - ollama ps 2>/dev/null | grep -q ornith
        failureThreshold: 120
        periodSeconds: 15
      volumeMounts:
@@ -91,13 +85,10 @@ spec:
        name: models
    deploymentStrategy:
      type: Recreate
-    # 2 replicas -- each its own GPU, each loading both ornith:35b and
-    # qwen2.5:3b-instruct -- so 2 concurrent implementer-style calls each
-    # get an independent instance instead of contending on one, at the
-    # cost of judge/qwen traffic still sharing whichever replica an
-    # implementer call also lands on.
-    maxReplicas: 2
-    minReplicas: 2
+    # 1 replica -- ornith:35b only. qwen2.5:3b moved to CPU on cp-2.
+    # Frees 1 GPU for ComfyUI.
+    maxReplicas: 1
+    minReplicas: 1
    nodeSelector:
      kubernetes.io/hostname: worker-1
    runtimeClassName: nvidia
@@ -0,0 +1,115 @@
+# qwen2.5:3b-instruct on CPU (talos-cp-2, 144GB RAM, 24 cores).
+# Moved off GPU to free a V100 for ComfyUI. Latency ~10x slower
+# than GPU but sufficient for lightweight tasks (summarization,
+# classification, quick answers).
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: qwen-cpu
+  namespace: llm-serving
+  labels:
+    app: qwen-cpu
+    app.kubernetes.io/name: qwen-cpu
+    app.kubernetes.io/part-of: llm-serving
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: qwen-cpu
+  template:
+    metadata:
+      labels:
+        app: qwen-cpu
+        app.kubernetes.io/name: qwen-cpu
+        app.kubernetes.io/part-of: llm-serving
+    spec:
+      nodeSelector:
+        kubernetes.io/hostname: talos-cp-2
+      tolerations:
+        - key: node-role.kubernetes.io/control-plane
+          operator: Exists
+          effect: NoSchedule
+      containers:
+        - name: ollama
+          image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
+          command: ["/bin/sh", "-c"]
+          args:
+            - |
+              ollama serve &
+              SERVE_PID=$!
+              until ollama list >/dev/null 2>&1; do sleep 2; done
+              ollama pull qwen2.5:3b-instruct
+              ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
+              wait $SERVE_PID
+          env:
+            - name: OLLAMA_HOST
+              value: "0.0.0.0:8080"
+            - name: OLLAMA_MODELS
+              value: /root/.ollama/models
+            - name: OLLAMA_CONTEXT_LENGTH
+              value: "32768"
+            - name: OLLAMA_KEEP_ALIVE
+              value: "-1"
+            - name: OLLAMA_MAX_LOADED_MODELS
+              value: "1"
+            - name: OLLAMA_NUM_PARALLEL
+              value: "2"
+          ports:
+            - containerPort: 8080
+              protocol: TCP
+          readinessProbe:
+            exec:
+              command: ["/bin/sh", "-c", "ollama ps 2>/dev/null | grep -q qwen2.5"]
+            periodSeconds: 10
+          startupProbe:
+            exec:
+              command: ["/bin/sh", "-c", "ollama ps 2>/dev/null | grep -q qwen2.5"]
+            failureThreshold: 60
+            periodSeconds: 10
+          resources:
+            requests:
+              cpu: "4"
+              memory: 4Gi
+            limits:
+              cpu: "8"
+              memory: 8Gi
+          volumeMounts:
+            - mountPath: /root/.ollama
+              name: ollama-data
+      volumes:
+        - name: ollama-data
+          persistentVolumeClaim:
+            claimName: qwen-cpu-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: qwen-cpu
+  namespace: llm-serving
+  labels:
+    app: qwen-cpu
+    app.kubernetes.io/part-of: llm-serving
+spec:
+  selector:
+    app: qwen-cpu
+  ports:
+    - port: 80
+      targetPort: 8080
+      protocol: TCP
+---
+# Small PVC for qwen2.5:3b model weights (~1.9GB).
+# Separate from llm-models PVC which is pinned to worker-1.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: qwen-cpu-data
+  namespace: llm-serving
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: longhorn
+  resources:
+    requests:
+      storage: 5Gi
@@ -18,7 +18,7 @@ spec:
      prune: true
      selfHeal: true
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/argocd/projects
  destination:
@@ -17,7 +17,7 @@ spec:
    syncOptions:
      - CreateNamespace=true
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    # ksops decrypts every *.enc.yaml here at kustomize-build time (repo-server
    # runs `kustomize build --enable-alpha-plugins --enable-exec`). Replaces the
@@ -21,7 +21,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/bootstrap/cert-manager/cert-manager-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -93,7 +93,7 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    # A real kustomization.yaml (resources: the 3 issuer/CA files) renders these
    # deterministically. The previous directory.include with bare filenames
@@ -127,7 +127,7 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/bootstrap/ingress
  destination:
@@ -149,7 +149,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/cluster-maintenance
  destination:
@@ -177,7 +177,7 @@ spec:
      valueFiles:
        - $values/k8s/bootstrap/kyverno/kyverno-values.yaml
  sources:
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -200,7 +200,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/bootstrap/kyverno
  destination:
@@ -19,7 +19,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/argocd-image-updater/values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -9,7 +9,7 @@ spec:
  project: homelab
  
  sources:
-  - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+  - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    path: k8s/apps/secret-rotation-controller
    targetRevision: main
  
@@ -17,7 +17,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/minio/minio-operator-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -41,7 +41,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/minio
  destination:
@@ -66,7 +66,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/longhorn
  destination:
@@ -102,7 +102,7 @@ spec:
        skipCrds: true
        valueFiles:
          - $values/k8s/infra/monitoring/prometheus-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -152,7 +152,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/monitoring/crds
  destination:
@@ -183,7 +183,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/monitoring
  destination:
@@ -213,7 +213,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/monitoring/blackbox-exporter-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -237,7 +237,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/tracing
  destination:
@@ -19,7 +19,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/logging/loki-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -53,7 +53,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/logging/grafana-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -87,7 +87,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/logging/promtail-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -17,7 +17,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/iam/vault-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -46,7 +46,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/infra/iam/authentik-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -68,7 +68,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/iam
  destination:
@@ -109,7 +109,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/bootstrap/phase3-forgejo/forgejo-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -162,7 +162,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/forgejo-runner
  destination:
@@ -190,7 +190,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/forgejo-runner
    helm:
@@ -221,7 +221,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/forgejo-runner
    helm:
@@ -14,7 +14,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/databases
  destination:
@@ -8,7 +8,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/messaging/memory-queues
  destination:
@@ -68,7 +68,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/messaging/kafka-cluster
  destination:
@@ -48,7 +48,7 @@ spec:
    - repoURL: https://forgejo.riotpiao.com/rock/homelab-frontend.git
      targetRevision: main
      path: k8s
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      path: k8s/apps/api
  destination:
@@ -20,7 +20,7 @@ spec:
  project: homelab
  revisionHistoryLimit: 3
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/llm-serving
  destination:
@@ -0,0 +1,32 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: comfyui
+  namespace: argocd
+  labels:
+    app.kubernetes.io/name: comfyui
+    app.kubernetes.io/component: image-generation
+  annotations:
+    argocd.argoproj.io/sync-wave: "8"
+spec:
+  project: homelab
+  revisionHistoryLimit: 3
+  source:
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    targetRevision: main
+    path: k8s/apps/comfyui
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: comfyui
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
+    retry:
+      limit: 5
+      backoff:
+        duration: 5s
+        factor: 2
+        maxDuration: 3m
@@ -0,0 +1,32 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: gotify
+  namespace: argocd
+  labels:
+    app.kubernetes.io/name: gotify
+    app.kubernetes.io/component: notifications
+  annotations:
+    argocd.argoproj.io/sync-wave: "8"
+spec:
+  project: homelab
+  revisionHistoryLimit: 3
+  source:
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    targetRevision: main
+    path: k8s/apps/gotify
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: notifications
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
+    retry:
+      limit: 5
+      backoff:
+        duration: 5s
+        factor: 2
+        maxDuration: 3m
@@ -19,10 +19,10 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/apps/temporal/temporal-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      path: k8s/apps/temporal
  destination:
@@ -51,7 +51,7 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/apps/portainer/portainer-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
  destination:
@@ -74,7 +74,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/cloudflared
  destination:
@@ -97,7 +97,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/agent-pod
  destination:
@@ -130,7 +130,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/sms
  destination:
@@ -157,7 +157,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/paperless
  destination:
@@ -189,7 +189,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/apps/immich
  destination:
@@ -220,10 +220,10 @@ spec:
      helm:
        valueFiles:
          - $values/k8s/apps/homarr/homarr-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      ref: values
-    - repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
      targetRevision: main
      path: k8s/apps/homarr  # PostSync hook: fix-probes-job.yaml
  destination:
@@ -281,7 +281,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/infra/rbac
  destination:
@@ -17,7 +17,7 @@ spec:
    - https://github.com/Riotpiaole/Poimen-workflows.git
    - https://github.com/Riotpiaole/poimen*.git
    # In-cluster Forgejo repos — explicit allowlist (no wildcard)
-    - https://forgejo.riotpiao.com/rock/homelab.git
+    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    - https://forgejo.riotpiao.com/rock/homelab-frontend.git
    - https://forgejo.riotpiao.com/rock/kmsvc-manage.git
    - https://forgejo.riotpiao.com/rock/poimen.git
@@ -12,7 +12,7 @@ metadata:
 spec:
  project: homelab
  source:
-    repoURL: https://forgejo.riotpiao.com/rock/homelab.git
+    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    targetRevision: main
    path: k8s/argocd/apps
    directory:
@@ -1,178 +0,0 @@
-#!/usr/bin/env bash
-# End-to-end auth flow tests for api-gateway.
-# Validates: JWT auth, identity headers, token exchange, error handling.
-#
-# Usage:
-#   ./scripts/test-auth-e2e.sh
-#
-# Requires: kubectl access to cluster, curl, jq
-
-set -euo pipefail
-
-API="https://api.riotpiao.com"
-TOKEN_URL="https://authentik.riotpiao.com/application/o/token/"
-PASS=0
-FAIL=0
-SKIP=0
-
-# Colors
-RED='\033[0;31m'
-GREEN='\033[0;32m'
-YELLOW='\033[0;33m'
-NC='\033[0m'
-
-check() {
-    local name="$1" expected="$2" actual="$3"
-    if [ "$expected" = "$actual" ]; then
-        echo -e "  ${GREEN}✓${NC} $name"
-        PASS=$((PASS + 1))
-    else
-        echo -e "  ${RED}✗${NC} $name (expected $expected, got $actual)"
-        FAIL=$((FAIL + 1))
-    fi
-}
-
-skip() {
-    echo -e "  ${YELLOW}○${NC} $1 (skipped: $2)"
-    SKIP=$((SKIP + 1))
-}
-
-# --- Load credentials ---
-echo "Loading credentials from cluster..."
-P_SECRET=$(kubectl get secret -n portfolio portfolio-agent-oidc -o jsonpath='{.data.CLIENT_SECRET}' | base64 -d 2>/dev/null) || true
-M_SECRET=$(kubectl get secret -n poimen memory-agent-oidc -o jsonpath='{.data.CLIENT_SECRET}' | base64 -d 2>/dev/null) || true
-
-if [ -z "$P_SECRET" ] || [ -z "$M_SECRET" ]; then
-    echo "ERROR: Cannot read service account secrets. Check kubectl context."
-    exit 1
-fi
-
-# --- Get tokens ---
-echo ""
-echo "=== Acquiring tokens ==="
-
-P_TOKEN=$(curl -s -X POST "$TOKEN_URL" \
-    -d "grant_type=client_credentials&client_id=portfolio-agent&client_secret=$P_SECRET&scope=openid roles" \
-    | jq -r '.access_token')
-
-M_TOKEN=$(curl -s -X POST "$TOKEN_URL" \
-    -d "grant_type=client_credentials&client_id=memory-agent&client_secret=$M_SECRET&scope=openid roles" \
-    | jq -r '.access_token')
-
-if [ "$P_TOKEN" = "null" ] || [ -z "$P_TOKEN" ]; then
-    echo "ERROR: Failed to get portfolio-agent token"
-    exit 1
-fi
-echo "  portfolio-agent: token acquired"
-echo "  memory-agent: token acquired"
-
-# --- Scenario 1: LLM inference ---
-echo ""
-echo "=== Scenario 1: LLM inference (llm:inference) ==="
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Authorization: Bearer $P_TOKEN" -H "Content-Type: application/json" \
-    -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"ok"}],"max_tokens":2}')
-check "portfolio-agent -> qwen chat" "200" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Authorization: Bearer $M_TOKEN" -H "Content-Type: application/json" \
-    -d '{"model":"reasoning","messages":[{"role":"user","content":"ok"}],"max_tokens":2}')
-check "memory-agent -> reasoning chat" "200" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/embeddings" \
-    -H "Authorization: Bearer $P_TOKEN" -H "Content-Type: application/json" \
-    -d '{"model":"nomic-ai/nomic-embed-text-v2-moe","input":"test"}')
-check "portfolio-agent -> embeddings" "200" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" "$API/v1/models")
-check "GET /v1/models (no auth)" "200" "$CODE"
-
-# --- Scenario 2: Auth enforcement ---
-echo ""
-echo "=== Scenario 2: Auth enforcement ==="
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Content-Type: application/json" \
-    -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"ok"}]}')
-check "no token -> 401" "401" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/v1/chat/completions" \
-    -H "Authorization: Bearer garbage-token" -H "Content-Type: application/json" \
-    -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"ok"}]}')
-check "bad token -> 403" "403" "$CODE"
-
-# --- Scenario 3: Token claims ---
-echo ""
-echo "=== Scenario 3: Token claims ==="
-
-decode_jwt() { python3 -c "import base64,json,sys; p=sys.argv[1].split('.')[1]; p+='='*(4-len(p)%4); print(json.dumps(json.loads(base64.urlsafe_b64decode(p))))" "$1"; }
-
-P_CLAIMS=$(decode_jwt "$P_TOKEN")
-P_ROLES=$(echo "$P_CLAIMS" | jq -r '.roles | join(",")')
-check "portfolio-agent has llm:inference" "true" "$(echo "$P_ROLES" | grep -q 'llm:inference' && echo true || echo false)"
-check "portfolio-agent has s3:read" "true" "$(echo "$P_ROLES" | grep -q 's3:read' && echo true || echo false)"
-check "portfolio-agent has sqs:read" "true" "$(echo "$P_ROLES" | grep -q 'sqs:read' && echo true || echo false)"
-
-M_CLAIMS=$(decode_jwt "$M_TOKEN")
-M_ROLES=$(echo "$M_CLAIMS" | jq -r '.roles | join(",")')
-check "memory-agent has memory:write" "true" "$(echo "$M_ROLES" | grep -q 'memory:write' && echo true || echo false)"
-check "memory-agent has s3:write" "true" "$(echo "$M_ROLES" | grep -q 's3:write' && echo true || echo false)"
-
-# --- Scenario 4: X-Service dispatch ---
-echo ""
-echo "=== Scenario 4: X-Service dispatch ==="
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" "$API/" \
-    -H "X-Service: s3" -H "X-Resource: list-objects" \
-    -H "Authorization: Bearer $P_TOKEN")
-check "portfolio-agent -> S3 list (reaches MinIO)" "403" "$CODE"
-# 403 = MinIO reached but rejects JWT (needs SigV4). Not a gateway auth issue.
-
-# Memory service might be down
-CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "$API/" \
-    -H "X-Service: memory" -H "X-Resource: skills" \
-    -H "Authorization: Bearer $P_TOKEN" 2>/dev/null)
-if [ "$CODE" = "200" ]; then
-    check "portfolio-agent -> memory skills" "200" "$CODE"
-elif [ "$CODE" = "502" ]; then
-    skip "portfolio-agent -> memory skills" "poimen-memory pods down"
-else
-    check "portfolio-agent -> memory skills" "200" "$CODE"
-fi
-
-# --- Scenario 5: Auth endpoints ---
-echo ""
-echo "=== Scenario 5: Auth endpoints ==="
-
-# /auth/token — may return 400 if password grant not working, but endpoint should respond
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/auth/token" \
-    -H "Content-Type: application/json" \
-    -d '{"username":"rock","password":"wrong"}')
-# Should get 400 (bad creds forwarded from Authentik) or 502 (upstream issue), not 404
-if [ "$CODE" != "404" ] && [ "$CODE" != "405" ]; then
-    check "/auth/token endpoint exists" "true" "true"
-else
-    check "/auth/token endpoint exists" "not-404" "$CODE"
-fi
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X GET "$API/auth/token")
-check "GET /auth/token -> 405" "405" "$CODE"
-
-CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST "$API/auth/refresh" \
-    -H "Content-Type: application/json" \
-    -d '{"refresh_token":"invalid"}')
-if [ "$CODE" != "404" ] && [ "$CODE" != "405" ]; then
-    check "/auth/refresh endpoint exists" "true" "true"
-else
-    check "/auth/refresh endpoint exists" "not-404" "$CODE"
-fi
-
-# --- Summary ---
-echo ""
-echo "========================================="
-TOTAL=$((PASS + FAIL + SKIP))
-echo -e "Results: ${GREEN}$PASS passed${NC}, ${RED}$FAIL failed${NC}, ${YELLOW}$SKIP skipped${NC} / $TOTAL total"
-echo "========================================="
-
-[ "$FAIL" -eq 0 ] && exit 0 || exit 1
@@ -36,6 +36,7 @@
    rewrite name paperless.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
    rewrite name img.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
    rewrite name api.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
+    rewrite name comfy.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
    rewrite name riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local

    kubernetes cluster.local in-addr.arpa ip6.arpa {
Author	SHA1	Message	Date
rock	f8baeed581	fix(argocd): update repoURL from rock/homelab to riotpiao-poimen/homelab Repo transferred to riotpiao-poimen org. Old URL returns 301 redirect which ArgoCD doesn't follow — all 26 apps stopped syncing. Live cluster already patched via kubectl. This commit persists the fix.	2026-09-10 10:41:04 +09:00
rock	128e76ce2d	feat(gotify): push notification server + SMTP email relay (#19 ) Co-authored-by: rock <[email protected]>	2026-09-10 01:35:59 +00:00
rockandpoimen	5b16b882be	fix(comfyui): port 8888, emptyDir storage, liveness probe (#18 ) - Change container port from 8188 to 8888 (Caddy proxy binding) - Remove PVC, use emptyDir for ephemeral models/output - Replace startup + readiness probes with single liveness probe - Remove explicit COMFYUI_FLAGS (container defaults work) - Pod now reaches Ready state immediately after image pull Fixes GPU contention by using ephemeral storage. ComfyUI now runs and is accessible at https://comfy.riotpiao.comReviewed-on: #18 Co-authored-by: poimen <[email protected]> Reviewed-on: riotpiao-poimen/homelab-frontend#21 Co-authored-by: rock <[email protected]>	2026-09-09 22:38:25 +00:00
rock	8790de6038	feat: add ComfyUI + rebalance GPU allocation (#17 ) ## GPU Rebalance (4× V100 32GB) \| Pod \| Before \| After \| \|-----\|--------\|-------\| \| reasoning (PP=2) \| 2 GPU \| 2 GPU \| \| ornith \| 2 GPU (2 replicas) \| 1 GPU (1 replica) \| \| comfyui \| — \| 1 GPU (new) \| \| qwen-cpu \| — \| CPU on cp-2 (new) \| \| embeddings/reranker \| CPU \| CPU \| ## Changes - `ornith.yaml`: scale 2→1, remove qwen2.5 co-loading, MAX_LOADED_MODELS=1 - `qwen-cpu.yaml`: new Ollama deployment on talos-cp-2 (144GB RAM), 5Gi PVC - `k8s/apps/comfyui/`: new ComfyUI deployment (1 GPU, 50Gi model PVC, ingress) - `58-comfyui.yaml`: ArgoCD Application (wave 8) Gateway route update in separate PR (homelab-frontend).Reviewed-on: #17 Co-authored-by: rock <[email protected]>	2026-09-09 02:11:28 +00:00