fix(cluster): grant queue-operator watches and retire the macOS workload

queue-operator could not list temporalworkers.kmsvc.io or deployments.apps at cluster scope, so its informers failed on every retry and it reconciled nothing. Add those rules to the ClusterRole. macos-bluebubbles pins itself to a node labelled workload=imessage, and no such node exists. Its PVC macos-disk uses StorageClass longhorn-imessage-local and stayed unbound for 71 days. Scale the Deployment to 0 and record why.
Merge pull request 'fix(sops): drop the orphan forgejo-registry-pat file' (#65 ) from fix/sops-drop-orphan-forgejo-pat into main
2026-10-24 04:03:02 -07:00 · 2026-10-24 10:47:14 +00:00 · 2026-10-24 03:46:29 -07:00 · 2026-10-24 10:41:17 +00:00 · 2026-10-24 10:41:09 +00:00 · 2026-10-24 03:35:52 -07:00
6 changed files with 19 additions and 67 deletions
@@ -12,6 +12,19 @@ rules:
  - apiGroups: ["kmsvc.io"]
    resources: ["queues"]
    verbs: ["get", "list", "watch", "update", "patch"]
  # The operator watches TemporalWorker objects and creates Deployments for
  # them. Without these two rules the controller logs
  # "cannot list resource temporalworkers/deployments ... at the cluster scope"
  # and never reconciles.
  - apiGroups: ["kmsvc.io"]
    resources: ["temporalworkers"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["kmsvc.io"]
    resources: ["temporalworkers/status"]
    verbs: ["get", "update", "patch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
  - apiGroups: ["kmsvc.io"]
    resources: ["queues/status"]
    verbs: ["get", "update", "patch"]
@@ -34,7 +34,12 @@ metadata:
    app.kubernetes.io/name: macos-bluebubbles
    app.kubernetes.io/part-of: sms
 spec:
-  replicas: 1
+  # Scaled to 0 on 2026-10-24. The pod pinned itself to a node labelled
  # workload=imessage, and no such node exists any more. The PVC macos-disk also
  # uses StorageClass longhorn-imessage-local, which pins the same dead node and
  # stayed unbound for 71 days. The macOS guest therefore cannot start. Raise
  # this to 1 again only after a KVM-capable node carries that label and taint.
  replicas: 0
  # Recreate: the qcow2 disk is RWO and a second pod must never attach it
  # concurrently — two QEMU processes on one image corrupts it.
  strategy:
@@ -1,36 +0,0 @@
 apiVersion: argoproj.io/v1alpha1
 kind: Application
 metadata:
  name: azhi-riotpiao
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: '8'
 spec:
  project: homelab
  source:
    # Private GitHub. The read-only credential is Secret azhi-infra-github in
    # namespace argocd, created out-of-band.
    repoURL: https://github.com/ToyoGlobal/azhi-infra.git
    # The repository's default branch. It is not `main`: this repo's work
    # happens on the staging slice branch, and `main` is behind it.
    targetRevision: feat/azi-riotpiao-cnpg
    # One directory, because ArgoCD reads a single path per source. That
    # directory is a root kustomization over foundation, postgres and
    # azhi-server.
    path: k8s/overlays/azi-riotpiao
  destination:
    server: https://kubernetes.default.svc
    namespace: azi-riotpiao
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
    # The two Secrets this namespace needs are credentials, so they are created
    # by hand and NOT by this Application:
    #   acr-pull                 — the private Alibaba ACR pull credential
    #   azhi-riotpiao-app-env    — DB credentials, JWT secrets, AES key
    # A missing imagePullSecret fails the pod outright and a missing app-env
    # Secret leaves it in CreateContainerConfigError, which is the intended
    # loud failure rather than a pod that runs against unknown values.
@@ -16,12 +16,6 @@ spec:
    - https://github.com/Riotpiaole/Poimen-memory.git
    - https://github.com/Riotpiaole/Poimen-workflows.git
    - https://github.com/Riotpiaole/poimen*.git
    # ToyoGlobal (GitHub, private). The azhi-infra Application reads its
    # manifests from here, so the URL must be listed or ArgoCD rejects the
    # Application with "repository not permitted". A read-only credential for
    # this repo lives in Secret azhi-infra-github in namespace argocd; it is
    # created out-of-band and is not committed, because it is a credential.
    - https://github.com/ToyoGlobal/azhi-infra.git
    # In-cluster Forgejo repos — explicit allowlist (no wildcard)
    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab-frontend.git
@@ -1,23 +0,0 @@
 apiVersion: ENC[AES256_GCM,data:ECE=,iv:bISz4HovH++X7DW1Qj8Cw0L6+EvPB0+68hh7tfyW5C0=,tag:w+SjD7MsfeIuSf62n+Zl7Q==,type:str]
 kind: ENC[AES256_GCM,data:AxXaV3Mb,iv:xfJb354Rrjz3zLctW2i6hl40yh9EsfIqKxDvmn8jqnU=,tag:1xQ/RR3KhPLKCTWCYjebJg==,type:str]
 metadata:
    name: ENC[AES256_GCM,data:gdQ4OoMYKUPEbsUeA5OI4i1xnh8=,iv:IA2kaCyqQmkYpLolvcTF4aleh+yd/ImXJMhRMvpGCgo=,tag:8qyYYw4EhBKKPzEmpepAeQ==,type:str]
    namespace: ENC[AES256_GCM,data:/mOSyXWmRg==,iv:cpeHUTlMqlJzJttGtuR3DoiMtvVqFmDS0/5Tl7K7c2M=,tag:q4GxWgWn2wQJxJHqnq4WQA==,type:str]
 type: ENC[AES256_GCM,data:92OcrsuV,iv:Z1XBy6iZ6unGrK4/SSdDa58pbPL3022gU/f1AOM4uvc=,tag:cd4wWRq/ohm6BJ/Eo6HOIw==,type:str]
 stringData:
    REGISTRY_PAT: ENC[AES256_GCM,data:zDKruOXhzsIcFZTTp4r4r9rwHih9uf1cCp06KZ6eIXSIZDvR6aQY4A==,iv:5Vch5Z7xn1KkRxrgOs2p/M58nd6WhtcuUhoZDCi8LSY=,tag:14SAZIWGOxJMqLBtOorjYQ==,type:str]
 sops:
    age:
        - enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBIaG5QVUYyNjhvaENPcUFy
            K3RSSVM2N2hTeFhNK0M5YWZmODhzZU1HaHlFCmNGZWdFcFEvanVwcXpGSmVvRHVx
            aG45c2lBY1RuSTYwbDZTVk1QeHNWOTQKLS0tIDg4TnVMNjNtaU1VQk5zQjUvU2hM
            R245ZVdqc0ZWWVhJb3dOZ3lpU3JjUlUKawSg09ZPq8FKx5tvOVZZ+K4yh7eTQsUp
            be8mWUpS0+eEmNqh35BwU3HrETMQFA6a1kjVp30JOMtqa5rbYlzF7w==
            -----END AGE ENCRYPTED FILE-----
          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
    lastmodified: "2026-08-23T23:14:53Z"
    mac: ENC[AES256_GCM,data:yzi6woUaUCi6w9pG/eKnU7k/VfZgoXg/tW8p9joG8p7iabLWMdlXlx23m/CItw/NE0zeGWA5iZPPFZXOit2vN36VzK3kQNoFW6QbhvYLZi+78C/RWIdcpZkxB/tPIRG0vq8Q1SA+rwGjG/0xeAFh+R7k+YBTMd2XWBH3P7EI5T4=,iv:kf6MDaAVrtDvPIEjHMMLxXDSDRC3I1GpsPeJFUYppiw=,tag:NukVbGLa9EoMYmRsa4nBtA==,type:str]
    unencrypted_suffix: _unencrypted
    version: 3.13.2
@@ -11,7 +11,6 @@ files:
    - agent-pod-ssh-key.enc.yaml
    - authentik-secrets.enc.yaml
    - cloudflare-secrets.enc.yaml
    - forgejo-registry-pat.enc.yaml
    - forgejo-registry-pat-api.enc.yaml
    - forgejo-registry-pull.enc.yaml
    - forgejo-runner-token.enc.yaml
Author	SHA1	Message	Date
rock	2f664c32d9	fix(cluster): grant queue-operator watches and retire the macOS workload queue-operator could not list temporalworkers.kmsvc.io or deployments.apps at cluster scope, so its informers failed on every retry and it reconciled nothing. Add those rules to the ClusterRole. macos-bluebubbles pins itself to a node labelled workload=imessage, and no such node exists. Its PVC macos-disk uses StorageClass longhorn-imessage-local and stayed unbound for 71 days. Scale the Deployment to 0 and record why.	2026-10-24 04:03:02 -07:00
rock	6c98ca7436	Merge pull request 'fix(sops): drop the orphan forgejo-registry-pat file' (#65 ) from fix/sops-drop-orphan-forgejo-pat into main	2026-10-24 10:47:14 +00:00
rock	6ee22401fd	fix(sops): drop the orphan forgejo-registry-pat file ksops failed the whole sops-secrets application on this file: it declares Secret forgejo-registry-pat in namespace forgejo, and no such namespace exists. The failure blocks every other file in the generator list, including zitadel-masterkey. The same Secret name is already generated in namespace api by forgejo-registry-pat-api.enc.yaml, and that is the copy the gateway reads. The forgejo namespace copy is stale: Forgejo runs in namespace cicd, and namespace forgejo was never created. Remove the file and its generator entry. ArgoCD never created the Secret, so pruning has nothing to remove.	2026-10-24 03:46:29 -07:00
rock	e9817283e6	Merge pull request 'feat(iam): wire Zitadel as the Authentik replacement' (#63 ) from harness/zitadel-iam-wiring into main	2026-10-24 10:41:17 +00:00
rock	6e9586aa15	Merge pull request 'chore(argocd): delete the azhi-riotpiao Application' (#64 ) from chore/remove-azhi-riotpiao-app into main	2026-10-24 10:41:09 +00:00
rock	8e9ac37583	chore(argocd): delete the azhi-riotpiao Application The Application pointed at github.com/ToyoGlobal/azhi-infra, branch feat/azi-riotpiao-cnpg, path k8s/overlays/azi-riotpiao. It never reconciled: status stayed Unknown because the source repository could not be read, and the destination namespace azi-riotpiao was never created. ArgoCD reported it as an OutOfSync managed resource of homelab-root with nothing behind it. Also drops the ToyoGlobal URL from the AppProject allowlist. No manifest from that repository is referenced any more, so the allowlist entry is dead weight. Not touched here, because it is not in git: Secret azhi-infra-github in namespace argocd holds the read-only GitHub credential for that repository. Delete it separately once no other Application needs the repo. The live Application object was deleted by hand. Until this commit reaches main, the next homelab-root sync recreates it, because main still declares it.	2026-10-24 03:35:52 -07:00