fix(longhorn): isolate GPU worker from general storage scheduling

- Remove expand-replicas-job (blindly forced all volumes to 3 replicas,
  ignoring StorageClass settings)
- Add diskSelector: 'storage' to longhorn and longhorn-cnpg StorageClasses
  so replicas only land on CP nodes (cp-1, cp-2, cp-3)
- Tag all CP node disks with 'storage' via PostSync job
  (disk names are runtime-discovered, can't hardcode in Node CRs)
- Disable scheduling on worker-1 Node CR — only longhorn-llm-local
  (diskSelector: 'llm') can use it
- worker-1 is GPU-only: llm-models and comfyui use dedicated SCs
This commit is contained in:
2026-09-11 09:33:52 +09:00
parent 7bd9f83fa7
commit 3462096311
6 changed files with 106 additions and 111 deletions
+19 -11
View File
@@ -1,16 +1,14 @@
# Longhorn Node CRDs for cp-2 and cp-3.
# These nodes have the control-plane taint, so Longhorn doesn't auto-discover them.
# Explicit Node CRDs + the taint-toleration setting enable storage across all 3 nodes.
# Longhorn Node CRDs for cp-2, cp-3, and worker-1.
# cp-2/cp-3 have the control-plane taint, so Longhorn doesn't auto-discover them.
# Explicit Node CRDs + the taint-toleration setting enable storage across all 3 CP nodes.
#
# `spec.disks` is deliberately absent. Longhorn owns disk identity: it names the
# entry itself (`default-disk-080400000000`, not `default-disk`) and writes
# `storageReserved`, `diskType` and `evictionRequested` into it. Declaring a
# `default-disk` key here never matched the live one, so the Application sat
# OutOfSync and selfHeal kept trying to add a SECOND disk record pointing at the
# same /var/lib/longhorn path — which is worse than the drift it was fixing.
# `spec.disks` is deliberately absent for CP nodes. Longhorn owns disk identity:
# it names the entry itself and writes `storageReserved`, `diskType` and
# `evictionRequested` into it. Disk tags are applied via kubectl patch (see
# longhorn-tag-disks-job.yaml) since disk names are runtime-discovered.
#
# What these objects are actually for is `allowScheduling: true` on tainted
# control-plane nodes. That is all they need to declare.
# worker-1 has scheduling disabled so only StorageClasses with explicit
# diskSelector (e.g. `llm` for longhorn-llm-local) can use it.
---
apiVersion: longhorn.io/v1beta2
kind: Node
@@ -31,3 +29,13 @@ spec:
name: talos-cp-3
allowScheduling: true
tags: []
---
apiVersion: longhorn.io/v1beta2
kind: Node
metadata:
name: worker-1
namespace: longhorn-system
spec:
name: worker-1
allowScheduling: false
tags: []