feat: stamp Queue shard status with availability zones

Resolves each shard topic's replica broker IDs (internal/kafka.Admin.
ReplicaBrokerIDs) to the topology.kubernetes.io/zone labels of the nodes
hosting those brokers (ZoneLocator), and writes the result into
ShardStatus.AvailabilityZones each reconcile. Uses mgr.GetAPIReader()
rather than the cached client for the Pod/Node lookups, since the cached
client would otherwise require cluster-wide list/watch RBAC on Pods just
to serve occasional point Gets.
This commit is contained in:
riotpiaole
2026-06-22 17:30:26 -07:00
parent 1ff1ecd563
commit c7eeed2617
12 changed files with 227 additions and 0 deletions
+9
View File
@@ -211,6 +211,15 @@ spec:
description: ShardStatus describes one shard backing a Queue (design.md
§2a/§2c).
properties:
availabilityZones:
description: |-
AvailabilityZones lists the topology.kubernetes.io/zone values of every
node currently hosting a Kafka replica of this shard's topic, resolved
from the broker pods' node placement each reconcile. Empty until the
first successful resolution (e.g. node lookup failed transiently).
items:
type: string
type: array
createdAt:
description: |-
CreatedAt timestamps when this shard was created, used to enforce
@@ -25,5 +25,13 @@ spec:
value: {{ .Values.redisAddr | quote }}
- name: GOMEMLIMIT
value: {{ .Values.goMemLimit | quote }}
- name: KMSVC_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: KMSVC_KAFKA_CLUSTER_NAME
value: {{ .Values.kafkaClusterName | quote }}
- name: KMSVC_KAFKA_POOL_NAME
value: {{ .Values.kafkaPoolName | quote }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
+3
View File
@@ -24,6 +24,9 @@ rules:
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "patch"]
- apiGroups: [""]
resources: ["pods", "nodes"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
+6
View File
@@ -8,6 +8,12 @@ image:
kafkaBrokers: "kmsvc-kafka-bootstrap.sqs.svc.cluster.local:9092"
redisAddr: "kmsvc-redis-master.sqs.svc.cluster.local:6379"
# Must match kafka-cluster chart's clusterName/derived pool name -- used to
# resolve "<kafkaClusterName>-<kafkaPoolName>-<brokerID>" broker pod names
# for AZ-aware Queue status (design.md §2a).
kafkaClusterName: kmsvc
kafkaPoolName: kmsvc-pool
resources:
requests:
cpu: 100m