Per-queue ticker that scans vis_index for expired in-flight messages and
drives each through the atomic reap.lua check-and-act, routing maxed-out
messages to their queue's configured DLQ via the existing SendMessage
path. Safe to run from multiple replicas against the same queue.
Implements SendMessage (size cap, FIFO dedup, shard routing), ReceiveMessage
(long-poll loop across active+closing shards, FIFO per-group exclusivity
gating, redelivery hand-out), DeleteMessage (ack + low-watermark offset
advancement), and ChangeMessageVisibility against the Redis state layer
and a real Kafka producer/consumer.
Reconciles Queue CRs into Kafka topics + Redis shard-map/queue-meta state:
creates shard-0 on first reconcile, splits a shard's hash range into two
children once its split threshold is crossed, drains and closes a parent
shard once its consumer group has fully caught up and its retention
window has elapsed, and tears down every shard's topic + Redis state on
deletion. Includes the manager entrypoint (cmd/queue-operator) and RBAC.
Implements the kmsvc: key schema (inflight, pending/watermark keyed by
shard+partition, vis_index, dedup, fifo_lock, queue meta, shard map) plus
the atomic reap/ack Lua scripts used for safe multi-replica redelivery
and DLQ routing.
QueueSpec/QueueStatus model the shard-based (Kinesis-style hash-range
splitting) queue lifecycle object, with the controller-gen-generated
deepcopy and CRD YAML.
Adds queue_service.proto (SendMessage, SendMessageBatch, ReceiveMessage,
DeleteMessage, DeleteMessageBatch, ChangeMessageVisibility(Batch)) with
grpc-gateway REST annotations, plus the generated Go server/client and
gateway stubs.
riotpiaole