Commit Graph
56 Commits
Author SHA1 Message Date
riotpiaole 3ac4083a68 feat(operator): add queue-operator CRD reconciler with shard split/drain
Reconciles Queue CRs into Kafka topics + Redis shard-map/queue-meta state:
creates shard-0 on first reconcile, splits a shard's hash range into two
children once its split threshold is crossed, drains and closes a parent
shard once its consumer group has fully caught up and its retention
window has elapsed, and tears down every shard's topic + Redis state on
deletion. Includes the manager entrypoint (cmd/queue-operator) and RBAC.
2026-06-21 17:08:17 -07:00
riotpiaole a04c76a684 feat(redis): add shard-aware key schema, Lua scripts, and atomic ops
Implements the kmsvc: key schema (inflight, pending/watermark keyed by
shard+partition, vis_index, dedup, fifo_lock, queue meta, shard map) plus
the atomic reap/ack Lua scripts used for safe multi-replica redelivery
and DLQ routing.
2026-06-21 17:05:38 -07:00
riotpiaole b704d401a1 feat(kafka): add shard-aware topology, admin, and client layer
Topic naming (kmsvc.{queue}.shard-{id} / .fifo.shard-{id} / .dlq.shard-{id}),
hash-range shard selection/splitting (murmur2-based, matching Kafka's
default partitioner), and topic admin/producer/consumer client
constructors shared by the operator and the message-plane.
2026-06-21 16:59:54 -07:00
riotpiaole 70e992bd96 feat(crd): add Queue CRD types and generated manifest
QueueSpec/QueueStatus model the shard-based (Kinesis-style hash-range
splitting) queue lifecycle object, with the controller-gen-generated
deepcopy and CRD YAML.
2026-06-21 16:59:47 -07:00
riotpiaole d38ce2ac92 feat(proto): define message-plane gRPC/REST API
Adds queue_service.proto (SendMessage, SendMessageBatch, ReceiveMessage,
DeleteMessage, DeleteMessageBatch, ChangeMessageVisibility(Batch)) with
grpc-gateway REST annotations, plus the generated Go server/client and
gateway stubs.
2026-06-21 16:59:41 -07:00
riotpiaole db02d0fa7f chore: scaffold Go module, codegen tooling, and shared runtime config
Sets up go.mod/go.sum, buf (proto codegen) configuration, and the
env-driven Config shared by the message-plane server and queue-operator.
2026-06-21 16:59:35 -07:00