kmsvc-sdk/PLAN.md

kmsvc-sdk — Go client SDK implementation plan

Context

Companion to kafaka_management_service (server), kmsvc-proto (shared wire contract), and kmsvc-cli (design.md §11a). Standalone Go module — own go.mod (forgejo.riotpiao.homelab.com/homelab/kmsvc-sdk), own Forgejo repo, own release pipeline — so any Go project (including kmsvc-cli) can go get it without pulling in the server's internal/ packages, Kafka admin clients, or Redis dependencies.

The SDK wraps the generated gRPC client for kafkamgmt.v1 with ergonomic Go methods, auth-token attachment, and long-poll handling — callers never touch raw protobuf types or gRPC metadata.

QueueService (the only service in kmsvc-proto) is message-plane only — SendMessage(Batch), ReceiveMessage, DeleteMessage(Batch), ChangeMessageVisibility. Queue lifecycle (create/delete/configure) is managed via the Queue CRD on the cluster, not exposed over gRPC — so this SDK has no queues.go/lifecycle methods.

Proto sourcing

kmsvc-proto publishes pre-generated Go code (gen/kafkamgmt/v1), committed and tagged in that repo. This SDK consumes it as a plain Go module dependency — go get forgejo.riotpiao.homelab.com/homelab/[email protected] with GOPRIVATE=forgejo.riotpiao.homelab.com set. No submodule, no local buf generate, no internal/genapi.

Repo layout

kmsvc-sdk/
  go.mod                          # module forgejo.riotpiao.homelab.com/homelab/kmsvc-sdk
  client.go                        # Client, Option, New()
  auth.go                          # TokenSource + per-call credential attachment
  messages.go                      # SendMessage(Batch)/DeleteMessage(Batch)/ChangeMessageVisibility(Batch)
  longpoll.go                      # ReceiveMessage wait_time_seconds handling, context-deadline-safe
  errors.go                        # typed error wrapping over gRPC status codes
  internal_test_fake_test.go       # bufconn-backed fake QueueServiceServer + newTestClient helper
  auth_test.go, messages_test.go, longpoll_test.go
  examples/sendreceive/main.go
  .forgejo/workflows/ci.yaml
  CHANGELOG.md

Implementation steps — status

Step 1 — Dependency wiring ✅ done

• go.mod + go get forgejo.riotpiao.homelab.com/homelab/[email protected].
• Verified: go build ./... succeeds.

Step 2 — Client + connection/auth plumbing ✅ done

• client.go: New(ctx, target string, opts ...Option) (*Client, error) — dials gRPC, holds the generated stub.
• Options implemented: WithTokenSource(TokenSource), WithTransportCredentials(credentials.TransportCredentials), WithDialTimeout(d).
• auth.go: TokenSource interface (Token(ctx) (string, error)); StaticToken for tests/scripts. A unary+stream client interceptor attaches authorization: Bearer <token> to every call when a TokenSource is configured.
• Verified: TestClientAttachesBearerToken / TestClientWithoutTokenSourceSendsNoAuthHeader against an in-process bufconn fake.
• Deferred: ClientCredentialsTokenSource doing the OAuth2 client-credentials flow against Authentik directly — not yet implemented; StaticToken is sufficient until kmsvc-cli/real auth wiring needs it.

Step 3 — Message-plane methods ✅ done

• messages.go: SendMessage, SendMessageBatch, DeleteMessage, DeleteMessageBatch, ChangeMessageVisibility plus a client-side MaxMessageBodyBytes (256KB) pre-check that fails fast before a round trip.
• Verified: table-driven tests in messages_test.go covering request/response mapping, the size-cap pre-check (single + batch), and error mapping.

Step 4 — Long-polling ReceiveMessage ✅ done

• longpoll.go: ReceiveMessage(ctx, queueName string, opts ReceiveOptions) ([]Message, error) wrapping wait_time_seconds, with an internal context deadline WaitTimeSeconds + 5s margin so a slow network round trip doesn't truncate a poll the server was about to satisfy — but never overrides a tighter deadline the caller's own ctx already has.
• Verified: TestReceiveMessageMapsResponse, TestReceiveMessageRespectsCallerDeadline (asserts the caller's shorter deadline wins, not the internal margin).

Step 5 — Typed errors ✅ done

• errors.go: maps gRPC status codes (NotFound, AlreadyExists, InvalidArgument, Unauthenticated, ResourceExhausted→ErrMessageTooLarge) to exported errors.Is-compatible sentinels.
• Verified: TestSendMessageMapsNotFoundError, TestSendMessageRejectsOversizedBody, etc.

Step 6 — Example + docs — pending

• examples/sendreceive/main.go: send → receive → delete, doubling as a manual smoke test against a real running server.
• README.md — done (install + quickstart + error handling).
• Verify: go run ./examples/sendreceive against a locally running kafaka_management_service (manual, once server task 9 ships) — not part of automated CI.

Step 7 — CI (Forgejo Actions) ✅ done

• .forgejo/workflows/ci.yaml: go build ./..., go vet ./..., go test ./... -race on every push/PR, GOPRIVATE=forgejo.riotpiao.homelab.com set so kmsvc-proto resolves.
• Verify: a real push runs the workflow successfully in Forgejo (pending first push).

Acceptance criteria (overall)

• go get forgejo.riotpiao.homelab.com/homelab/kmsvc-sdk@<tag> works from a clean external module given GOPRIVATE set, no replace directives.
• No kafkamgmt/v1 (generated) types appear in any exported function signature.
• All unit tests pass against the bufconn fake server; no real Kafka/Redis/network dependency in CI.
• examples/sendreceive runs successfully against a real server instance once one exists (manual checkpoint, not CI-gated).
• CI workflow passes on a real Forgejo push (pending first push/tag).