Phase 6.6: Add Poimen Memory DLQ queues (ArgoCD managed)

ArgoCD Application: memory-queues
  ├─ Sync wave: 7 (messaging wave)
  ├─ Path: k8s/apps/messaging/memory-queues
  ├─ Namespace: sqs
  └─ Auto-sync: enabled (prune + selfHeal)

Helm Chart: memory-queues
  ├─ Chart.yaml: v0.1.0
  ├─ values.yaml: Queue config
  └─ templates/queues.yaml: Queue CRD resources

Queues Created:

1. poimen-memory-dlq
   ├─ Purpose: Extraction + webhook + agent failures
   ├─ Partitions: 3
   ├─ Replication factor: 1
   ├─ Retention: 14 days (1,209,600 seconds)
   └─ Visibility timeout: 5 minutes (300 seconds)

2. poimen-memory-metric-dlq
   ├─ Purpose: Metrics persistence failures
   ├─ Partitions: 3
   ├─ Replication factor: 1
   ├─ Retention: 14 days
   └─ Visibility timeout: 5 minutes

Resource: Queue CRD (kmsvc.io/v1alpha1)
  └─ Managed by: queue-operator (already running in sqs ns)

Deployment Flow:
  ArgoCD (homelab) → sync wave 7 → deploy queues
  Memory app (poimen) → connects to kmsvc → sends DLQ messages

Files:
  ├─ k8s/apps/messaging/memory-queues/Chart.yaml (new)
  ├─ k8s/apps/messaging/memory-queues/values.yaml (new)
  ├─ k8s/apps/messaging/memory-queues/templates/queues.yaml (new)
  └─ k8s/argocd/apps/50-memory-queues.yaml (new)
This commit is contained in:
2026-09-05 01:10:51 -07:00
parent 266f0637a4
commit 6743f7c25f
5 changed files with 71 additions and 0 deletions
View File
@@ -0,0 +1,6 @@
apiVersion: v2
name: memory-queues
description: Kafka queues (DLQ) for Poimen Memory service (Phase 6.6)
type: application
version: 0.1.0
appVersion: "1.0"
@@ -0,0 +1,20 @@
{{- range .Values.queues }}
---
apiVersion: kmsvc.io/v1alpha1
kind: Queue
metadata:
name: {{ .name }}
namespace: {{ $.Values.namespace }}
labels:
app: memory-service
queue: dlq
spec:
name: {{ .name }}
description: {{ .description }}
partitions: {{ .partitions }}
replicationFactor: {{ .replicationFactor }}
config:
retention.ms: "{{ .config.retention.ms }}"
message.retention.seconds: "{{ .config.message.retention.seconds }}"
visibility.timeout.seconds: "{{ .config.visibility.timeout.seconds }}"
{{- end }}
@@ -0,0 +1,25 @@
# Poimen Memory Service Kafka Queues (kmsvc)
# Phase 6.6: DLQ topics for webhook + metrics failures
queues:
# DLQ for extraction, webhook, and agent failures
- name: poimen-memory-dlq
description: "DLQ for extraction, webhook, and agent failures"
partitions: 3
replicationFactor: 1
config:
retention.ms: "1209600000" # 14 days
message.retention.seconds: "1209600"
visibility.timeout.seconds: "300"
# DLQ for metrics persistence failures
- name: poimen-memory-metric-dlq
description: "DLQ for metrics persistence failures"
partitions: 3
replicationFactor: 1
config:
retention.ms: "1209600000" # 14 days
message.retention.seconds: "1209600"
visibility.timeout.seconds: "300"
namespace: sqs
+20
View File
@@ -0,0 +1,20 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: memory-queues
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "7"
spec:
project: homelab
source:
repoURL: https://forgejo.riotpiao.com/rock/homelab.git
targetRevision: main
path: k8s/apps/messaging/memory-queues
destination:
server: https://kubernetes.default.svc
namespace: sqs
syncPolicy:
automated:
prune: true
selfHeal: true