[Observer P2] LLM-powered pattern detection + anomaly analysis #25

New Issue

2026-09-09T23:42:54Z

poimen commented

2026-09-09 23:42:54 +00:00

What

Use the existing LLM infrastructure to analyze event streams for patterns, anomalies, and correlations that rule-based goals cannot catch.

Why

Simple threshold goals ("build failure rate > 5%") catch known-knowns. LLM analysis catches unknown-unknowns: "these 3 PRs touched the same file and all caused test failures" or "deploy latency correlates with this config change".

Design

Reuses action/llm/client.go from poimen-workflows. ObserverWorkflow batches recent events and periodically calls an LLM activity.

Depends On

#24 ObserverWorkflow
#21 Event router (events must be flowing)

Acceptance

LLM Analysis Activity

AnalyzeEventsActivity accepts batch of events (max 50) + current goal state → returns {patterns: [], confidence: float, suggested_actions: []}
LLM prompt includes event batch as structured JSON, not free text
LLM response parsed into typed struct — malformed LLM output returns error, not panic
LLM call timeout: 120s (reasoning model on V100 can be slow)
LLM unreachable → activity retried 2x, then returns empty patterns (graceful degradation)

Integration with Observer

ObserverWorkflow triggers analysis every N events (configurable, default 50) or every M minutes (default 30), whichever comes first
Detected patterns with confidence > 0.7 surfaced as Gotify notifications
Detected patterns with confidence > 0.9 trigger configured actions (e.g., start remediation)
Patterns with confidence < 0.5 logged but not acted on

Tests

go test ./action/observer_analyze_test.go — mocked LLM returns valid pattern → parsed correctly
go test ./action/observer_analyze_test.go — mocked LLM returns garbage → error returned, no panic
go test ./action/observer_analyze_test.go — mocked LLM timeout → retried, then graceful empty result
go test ./tests/observer_workflow_test.go — 50 events accumulated → analysis triggered → patterns dispatched

## What Use the existing LLM infrastructure to analyze event streams for patterns, anomalies, and correlations that rule-based goals cannot catch. ## Why Simple threshold goals ("build failure rate > 5%") catch known-knowns. LLM analysis catches unknown-unknowns: "these 3 PRs touched the same file and all caused test failures" or "deploy latency correlates with this config change". ## Design Reuses `action/llm/client.go` from poimen-workflows. ObserverWorkflow batches recent events and periodically calls an LLM activity. ## Depends On - #24 ObserverWorkflow - #21 Event router (events must be flowing) ## Acceptance ### LLM Analysis Activity - [ ] `AnalyzeEventsActivity` accepts batch of events (max 50) + current goal state → returns `{patterns: [], confidence: float, suggested_actions: []}` - [ ] LLM prompt includes event batch as structured JSON, not free text - [ ] LLM response parsed into typed struct — malformed LLM output returns error, not panic - [ ] LLM call timeout: 120s (reasoning model on V100 can be slow) - [ ] LLM unreachable → activity retried 2x, then returns empty patterns (graceful degradation) ### Integration with Observer - [ ] ObserverWorkflow triggers analysis every N events (configurable, default 50) or every M minutes (default 30), whichever comes first - [ ] Detected patterns with confidence > 0.7 surfaced as Gotify notifications - [ ] Detected patterns with confidence > 0.9 trigger configured actions (e.g., start remediation) - [ ] Patterns with confidence < 0.5 logged but not acted on ### Tests - [ ] `go test ./action/observer_analyze_test.go` — mocked LLM returns valid pattern → parsed correctly - [ ] `go test ./action/observer_analyze_test.go` — mocked LLM returns garbage → error returned, no panic - [ ] `go test ./action/observer_analyze_test.go` — mocked LLM timeout → retried, then graceful empty result - [ ] `go test ./tests/observer_workflow_test.go` — 50 events accumulated → analysis triggered → patterns dispatched

poimen added this to the Event-Driven Platform milestone 2026-09-09 23:42:54 +00:00

poimen added the priority/medium area/observer status/todo type/feature labels 2026-09-09 23:42:54 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: riotpiao-poimen/homelab#25