[Phase 2.3] Compaction workflow definition #15

New Issue

2026-09-09T01:04:02Z

poimen commented

2026-09-09 01:04:02 +00:00

Goal

Define a Temporal workflow that runs 2-tier compaction: exact dedup (Tier 1, no LLM) then semantic dedup (Tier 2, LLM-gated).

Workflow Stages

ExactDedup — find edges with identical (source, target, relation, fact), keep newest, soft-delete rest
StaleGC — find edges where t_invalid IS NOT NULL and age > 30 days, hard-delete
SemanticDedup — pre-filter candidate pairs by embedding cosine > 0.9, then LLM verify merge
AuditLog — write compaction summary to memory service

Types

CompactionInput { project, dry_run, tier } (tier: 1=exact only, 2=exact+semantic)
CompactionResult { exact_deduped, stale_gc, semantic_merged, audit_id, duration }

Acceptance Test

// workflow/compaction_test.go using Temporal test framework
func TestCompactionWorkflow_Tier1Only(t *testing.T) {
    // Mock ExactDedupActivity returns 5 deduped
    // Mock StaleGCActivity returns 3 cleaned
    // SemanticDedup NOT called (tier=1)
    // Assert: result.exact_deduped == 5, result.stale_gc == 3, result.semantic_merged == 0
}

func TestCompactionWorkflow_Tier2(t *testing.T) {
    // All 4 stages called
    // Assert: all counts populated, audit_id non-empty
}

func TestCompactionWorkflow_DryRun(t *testing.T) {
    // All stages called but PersistCompaction receives dry_run=true
    // Assert: no actual mutations, audit shows dry_run
}

Effort

~150 LOC, 2 days

## Goal Define a Temporal workflow that runs 2-tier compaction: exact dedup (Tier 1, no LLM) then semantic dedup (Tier 2, LLM-gated). ## Workflow Stages 1. **ExactDedup** — find edges with identical (source, target, relation, fact), keep newest, soft-delete rest 2. **StaleGC** — find edges where `t_invalid IS NOT NULL` and age > 30 days, hard-delete 3. **SemanticDedup** — pre-filter candidate pairs by embedding cosine > 0.9, then LLM verify merge 4. **AuditLog** — write compaction summary to memory service ## Types - `CompactionInput { project, dry_run, tier }` (tier: 1=exact only, 2=exact+semantic) - `CompactionResult { exact_deduped, stale_gc, semantic_merged, audit_id, duration }` ## Acceptance Test ```go // workflow/compaction_test.go using Temporal test framework func TestCompactionWorkflow_Tier1Only(t *testing.T) { // Mock ExactDedupActivity returns 5 deduped // Mock StaleGCActivity returns 3 cleaned // SemanticDedup NOT called (tier=1) // Assert: result.exact_deduped == 5, result.stale_gc == 3, result.semantic_merged == 0 } func TestCompactionWorkflow_Tier2(t *testing.T) { // All 4 stages called // Assert: all counts populated, audit_id non-empty } func TestCompactionWorkflow_DryRun(t *testing.T) { // All stages called but PersistCompaction receives dry_run=true // Assert: no actual mutations, audit shows dry_run } ``` ## Effort ~150 LOC, 2 days

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: riotpiao-poimen/poimen-workflows#15