Author SHA1 Message Date
poimen 7be93d1d16 fix: remove duplicate PersistInput type definition
CI / CI (pull_request) Successful in 4m33s
Type already re-exported from pkg/types at top of file
2026-09-08 17:06:44 -07:00
poimen 5ccc3711e6 refactor: extract synthesis types to pkg/types
CI / CI (pull_request) Failing after 2m17s
DRY fix: Shared types (SynthesisInput, ExtractedEntity, ExtractedFact,
ContradictionResult, PersistInput) moved to pkg/types/synthesis.go.

Both workflow and activity packages now import from pkg/types.
Re-exported as type aliases for backward compatibility.

Fixes: Duplicate type definitions between workflow and activity packages.
2026-09-08 17:05:17 -07:00
poimen 9020ccf839 feat(phase-2.1): synthesis workflow definition
CI / CI (pull_request) Successful in 4m1s
4-stage pipeline as Temporal workflow:
  Stage 1: ChunkAndEmbed — chunk text + generate embeddings
  Stage 2: ExtractEntities — LLM entity extraction with reflection
  Stage 3: ExtractFacts — pattern + LLM fact extraction
  Stage 4: DetectContradictions — pre-filter + LLM verification
  Stage 5: PersistSynthesis — save all results to DB

Types: SynthesisInput, SynthesisResult, ExtractedEntity,
       ExtractedFact, ContradictionResult, PersistInput

Retry: 3 attempts, exponential backoff (1s → 2s → 4s)
Each stage fails independently with wrapped errors.

Tests: 5 pass (success, contradictions, entity fail, chunk fail, fields)
Build: clean, 36 packages pass
2026-09-08 16:35:36 -07:00
rock e5a773054b ci: unified workflow - single job, DOCKER_HOST, build+push on all events (#7)
CI / CI (push) Successful in 5m7s
- Single job (no split test/build-push)
- DOCKER_HOST=tcp://localhost:2375 for dind
- Build + push on PRs too (verify before merge)
- workflow_dispatch for manual trigger

---------

Reviewed-on: rock/poimen-workflows#7
2026-09-07 20:53:36 +00:00
rockandTest 70442e94b4 fix: standardize poimen-workflows CI to unified pattern (#5)
CI / Test (push) Successful in 2m10s
CI / Build & Push Image (push) Failing after 1m13s
Unified pattern enforced:
- test job: runs on all branches + PRs
- build-push job: only on main push, depends on test
- Proper env vars (GOPRIVATE, REGISTRY, IMAGE)
- Install Node.js before checkout
- Install docker only in build-push
- Docker login + build + push + prune

---------

Co-authored-by: Test <[email protected]>
Reviewed-on: rock/poimen-workflows#5
2026-09-07 07:14:46 +00:00
rockandTest 45b7f8ca61 fix: use env vars for docker registry credentials (#4)
CI / Test (push) Successful in 2m7s
CI / Build & Push Image (push) Failing after 1m5s
Fix registry login by passing FORGEJO_REGISTRY_USER and FORGEJO_REGISTRY_TOKEN via environment variables instead of direct secret interpolation.

Uses the proven pattern from riotpiao.com reference commit.

This prevents credentials from being exposed in logs or shell history while keeping the standard docker login approach.

After merge + org-level secrets configured:
- All repos inherit FORGEJO_REGISTRY_USER and FORGEJO_REGISTRY_TOKEN
- CI validates credentials exist before docker login
- Image pushed to registry on main push

---------

Co-authored-by: Test <[email protected]>
Reviewed-on: rock/poimen-workflows#4
2026-09-07 06:48:25 +00:00
5 changed files with 374 additions and 98 deletions
+11 -20
View File
@@ -5,19 +5,23 @@ on:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
env:
GOPRIVATE: forgejo.riotpiao.com
REGISTRY: forgejo.riotpiao.com
IMAGE: forgejo.riotpiao.com/rock/poimen-workflows
DOCKER_HOST: tcp://localhost:2375
jobs:
test:
name: Test
ci:
name: CI
runs-on: golang
steps:
- name: Install Node.js for actions runtime
run: apt-get update && apt-get install -y nodejs
- name: Install Node.js and Docker
run: |
apt-get update
apt-get install -y nodejs docker.io
- name: Checkout code
uses: actions/checkout@v4
@@ -34,18 +38,6 @@ jobs:
- name: Build binary
run: CGO_ENABLED=0 GOOS=linux go build -o /tmp/poimen-worker ./cmd/worker
build-push:
name: Build & Push Image
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: golang
steps:
- name: Install Node.js and Docker
run: apt-get update && apt-get install -y nodejs docker.io
- name: Checkout code
uses: actions/checkout@v4
- name: Get short SHA
id: sha
run: echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
@@ -58,18 +50,17 @@ jobs:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
- name: Build and push image
- name: Build Docker image
run: |
docker build --no-cache \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
-t "${IMAGE}:latest" .
- name: Push Docker image
run: |
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
echo "✓ Image pushed: ${IMAGE}:${{ steps.sha.outputs.short_sha }}"
echo "✓ Pushed: ${IMAGE}:${{ steps.sha.outputs.short_sha }}"
- name: Prune unused images
run: docker image prune -a --force 2>&1 | tail -3 || true
-78
View File
@@ -1,78 +0,0 @@
# Forgejo Registry Secrets Configuration
## One-Time Setup (Org Level)
All repos in the `rock` org share the same Forgejo registry credentials.
### Configure at Organization Level
1. Navigate to: https://forgejo.riotpiao.com/rock
2. Click Settings (gear icon)
3. Go to: Actions → Secrets
4. Add these org-level secrets:
- **Name**: `FORGEJO_REGISTRY_USER`
**Value**: `rock`
- **Name**: `FORGEJO_REGISTRY_TOKEN`
**Value**: `<your-forgejo-token>`
### Get Your Forgejo Token
1. Go to: https://forgejo.riotpiao.com/user/settings/applications
2. Click "Generate New Token"
3. Set scopes: `api`, `read:registry`, `write:registry`
4. Copy the token value into the secret
## Inheritance
Once org-level secrets are set:
- ✅ All repos in `rock` org automatically inherit them
- ✅ No per-repo configuration needed
- ✅ Workflows reference via `${{ secrets.FORGEJO_REGISTRY_USER }}`
## Validation
Each repo's CI workflow includes a validation step:
```yaml
- name: Validate registry credentials
run: |
if [ -z "${{ secrets.FORGEJO_REGISTRY_USER }}" ] || [ -z "${{ secrets.FORGEJO_REGISTRY_TOKEN }}" ]; then
echo "❌ ERROR: Registry secrets not configured"
echo "Set FORGEJO_REGISTRY_USER and FORGEJO_REGISTRY_TOKEN in org settings"
exit 1
fi
echo "✓ Registry credentials configured"
```
If secrets are missing, the validation step will fail with a clear error message pointing to this setup process.
## Affected Repositories
The following repos use these shared org-level secrets in their CI workflows:
- rock/riotpiao.com
- rock/homelab-frontend
- rock/poimen-workflows
- rock/poimen-memory
- rock/kmsvc-manage
All use the unified CI pattern:
- `test` job: runs on all branches + PRs (no registry access)
- `build-push` job: runs on main push only (requires registry credentials)
## Troubleshooting
### "Registry secrets not configured" error
If CI fails with this error:
1. Check org settings: https://forgejo.riotpiao.com/rock/settings/actions/secrets
2. Verify both secrets exist and are not empty
3. Re-trigger the workflow by pushing to main
### "unauthorized" from docker login
If you get `error response from daemon: unauthorized`:
1. Check the token value is correct (copy-paste carefully)
2. Verify token has `read:registry` and `write:registry` scopes
3. Generate a new token if the old one expired
+59
View File
@@ -0,0 +1,59 @@
package types
import "time"
// SynthesisInput contains the input for the synthesis workflow.
type SynthesisInput struct {
Project string `json:"project"`
Source string `json:"source"`
Text string `json:"text"`
Kind string `json:"kind"` // L1, L2, reference
Tags []string `json:"tags,omitempty"`
}
// SynthesisResult contains the output of the synthesis workflow.
type SynthesisResult struct {
ChunkID string `json:"chunk_id"`
EntitiesExtracted int `json:"entities_extracted"`
FactsExtracted int `json:"facts_extracted"`
Contradictions int `json:"contradictions"`
ReviewQueued int `json:"review_queued"`
Entities []ExtractedEntity `json:"entities"`
Facts []ExtractedFact `json:"facts"`
Duration time.Duration `json:"duration"`
}
// ExtractedEntity represents an entity found during synthesis.
type ExtractedEntity struct {
Name string `json:"name"`
EntityType string `json:"entity_type"`
Confidence float64 `json:"confidence"`
}
// ExtractedFact represents a fact extracted during synthesis.
type ExtractedFact struct {
Subject string `json:"subject"`
Predicate string `json:"predicate"`
Object string `json:"object"`
Confidence float64 `json:"confidence"`
}
// ContradictionResult represents a contradiction detection result.
type ContradictionResult struct {
FactA ExtractedFact `json:"fact_a"`
FactB ExtractedFact `json:"fact_b"`
Severity string `json:"severity"` // low, medium, high
AutoResolved bool `json:"auto_resolved"`
QueuedReview bool `json:"queued_review"`
}
// PersistInput groups all synthesis results for persistence.
type PersistInput struct {
ChunkID string `json:"chunk_id"`
Project string `json:"project"`
Source string `json:"source"`
Kind string `json:"kind"`
Entities []ExtractedEntity `json:"entities"`
Facts []ExtractedFact `json:"facts"`
Contradictions []ContradictionResult `json:"contradictions"`
}
+125
View File
@@ -0,0 +1,125 @@
package workflow
import (
"fmt"
"time"
"go.temporal.io/sdk/temporal"
"go.temporal.io/sdk/workflow"
"github.com/rockliang/poimen/workflows/pkg/types"
)
// Re-export shared types from pkg/types for backward compatibility
type SynthesisInput = types.SynthesisInput
type SynthesisResult = types.SynthesisResult
type ExtractedEntity = types.ExtractedEntity
type ExtractedFact = types.ExtractedFact
type ContradictionResult = types.ContradictionResult
type PersistInput = types.PersistInput
var synthesisActivityOptions = workflow.ActivityOptions{
StartToCloseTimeout: 60 * time.Second,
RetryPolicy: &temporal.RetryPolicy{
InitialInterval: time.Second,
BackoffCoefficient: 2.0,
MaximumInterval: 30 * time.Second,
MaximumAttempts: 3,
},
}
// SynthesisWorkflow orchestrates the 4-stage memory synthesis pipeline.
//
// Stage 1: Chunk + embed text
// Stage 2: Extract entities (LLM + reflection)
// Stage 3: Extract facts (pattern + LLM)
// Stage 4: Detect contradictions (pre-filter + LLM)
//
// Each stage is an activity with independent retry policy.
func SynthesisWorkflow(ctx workflow.Context, input SynthesisInput) (*SynthesisResult, error) {
logger := workflow.GetLogger(ctx)
startTime := workflow.Now(ctx)
logger.Info("synthesis started",
"project", input.Project,
"source", input.Source,
"kind", input.Kind,
)
actCtx := workflow.WithActivityOptions(ctx, synthesisActivityOptions)
// Stage 1: Chunk + Embed
var chunkID string
err := workflow.ExecuteActivity(actCtx, "ChunkAndEmbedActivity", input).Get(ctx, &chunkID)
if err != nil {
return nil, fmt.Errorf("stage 1 chunk+embed: %w", err)
}
logger.Info("stage 1 complete", "chunk_id", chunkID)
// Stage 2: Entity Extraction
var entities []ExtractedEntity
err = workflow.ExecuteActivity(actCtx, "ExtractEntitiesActivity", chunkID, input.Text).Get(ctx, &entities)
if err != nil {
return nil, fmt.Errorf("stage 2 entity extraction: %w", err)
}
logger.Info("stage 2 complete", "entities", len(entities))
// Stage 3: Fact Extraction
var facts []ExtractedFact
err = workflow.ExecuteActivity(actCtx, "ExtractFactsActivity", chunkID, input.Text, entities).Get(ctx, &facts)
if err != nil {
return nil, fmt.Errorf("stage 3 fact extraction: %w", err)
}
logger.Info("stage 3 complete", "facts", len(facts))
// Stage 4: Contradiction Detection
var contradictions []ContradictionResult
err = workflow.ExecuteActivity(actCtx, "DetectContradictionsActivity", input.Project, facts).Get(ctx, &contradictions)
if err != nil {
return nil, fmt.Errorf("stage 4 contradiction detection: %w", err)
}
reviewQueued := 0
for _, c := range contradictions {
if c.QueuedReview {
reviewQueued++
}
}
logger.Info("stage 4 complete", "contradictions", len(contradictions), "review_queued", reviewQueued)
// Stage 5: Persist results
persistInput := PersistInput{
ChunkID: chunkID,
Project: input.Project,
Source: input.Source,
Kind: input.Kind,
Entities: entities,
Facts: facts,
Contradictions: contradictions,
}
err = workflow.ExecuteActivity(actCtx, "PersistSynthesisActivity", persistInput).Get(ctx, nil)
if err != nil {
return nil, fmt.Errorf("stage 5 persist: %w", err)
}
duration := workflow.Now(ctx).Sub(startTime)
result := &SynthesisResult{
ChunkID: chunkID,
EntitiesExtracted: len(entities),
FactsExtracted: len(facts),
Contradictions: len(contradictions),
ReviewQueued: reviewQueued,
Entities: entities,
Facts: facts,
Duration: duration,
}
logger.Info("synthesis complete",
"chunk_id", chunkID,
"entities", len(entities),
"facts", len(facts),
"contradictions", len(contradictions),
"duration", duration,
)
return result, nil
}
+179
View File
@@ -0,0 +1,179 @@
package workflow
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"go.temporal.io/sdk/testsuite"
)
// Stub activity functions for test registration
func ChunkAndEmbedActivity(_ context.Context, _ SynthesisInput) (string, error) { return "", nil }
func ExtractEntitiesActivity(_ context.Context, _ string, _ string) ([]ExtractedEntity, error) { return nil, nil }
func ExtractFactsActivity(_ context.Context, _ string, _ string, _ []ExtractedEntity) ([]ExtractedFact, error) { return nil, nil }
func DetectContradictionsActivity(_ context.Context, _ string, _ []ExtractedFact) ([]ContradictionResult, error) { return nil, nil }
func PersistSynthesisActivity(_ context.Context, _ PersistInput) error { return nil }
func registerSynthesisActivities(env *testsuite.TestWorkflowEnvironment) {
env.RegisterActivity(ChunkAndEmbedActivity)
env.RegisterActivity(ExtractEntitiesActivity)
env.RegisterActivity(ExtractFactsActivity)
env.RegisterActivity(DetectContradictionsActivity)
env.RegisterActivity(PersistSynthesisActivity)
}
func TestSynthesisWorkflow_Success(t *testing.T) {
ts := &testsuite.WorkflowTestSuite{}
env := ts.NewTestWorkflowEnvironment()
input := SynthesisInput{
Project: "poimen",
Source: "transcript://test-123",
Text: "Kubernetes uses port 8080 for the API server",
Kind: "L1",
}
registerSynthesisActivities(env)
// Stage 1: Chunk + Embed
env.OnActivity(ChunkAndEmbedActivity, mock.Anything, input).Return("chunk-abc123", nil)
// Stage 2: Entity Extraction
entities := []ExtractedEntity{
{Name: "Kubernetes", EntityType: "tool", Confidence: 0.95},
{Name: "API server", EntityType: "component", Confidence: 0.90},
}
env.OnActivity(ExtractEntitiesActivity, mock.Anything, "chunk-abc123", input.Text).Return(entities, nil)
// Stage 3: Fact Extraction
facts := []ExtractedFact{
{Subject: "Kubernetes", Predicate: "uses_port", Object: "8080", Confidence: 0.85},
}
env.OnActivity(ExtractFactsActivity, mock.Anything, "chunk-abc123", input.Text, entities).Return(facts, nil)
// Stage 4: Contradiction Detection
contradictions := []ContradictionResult{}
env.OnActivity(DetectContradictionsActivity, mock.Anything, "poimen", facts).Return(contradictions, nil)
// Stage 5: Persist
env.OnActivity(PersistSynthesisActivity, mock.Anything, mock.Anything).Return(nil)
env.ExecuteWorkflow(SynthesisWorkflow, input)
assert.True(t, env.IsWorkflowCompleted())
assert.NoError(t, env.GetWorkflowError())
var result SynthesisResult
assert.NoError(t, env.GetWorkflowResult(&result))
assert.Equal(t, "chunk-abc123", result.ChunkID)
assert.Equal(t, 2, result.EntitiesExtracted)
assert.Equal(t, 1, result.FactsExtracted)
assert.Equal(t, 0, result.Contradictions)
assert.Equal(t, 0, result.ReviewQueued)
}
func TestSynthesisWorkflow_WithContradictions(t *testing.T) {
ts := &testsuite.WorkflowTestSuite{}
env := ts.NewTestWorkflowEnvironment()
registerSynthesisActivities(env)
input := SynthesisInput{
Project: "poimen",
Source: "transcript://test-456",
Text: "Port 8080 is used by nginx",
Kind: "L1",
}
env.OnActivity(ChunkAndEmbedActivity, mock.Anything, input).Return("chunk-def456", nil)
entities := []ExtractedEntity{
{Name: "nginx", EntityType: "tool", Confidence: 0.92},
}
env.OnActivity(ExtractEntitiesActivity, mock.Anything, "chunk-def456", input.Text).Return(entities, nil)
facts := []ExtractedFact{
{Subject: "nginx", Predicate: "uses_port", Object: "8080", Confidence: 0.88},
}
env.OnActivity(ExtractFactsActivity, mock.Anything, "chunk-def456", input.Text, entities).Return(facts, nil)
contradictions := []ContradictionResult{
{
FactA: ExtractedFact{Subject: "Kubernetes", Predicate: "uses_port", Object: "8080"},
FactB: ExtractedFact{Subject: "nginx", Predicate: "uses_port", Object: "8080"},
Severity: "medium",
AutoResolved: false,
QueuedReview: true,
},
}
env.OnActivity(DetectContradictionsActivity, mock.Anything, "poimen", facts).Return(contradictions, nil)
env.OnActivity(PersistSynthesisActivity, mock.Anything, mock.Anything).Return(nil)
env.ExecuteWorkflow(SynthesisWorkflow, input)
assert.True(t, env.IsWorkflowCompleted())
assert.NoError(t, env.GetWorkflowError())
var result SynthesisResult
assert.NoError(t, env.GetWorkflowResult(&result))
assert.Equal(t, 1, result.Contradictions)
assert.Equal(t, 1, result.ReviewQueued)
}
func TestSynthesisWorkflow_EntityExtractionFails(t *testing.T) {
ts := &testsuite.WorkflowTestSuite{}
env := ts.NewTestWorkflowEnvironment()
registerSynthesisActivities(env)
input := SynthesisInput{
Project: "poimen",
Source: "transcript://test-789",
Text: "Some text",
Kind: "L1",
}
env.OnActivity(ChunkAndEmbedActivity, mock.Anything, input).Return("chunk-xyz", nil)
env.OnActivity(ExtractEntitiesActivity, mock.Anything, "chunk-xyz", input.Text).
Return(nil, assert.AnError)
env.ExecuteWorkflow(SynthesisWorkflow, input)
assert.True(t, env.IsWorkflowCompleted())
assert.Error(t, env.GetWorkflowError())
assert.Contains(t, env.GetWorkflowError().Error(), "stage 2 entity extraction")
}
func TestSynthesisWorkflow_ChunkFails(t *testing.T) {
ts := &testsuite.WorkflowTestSuite{}
env := ts.NewTestWorkflowEnvironment()
registerSynthesisActivities(env)
input := SynthesisInput{
Project: "poimen",
Source: "transcript://test-fail",
Text: "Bad text",
Kind: "L1",
}
env.OnActivity(ChunkAndEmbedActivity, mock.Anything, input).Return("", assert.AnError)
env.ExecuteWorkflow(SynthesisWorkflow, input)
assert.True(t, env.IsWorkflowCompleted())
assert.Error(t, env.GetWorkflowError())
assert.Contains(t, env.GetWorkflowError().Error(), "stage 1 chunk+embed")
}
func TestSynthesisInput_Fields(t *testing.T) {
input := SynthesisInput{
Project: "test",
Source: "source://1",
Text: "hello",
Kind: "L2",
Tags: []string{"tag1", "tag2"},
}
assert.Equal(t, "test", input.Project)
assert.Equal(t, "L2", input.Kind)
assert.Len(t, input.Tags, 2)
}