rock/poimen-memory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
cdfdae769b305852f9a51a691b517f7f54df21d8
poimen-memory/TEMPORAL_SKILLS_GUIDE.md
T

9.7 KiB
Raw Blame History

Guide: Writing & Uploading Skills/Memory for Temporal Workflows

Use the memory service to store context, tool patterns, and solutions that Temporal workflows can retrieve and use.

Overview

Three ways to get data into memory:

  1. Ingest transcripts (dialog with tool use + results) → system extracts skills
  2. Direct skill upload (manual structured skill)
  3. Git corpus (reference documentation, no extraction needed)

For Temporal workflows, option 1 (transcripts) is most powerful: you capture a successful workflow execution, memory system learns the pattern, and future workflows query it.

Format 1: Transcript-Based (Recommended)

Write a conversation showing a workflow using tools successfully. Memory extracts reusable skills.

File Format

Create JSONL (one JSON object per line):

{"role":"user","text":"Deploy service foo to prod","timestamp":"2025-01-15T10:00:00Z","source_position":0}
{"role":"assistant","text":"I'll deploy foo using kubectl","timestamp":"2025-01-15T10:00:01Z","source_position":1}
{"role":"tool_result","text":"kubectl apply -f foo.yaml\nDeployment foo created","timestamp":"2025-01-15T10:00:02Z","source_position":2}
{"role":"assistant","text":"Deployment successful","timestamp":"2025-01-15T10:00:03Z","source_position":3}

Fields:

  • role — user, assistant, tool_result, system
  • text — actual message/command/result
  • timestamp — ISO8601 (e.g., 2025-01-15T10:00:00Z)
  • source_position — line number in original source (for tracking)

Upload via HTTP

curl -X POST http://localhost:8080/memory/ingest \
  -H "apikey: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "temporal-workflows",
    "source": "transcript:slack/deployment-patterns",
    "ingest_id": "deploy-2025-01-15-abc123",
    "records": [
      {
        "role": "user",
        "text": "Deploy service foo to prod",
        "timestamp": "2025-01-15T10:00:00Z",
        "source_position": 0
      },
      {
        "role": "assistant",
        "text": "I'll deploy foo using kubectl apply",
        "timestamp": "2025-01-15T10:00:01Z",
        "source_position": 1
      },
      {
        "role": "tool_result",
        "text": "kubectl apply -f foo.yaml\nDeployment foo created",
        "timestamp": "2025-01-15T10:00:02Z",
        "source_position": 2
      }
    ]
  }'

Response:

{
  "ingest_id": "deploy-2025-01-15-abc123",
  "status": "pending",
  "status_url": "/memory/ingest/deploy-2025-01-15-abc123"
}

Check status:

curl -H "apikey: YOUR_API_KEY" \
  http://localhost:8080/memory/ingest/deploy-2025-01-15-abc123

Format 2: Structured Skill (Direct)

If you want to upload a pre-written skill without going through extraction:

YAML Format (for manual storage)

Create skills/temporal-patterns.yaml:

name: "kubernetes_deploy_pattern"
description: "Safe deployment pattern using kubectl apply with validation"
when_to_use: "When deploying services to Kubernetes cluster"
examples:
  - |
    kubectl apply -f service.yaml
    kubectl rollout status deployment/service -n default
    kubectl get pods -n default
prerequisites:
  - "kubectl binary installed"
  - "kubeconfig configured"
  - "deployment manifest exists"
steps:
  - "Validate manifest: kubectl apply -f service.yaml --dry-run=client"
  - "Apply: kubectl apply -f service.yaml"
  - "Monitor: kubectl rollout status deployment/service -n default"
  - "Verify: kubectl get pods, check for Ready status"
precautions:
  - "Never use --force unless necessary"
  - "Always check diff before applying to prod"
  - "Rollback plan: kubectl rollout undo deployment/service"

Then ingest as system memory:

curl -X POST http://localhost:8080/memory/ingest \
  -H "apikey: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "temporal-workflows",
    "source": "skill:manual/kubernetes",
    "ingest_id": "skill-k8s-deploy-001",
    "records": [
      {
        "role": "system",
        "text": "SKILL: kubernetes_deploy_pattern\n\nSafe deployment pattern using kubectl apply with validation\n\nWhen to use: When deploying services to Kubernetes cluster\n\nSteps:\n1. Validate manifest: kubectl apply -f service.yaml --dry-run=client\n2. Apply: kubectl apply -f service.yaml\n3. Monitor: kubectl rollout status deployment/service -n default\n4. Verify: kubectl get pods, check for Ready status\n\nPrecautions:\n- Never use --force unless necessary\n- Always check diff before applying to prod\n- Rollback plan: kubectl rollout undo deployment/service",
        "timestamp": "2025-01-15T10:00:00Z",
        "source_position": 0
      }
    ]
  }'

Format 3: Git Corpus (Reference Docs)

Documentation (never evidence, read-only for context).

Store in your obsidian-memory repo, then:

mem ingest --source git:ssh://[email protected]:2222/rock/poimen-obesdient-memory.git \
  --project temporal-workflows

(This will be auto-triggered by CI once M3.5.9 is done.)

Querying Skills in Temporal Workflows

Get all skills for a project

curl -H "apikey: YOUR_API_KEY" \
  "http://localhost:8080/memory/skills?project=temporal-workflows"

Response:

{
  "skills": [
    {
      "name": "kubernetes_deploy_pattern",
      "description": "Safe deployment pattern using kubectl apply with validation",
      "when_to_use": "When deploying services to Kubernetes cluster"
    },
    {
      "name": "postgres_backup_pattern",
      "description": "Automated backup with verification",
      "when_to_use": "Backup Postgres database before migrations"
    }
  ],
  "count": 2
}

Get tool context (tool failure context + similar cases)

curl -H "apikey: YOUR_API_KEY" \
  "http://localhost:8080/memory/context?tool=kubectl&error=connection+refused"

Returns:

  • Tier 1 — Exact match (same error + context)
  • Tier 2 — Similar symptom (vector search)
  • Tier 3 — Reference docs (R corpus)

Note: This endpoint is M3.7.4 (in progress).

Example: Temporal Activity + Memory Query

// activity.go
func QueryMemoryForPattern(ctx context.Context, toolName string, errorMsg string) (string, error) {
    resp, err := http.Get(fmt.Sprintf(
        "http://memory-service/memory/context?tool=%s&error=%s",
        url.QueryEscape(toolName),
        url.QueryEscape(errorMsg),
    ))
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()
    
    var result map[string]interface{}
    json.NewDecoder(resp.Body).Decode(&result)
    
    // Use tier 1 (exact match) if available, else tier 2 (symptom), else tier 3 (docs)
    if tier1, ok := result["tier_1"].(string); ok && tier1 != "" {
        return tier1, nil
    }
    // ... same for tier 2, tier 3
    
    return "", nil
}

Best Practices

  1. Source naming — source field identifies where data came from:

    • transcript:slack/topic
    • transcript:github/issue-123
    • skill:manual/pattern-name
    • git:ssh://git@.../repo.git
    • doc:obsidian-vault/path/to/note

  2. Idempotency — ingest_id must be stable (use SHA256 of content):

    ingest_id=$(echo "temporal-deploy-pattern-2025-01-15" | sha256sum | cut -d' ' -f1)

  3. Batch ingests — upload multiple transcripts in one request to reduce overhead.

  4. Timestamps — use workflow execution time, not current time. Helps memory system understand sequence.

  5. Project naming — use consistent project keys (e.g., temporal-workflows, agent-rust, poimen).

Ingesting from Temporal Directly

Pseudo-code (implement in your Temporal activity):

func IngestWorkflowToMemory(ctx context.Context, execution WorkflowExecution) error {
    records := []Record{}
    
    // Walk through history and extract tool results
    for _, event := range execution.History.Events {
        if event.Type == "ActivityCompleted" {
            records = append(records, Record{
                Role:      "tool_result",
                Text:      event.Result,
                Timestamp: event.Timestamp,
            })
        }
    }
    
    // POST to /memory/ingest
    body := map[string]interface{}{
        "project": "temporal-workflows",
        "source": "temporal:workflow/" + execution.WorkflowID,
        "ingest_id": execution.RunID, // idempotent
        "records": records,
    }
    
    resp, err := http.Post("http://memory-service/memory/ingest", 
        "application/json", 
        jsonBody(body),
    )
    // ...
}

Then, in your workflow, query back:

func QueryMemoryInWorkflow(ctx context.Context, q string) ([]Result, error) {
    resp, _ := http.Get(fmt.Sprintf(
        "http://memory-service/memory/query?project=temporal-workflows&query=%s",
        url.QueryEscape(q),
    ))
    // ...
}

Troubleshooting

Issue Cause Fix
401 unauthorized Missing apikey header Add -H "apikey: YOUR_KEY"
202 then status pending forever Ingest worker not running Check mem serve is running with DB connection
Skills not appearing M4.1 (extraction) not implemented yet Use Format 2 (direct skill) for now
Rate limited (429) Hit limit for project Check rate limit, wait or use different apikey
Duplicate ingest_id Same payload ingested twice Intentional (idempotency); returns same job_id

Timeline

Task Status Impact
M3.5.2 (ingest endpoint) ✅ Upload transcripts now
M3.5.5 (skills endpoint) ✅ Query skills now
M4.1 (skill extraction) 🟡 Automatic extraction in progress
M3.7.4 (context endpoint) ⬜ Tier-1 lookup not yet available
M3.7.8 (symptom projection) ⬜ Tier-2 vector lookup not yet available

Actionable now: Formats 1 & 2, endpoints work. Extract by hand or via M4.1 when ready.

Reference in New Issue View Git Blame Copy Permalink