poimen-memory/TEMPORAL_SKILLS_GUIDE.md

# Guide: Writing & Uploading Skills/Memory for Temporal Workflows

Use the memory service to store context, tool patterns, and solutions that Temporal workflows can retrieve and use.

## Overview

**Three ways to get data into memory:**

1. **Ingest transcripts** (dialog with tool use + results) → system extracts skills
2. **Direct skill upload** (manual structured skill)
3. **Git corpus** (reference documentation, no extraction needed)

For Temporal workflows, option **1 (transcripts)** is most powerful: you capture a successful workflow execution, memory system learns the pattern, and future workflows query it.

---

## Format 1: Transcript-Based (Recommended)

Write a conversation showing a workflow using tools successfully. Memory extracts reusable skills.

### File Format

Create JSONL (one JSON object per line):

```jsonl
{"role":"user","text":"Deploy service foo to prod","timestamp":"2025-01-15T10:00:00Z","source_position":0}
{"role":"assistant","text":"I'll deploy foo using kubectl","timestamp":"2025-01-15T10:00:01Z","source_position":1}
{"role":"tool_result","text":"kubectl apply -f foo.yaml\nDeployment foo created","timestamp":"2025-01-15T10:00:02Z","source_position":2}
{"role":"assistant","text":"Deployment successful","timestamp":"2025-01-15T10:00:03Z","source_position":3}
```

**Fields:**
- `role` — `user`, `assistant`, `tool_result`, `system`
- `text` — actual message/command/result
- `timestamp` — ISO8601 (e.g., `2025-01-15T10:00:00Z`)
- `source_position` — line number in original source (for tracking)

### Upload via HTTP

```bash
curl -X POST http://localhost:8080/memory/ingest \
  -H "apikey: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "temporal-workflows",
    "source": "transcript:slack/deployment-patterns",
    "ingest_id": "deploy-2025-01-15-abc123",
    "records": [
      {
        "role": "user",
        "text": "Deploy service foo to prod",
        "timestamp": "2025-01-15T10:00:00Z",
        "source_position": 0
      },
      {
        "role": "assistant",
        "text": "I'll deploy foo using kubectl apply",
        "timestamp": "2025-01-15T10:00:01Z",
        "source_position": 1
      },
      {
        "role": "tool_result",
        "text": "kubectl apply -f foo.yaml\nDeployment foo created",
        "timestamp": "2025-01-15T10:00:02Z",
        "source_position": 2
      }
    ]
  }'
```

**Response:**
```json
{
  "ingest_id": "deploy-2025-01-15-abc123",
  "status": "pending",
  "status_url": "/memory/ingest/deploy-2025-01-15-abc123"
}
```

Check status:
```bash
curl -H "apikey: YOUR_API_KEY" \
  http://localhost:8080/memory/ingest/deploy-2025-01-15-abc123
```

---

## Format 2: Structured Skill (Direct)

If you want to upload a pre-written skill without going through extraction:

### YAML Format (for manual storage)

Create `skills/temporal-patterns.yaml`:

```yaml
name: "kubernetes_deploy_pattern"
description: "Safe deployment pattern using kubectl apply with validation"
when_to_use: "When deploying services to Kubernetes cluster"
examples:
  - |
    kubectl apply -f service.yaml
    kubectl rollout status deployment/service -n default
    kubectl get pods -n default
prerequisites:
  - "kubectl binary installed"
  - "kubeconfig configured"
  - "deployment manifest exists"
steps:
  - "Validate manifest: kubectl apply -f service.yaml --dry-run=client"
  - "Apply: kubectl apply -f service.yaml"
  - "Monitor: kubectl rollout status deployment/service -n default"
  - "Verify: kubectl get pods, check for Ready status"
precautions:
  - "Never use --force unless necessary"
  - "Always check diff before applying to prod"
  - "Rollback plan: kubectl rollout undo deployment/service"
```

Then ingest as system memory:

```bash
curl -X POST http://localhost:8080/memory/ingest \
  -H "apikey: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "temporal-workflows",
    "source": "skill:manual/kubernetes",
    "ingest_id": "skill-k8s-deploy-001",
    "records": [
      {
        "role": "system",
        "text": "SKILL: kubernetes_deploy_pattern\n\nSafe deployment pattern using kubectl apply with validation\n\nWhen to use: When deploying services to Kubernetes cluster\n\nSteps:\n1. Validate manifest: kubectl apply -f service.yaml --dry-run=client\n2. Apply: kubectl apply -f service.yaml\n3. Monitor: kubectl rollout status deployment/service -n default\n4. Verify: kubectl get pods, check for Ready status\n\nPrecautions:\n- Never use --force unless necessary\n- Always check diff before applying to prod\n- Rollback plan: kubectl rollout undo deployment/service",
        "timestamp": "2025-01-15T10:00:00Z",
        "source_position": 0
      }
    ]
  }'
```

---

## Format 3: Git Corpus (Reference Docs)

Documentation (never evidence, read-only for context).

Store in your obsidian-memory repo, then:

```bash
mem ingest --source git:ssh://[email protected]:2222/rock/poimen-obesdient-memory.git \
  --project temporal-workflows
```

(This will be auto-triggered by CI once M3.5.9 is done.)

---

## Querying Skills in Temporal Workflows

### Get all skills for a project

```bash
curl -H "apikey: YOUR_API_KEY" \
  "http://localhost:8080/memory/skills?project=temporal-workflows"
```

**Response:**
```json
{
  "skills": [
    {
      "name": "kubernetes_deploy_pattern",
      "description": "Safe deployment pattern using kubectl apply with validation",
      "when_to_use": "When deploying services to Kubernetes cluster"
    },
    {
      "name": "postgres_backup_pattern",
      "description": "Automated backup with verification",
      "when_to_use": "Backup Postgres database before migrations"
    }
  ],
  "count": 2
}
```

### Get tool context (tool failure context + similar cases)

```bash
curl -H "apikey: YOUR_API_KEY" \
  "http://localhost:8080/memory/context?tool=kubectl&error=connection+refused"
```

Returns:
- **Tier 1** — Exact match (same error + context)
- **Tier 2** — Similar symptom (vector search)
- **Tier 3** — Reference docs (R corpus)

*Note: This endpoint is M3.7.4 (in progress).*

---

## Example: Temporal Activity + Memory Query

```go
// activity.go
func QueryMemoryForPattern(ctx context.Context, toolName string, errorMsg string) (string, error) {
    resp, err := http.Get(fmt.Sprintf(
        "http://memory-service/memory/context?tool=%s&error=%s",
        url.QueryEscape(toolName),
        url.QueryEscape(errorMsg),
    ))
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()
    
    var result map[string]interface{}
    json.NewDecoder(resp.Body).Decode(&result)
    
    // Use tier 1 (exact match) if available, else tier 2 (symptom), else tier 3 (docs)
    if tier1, ok := result["tier_1"].(string); ok && tier1 != "" {
        return tier1, nil
    }
    // ... same for tier 2, tier 3
    
    return "", nil
}
```

---

## Best Practices

1. **Source naming** — `source` field identifies where data came from:
   - `transcript:slack/topic`
   - `transcript:github/issue-123`
   - `skill:manual/pattern-name`
   - `git:ssh://git@.../repo.git`
   - `doc:obsidian-vault/path/to/note`

2. **Idempotency** — `ingest_id` must be stable (use SHA256 of content):
   ```bash
   ingest_id=$(echo "temporal-deploy-pattern-2025-01-15" | sha256sum | cut -d' ' -f1)
   ```

3. **Batch ingests** — upload multiple transcripts in one request to reduce overhead.

4. **Timestamps** — use workflow execution time, not current time. Helps memory system understand sequence.

5. **Project naming** — use consistent project keys (e.g., `temporal-workflows`, `agent-rust`, `poimen`).

---

## Ingesting from Temporal Directly

**Pseudo-code** (implement in your Temporal activity):

```go
func IngestWorkflowToMemory(ctx context.Context, execution WorkflowExecution) error {
    records := []Record{}
    
    // Walk through history and extract tool results
    for _, event := range execution.History.Events {
        if event.Type == "ActivityCompleted" {
            records = append(records, Record{
                Role:      "tool_result",
                Text:      event.Result,
                Timestamp: event.Timestamp,
            })
        }
    }
    
    // POST to /memory/ingest
    body := map[string]interface{}{
        "project": "temporal-workflows",
        "source": "temporal:workflow/" + execution.WorkflowID,
        "ingest_id": execution.RunID, // idempotent
        "records": records,
    }
    
    resp, err := http.Post("http://memory-service/memory/ingest", 
        "application/json", 
        jsonBody(body),
    )
    // ...
}
```

Then, in your workflow, query back:

```go
func QueryMemoryInWorkflow(ctx context.Context, q string) ([]Result, error) {
    resp, _ := http.Get(fmt.Sprintf(
        "http://memory-service/memory/query?project=temporal-workflows&query=%s",
        url.QueryEscape(q),
    ))
    // ...
}
```

---

## Troubleshooting

| Issue | Cause | Fix |
|---|---|---|
| `401 unauthorized` | Missing apikey header | Add `-H "apikey: YOUR_KEY"` |
| `202` then status `pending` forever | Ingest worker not running | Check `mem serve` is running with DB connection |
| Skills not appearing | M4.1 (extraction) not implemented yet | Use Format 2 (direct skill) for now |
| Rate limited (429) | Hit limit for project | Check rate limit, wait or use different apikey |
| Duplicate `ingest_id` | Same payload ingested twice | Intentional (idempotency); returns same job_id |

---

## Timeline

| Task | Status | Impact |
|---|---|---|
| M3.5.2 (ingest endpoint) | ✅ | Upload transcripts now |
| M3.5.5 (skills endpoint) | ✅ | Query skills now |
| M4.1 (skill extraction) | 🟡 | Automatic extraction in progress |
| M3.7.4 (context endpoint) | ⬜ | Tier-1 lookup not yet available |
| M3.7.8 (symptom projection) | ⬜ | Tier-2 vector lookup not yet available |

**Actionable now:** Formats 1 & 2, endpoints work. Extract by hand or via M4.1 when ready.