(plan) system review and break down plans

2026-08-19 09:52:07 -07:00
commit 9d28b63ff2
46 changed files with 5620 additions and 0 deletions
@@ -0,0 +1,92 @@
+# M0.6 — Claude transcript adapter
+
+| Field | Value |
+|---|---|
+| Phase | M0 — Read-only spine |
+| Size | S — under 1 day |
+| Status | ⬜ Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | M0.5 |
+
+## Goal
+
+The second `RecordSource`, which is the one that proves the trait is real.
+
+## Facts (inlined — no spec read needed)
+
+```
+~/.claude/projects/-Users-rockliang-workplace-Poimen/<uuid>.jsonl
+                   └─ cwd, / replaced by -, no wrapping dashes (differs from pi)
+```
+
+15 MB across 7 transcripts on this machine. Record types observed:
+
+```
+attachment 150   queue-operation 138   assistant 134   user 81
+file-history-snapshot 69   system 69   ai-title 21   last-prompt 21   mode 21
+```
+
+Fields, read from any line: `sessionId`, `cwd`, `gitBranch`.
+Typed records dispatch on `type`; the ones that matter here:
+
+- `user` / `assistant` — content under `message.content`
+- `system` with `subtype == "api_error"` — a failed turn, worth keeping
+- `summary` — carries `summary`
+- everything else — ignore
+
+The filename stem is a UUID. The encoding differs from pi (no wrapping `--`),
+which is exactly why the project key must come from the `cwd` **field**, as in
+M0.5, and not from the directory name.
+
+## Steps
+
+1. Implement `ClaudeTranscriptSource` in `mem-ingest`, implementing `RecordSource`.
+2. Take `cwd` from the first line carrying it; error if no line does.
+3. Emit records for `user`, `assistant`, and `system`+`api_error`.
+4. Flatten `message.content` across its shapes, as in M0.5.
+5. `Provenance` = `claude:<uuid>` plus line offset.
+6. Reuse the content-flattening and malformed-line handling from M0.5 — extract
+   them into a shared helper rather than copying, since divergence between two
+   flatteners is a bug that only shows up on one source.
+
+## Acceptance
+
+- Both sources satisfy `RecordSource` with no changes to `mem-chunk`.
+- Project keys from pi and claude for the same directory resolve to the same
+  `ProjectId`.
+- `api_error` system records survive into the stream.
+
+## Verify
+
+**Harness:** a committed transcript fixture plus a real one, secrets scrubbed.
+
+**Integration test** — `tests/it_claude_source.rs`:
+1. `a1_project_from_cwd_field` — assert the key comes from `cwd`, not the
+   directory name, using a fixture where they would differ.
+2. `a2_same_project_across_sources` — a pi session and a claude transcript for
+   the same directory yield an equal `ProjectId`. This is the assertion that
+   makes cross-source memory possible at all.
+3. `a3_role_mapping` — user/assistant/api_error appear; `attachment`,
+   `queue-operation`, `file-history-snapshot` do not.
+4. `a4_shared_flattener` — the same content-block fixture flattens identically
+   through both sources (call both, compare strings).
+
+**Command:** `cargo test -p mem-ingest claude_source`
+
+**False pass:**
+- Testing each source in isolation. The point of this task is that they agree;
+  assertions 2 and 4 are the only ones that check it, and both are cross-source.
+
+## Traps
+
+- Copying the flattener instead of sharing it. The two will drift, and the
+  resulting bug looks like "claude transcripts lose tool output" rather than
+  "there are two flatteners".
+- Assuming the pi encoding. Claude has no wrapping `--`, so a decoder written
+  against pi silently produces a different project key for the same directory —
+  and cross-source memory quietly splits in two.
+
+---
+
+Background: [DESIGN.md](../DESIGN.md) — Context