rock/poimen-memory
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
631cbfa3e95933dee62fbd2363d46f1c79aa8203
poimen-memory/tasks/INDEX.md
T
Story Crater Bot 631cbfa3e9 feat: complete M0.1-M0.4 phases 
M0.1 - Cargo workspace + crate skeletons
  - 6-crate workspace with correct dependency direction
  - CI/CD pipeline with GitHub Actions
  - Integration tests verifying build and dependency structure

M0.2 - Domain types and sha256 identity
  - Level (L0, L1, L2) enum with proper serde formatting
  - Role enum (User, Assistant, ToolResult, System)
  - Record, Chunk, and MemoryNode domain types
  - Content-hash identity system ensuring rebuild idempotence
  - Newtypes (ProjectId, QueryId, RunId) with validation
  - Round-trip serde tests for all types

M0.3 - RecordSource trait + ChunkPolicy
  - RecordSource trait for streaming record sources
  - Chunk policy with token budgets and boundary modes
  - TokenCounter trait with CharsOverFourCounter stub
  - Chunking stream that respects budgets without splitting records
  - VecSource for testing
  - Integration tests verifying lossless chunking and budget adherence

M0.4 - Tokenizer-backed chunk sizing
  - Vendored Qwen2 tokenizer with hash verification
  - QwenTokenCounter implementing proper token counting
  - Hash guard that fails on modified tokenizer
  - mem tokens CLI subcommand for token counting
  - Integration tests with known string counts, hash guards, and budget verification

Total: 19 integration tests passing, all phases verified to compose correctly
Workspace builds cleanly with no clippy warnings
2026-08-22 23:13:42 -07:00

193 lines
10 KiB
Markdown
Raw Blame History

# poimen-memory — task board
43 tasks — 36 build tasks plus **7 composition gates**, one per phase. One file
per task, **self-contained**: inlined design facts, executable steps, acceptance
criteria, a `Verify` section written for someone who did not build the thing, and
the traps worth naming. Reading `DESIGN.md` is not required to do a task — it is
linked as background only.
Each `Verify` section names the harness, the integration test with numbered
assertions, the command to run, and the **false pass** — the shape of test that
goes green while the feature is broken. Treat the false-pass list as part of the
acceptance criteria, not commentary.
## Ordering — declared, never derived
**Phase order is the list below. Task ids are opaque and frozen.**
`M0.3` is `M0.3` forever, in whatever phase it currently sits, because its
artifacts and cross-references key on that id. New tasks take new ids rather than
renumbering neighbours. This is the `StepId` rule applied to the board itself —
a board that renumbers to reorder has the bug it warns its own users about.
No phase starts until its predecessor's gate is green. The `gate` task at the end
of each phase **is** that gate: it proves the phase's parts compose and that its
swappable parts are genuinely swappable. Every build task is verified alone; the
gate verifies the properties no single task owns.
## Two rules this board exists to protect
**1. The JSONL log is authoritative; the vault and the vector index are
projections.** Anything that cannot be dropped and rebuilt byte-identically from
the log has hidden inputs, and that is a bug. `M2.8` is the gate that enforces it.
**2. The update gate must discriminate, not summarize.** Agent transcripts are
~43% tool results and mostly evidence-free. A gate that accepts most chunks is an
expensive summarizer that will reproduce the memory-explosion failure the whole
design exists to avoid. `M1.8` is the gate that enforces it, and **update-rate is
the single number to watch.**
## Verification practice — script first, source second
A task is verified by running its command and reading the output, then opening
the source. Reviewing the diff first is how an assertion that was quietly dropped
still gets called done: the code looks right, and nothing proves the test ran.
Numbered assertion N in a `Verify` section is test fn `aN_<slug>`. The numbering
is the contract — a test fn that does not exist should report missing rather than
be silently absent from a green summary.
`cargo test` reporting `ok` with zero tests run is not a pass.
## Progress
**Source of truth is the `Status` field in each task file.** The tables below
mirror it; a status changed here and not there is a lie.
Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
| # | Phase | Ids | Tasks | ✅ | 🟡 | ⬜ | Gate |
|---|---|---|---|---|---|---|---|
| 1 | Read-only spine | M0.x | 8 | 0 | 0 | 8 | ⬜ M0.8 |
| 2 | Gated loop at L1 | M1.x | 8 | 0 | 0 | 8 | ⬜ M1.8 |
| 3 | Projections | M2.x | 8 | 0 | 0 | 8 | ⬜ M2.8 |
| 4 | L2 synthesis + retrieval | M3.x | 4 | 0 | 0 | 4 | ⬜ M3.4 |
| 4.5 | Distributed API Layer | M3.5.x | 8 | 0 | 0 | 8 | ⬜ M3.5.8 |
| 5 | Skills | M4.x | 3 | 0 | 0 | 3 | ⬜ M4.3 |
| 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 |
| 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
| | **Total** | | **51** | **0** | **0** | **51** | 0/8 green |
**Where the line is — 2026-08-20.** Nothing started. No crate exists yet: there
is no `Cargo.toml` under `memory/`, so every task below is design only. M0.1 is
the first thing that has to happen. `M2.2` (the CNPG manifest), `M5.4` (vLLM
with LoRA), `M3.5.x` (API layer), and all of `M6.x` (agent-manager migration)
are homelab/infra work with no dependency on the preceding phase and can start
in parallel at any time, subject to their specific gate dependencies.
**M6 is a different repo, not a dependency of M0-M5.** It migrates
`github.com/Riotpiaole/agent-manager`'s session store (a separate Go CLI tool,
unrelated to this project's own memory system) from local sqlite to its own
dedicated CNPG cluster. It rides in this board because it's homelab work
happening alongside M2.2/M5.4, and because the two projects' Postgres schemas
landing in the same cluster around the same time need to look like siblings,
not strangers — see M6.6's convention-consistency check.
## 1 — Read-only spine · M0.x
No model calls anywhere in this phase. The point is to prove the corpus parses
and chunks sanely before spending inference on it.
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M0.1](M0.1-cargo-workspace.md) | Cargo workspace + crate skeletons | S | — | ⬜ |
| [M0.2](M0.2-domain-types.md) | Domain types and sha256 identity | S | — | ⬜ |
| [M0.3](M0.3-recordsource-and-chunkpolicy.md) | `RecordSource` trait + `ChunkPolicy` | M | — | ⬜ |
| [M0.4](M0.4-tokenizer-sizing.md) | Tokenizer-backed chunk sizing | M | — | ⬜ |
| [M0.5](M0.5-pi-session-adapter.md) | pi session adapter | M | — | ⬜ |
| [M0.6](M0.6-claude-transcript-adapter.md) | Claude transcript adapter | S | — | ⬜ |
| [M0.7](M0.7-ingest-dry-run.md) | `mem ingest --dry-run` | S | — | ⬜ |
| [M0.8](M0.8-m0-gate.md) | **M0 composition gate** | M | gate | ⬜ |
## 2 — Gated loop at L1 · M1.x
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M1.1](M1.1-llm-chat-client.md) | `mem-llm` chat client | M | — | ⬜ |
| [M1.2](M1.2-standing-query-loader.md) | Standing-query YAML loader | M | — | ⬜ |
| [M1.3](M1.3-prompt-template.md) | GRU-Mem prompt template | M | — | ⬜ |
| [M1.4](M1.4-gate-response-parser.md) | Gate-response parser | M | — | ⬜ |
| [M1.5](M1.5-gated-loop.md) | The gated loop | L | — | ⬜ |
| [M1.6](M1.6-jsonl-event-log.md) | JSONL event log writer | M | — | ⬜ |
| [M1.7](M1.7-ingest-end-to-end.md) | `mem ingest` end to end | M | — | ⬜ |
| [M1.8](M1.8-m1-gate.md) | **M1 composition gate** | M | gate | ⬜ |
## 3 — Projections · M2.x
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M2.1](M2.1-embeddings-client.md) | Embeddings client | S | — | ⬜ |
| [M2.2](M2.2-memory-db-manifest.md) | CNPG `memory-db` + pgvector | M | homelab | ⬜ |
| [M2.3](M2.3-schema-and-migrations.md) | Schema + sqlx migrations | M | — | ⬜ |
| [M2.4](M2.4-pgvector-repo.md) | pgvector repository | M | — | ⬜ |
| [M2.5](M2.5-obsidian-projector.md) | Obsidian projector | M | — | ⬜ |
| [M2.6](M2.6-rebuild-from-log.md) | `mem rebuild --from-log` | M | — | ⬜ |
| [M2.7](M2.7-verify-edges.md) | `mem verify` — edge closure | S | — | ⬜ |
| [M2.8](M2.8-m2-gate.md) | **M2 composition gate** | M | gate | ⬜ |
## 4 — L2 synthesis and retrieval · M3.x
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M3.1](M3.1-l2-synthesis.md) | L2 synthesis pass | M | — | ⬜ |
| [M3.2](M3.2-rerank-client.md) | Rerank client | S | — | ⬜ |
| [M3.3](M3.3-mem-query.md) | `mem query` with provenance | M | — | ⬜ |
| [M3.4](M3.4-m3-gate.md) | **M3 composition gate** | M | gate | ⬜ |
## 4.5 — Distributed API Layer · M3.5.x
Homelab frontend integration: HTTP facade via `api.riotpiao.com`. Runs in parallel with M4 and M5 after M3.4 green.
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M3.5.1](M3.5.1-http-server.md) | HTTP server + router, Kong auth, metrics | M | — | ⬜ |
| [M3.5.2](M3.5.2-ingest-endpoint.md) | POST /ingest async queue, idempotency | M | — | ⬜ |
| [M3.5.3](M3.5.3-query-endpoint.md) | GET /query HNSW + rerank + edge-walk | M | — | ⬜ |
| [M3.5.4](M3.5.4-query-federation.md) | Query federation across projects | M | — | ⬜ |
| [M3.5.5](M3.5.5-skills-endpoint.md) | GET /skills and /skills/{name} | M | — | ⬜ |
| [M3.5.6](M3.5.6-projects-endpoint.md) | GET /projects and /projects/{id}/status | S | — | ⬜ |
| [M3.5.7](M3.5.7-rate-limiting.md) | Rate limiting + idempotency by sha256 | M | — | ⬜ |
| [M3.5.8](M3.5.8-m3.5-gate.md) | **M3.5 composition gate** | M | gate | ⬜ |
## 5 — Skills · M4.x
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M4.1](M4.1-skill-draft.md) | `mem skill draft` | M | — | ⬜ |
| [M4.2](M4.2-derived-filter.md) | `derived: true` ingest filter | M | — | ⬜ |
| [M4.3](M4.3-m4-gate.md) | **M4 composition gate** | M | gate | ⬜ |
## 6 — Post-training · M5.x
Python, separate from the Rust workspace. The boundary is the JSONL log.
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M5.1](M5.1-evidence-labeler.md) | `mem label` — evidence labeler | M | — | ⬜ |
| [M5.2](M5.2-labeler-calibration.md) | Labeler calibration | M | — | ⬜ |
| [M5.3](M5.3-training-corpus-export.md) | Training corpus export | M | — | ⬜ |
| [M5.4](M5.4-vllm-lora-serving.md) | vLLM + `--enable-lora` | L | homelab | ⬜ |
| [M5.5](M5.5-verl-training-loop.md) | verl training loop | L | — | ⬜ |
| [M5.6](M5.6-m5-gate.md) | **M5 composition gate** | L | gate | ⬜ |
## 7 — agent-manager migration · M6.x
Separate repo (`github.com/Riotpiaole/agent-manager`, fork branch
`add-headless-spawn`), separate cluster resource, no Rust/GRU-Mem
dependency. Moves its session store off local sqlite onto a dedicated CNPG
Postgres, reachable from the Mac client through a dedicated nginx route —
durability-of-location, not a multi-host requirement.
| Task | Title | Size | Flags | Status |
|---|---|---|---|---|
| [M6.1](M6.1-agent-manager-db-manifest.md) | CNPG `agent-manager-db` manifest | M | homelab | ⬜ |
| [M6.2](M6.2-schema-port.md) | Postgres schema for agent-manager sessions | M | — | ⬜ |
| [M6.3](M6.3-store-query-port.md) | store.go query port to Postgres | L | — | ⬜ |
| [M6.4](M6.4-nginx-stream-routing.md) | nginx TCP routing to `agent-manager-db` | S | homelab | ⬜ |
| [M6.5](M6.5-credentials-secret.md) | Postgres credentials for the Mac client | S | homelab | ⬜ |
| [M6.6](M6.6-m6-gate.md) | **M6 composition gate** | M | gate | ⬜ |
---
Background: [DESIGN.md](../DESIGN.md) · GRU-Mem, arXiv 2602.10560 · `internal/store/store.go` (agent-manager, `add-headless-spawn` branch)
Reference in New Issue View Git Blame Copy Permalink