103 lines
4.0 KiB
Markdown
103 lines
4.0 KiB
Markdown
|
# M1.7 — `mem ingest` end to end
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | M1 — Gated loop at L1 |
|
|||
|
|
| Size | M — 1–3 days |
|
|||
|
|
| Status | ⬜ Not started |
|
|||
|
|
| Flags | — |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | M1.6 |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
One command that reads a real project and produces a real log — and reports the
|
|||
|
|
number that says whether the gate works.
|
|||
|
|
|
|||
|
|
## Facts (inlined — no spec read needed)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
mem ingest --project poimen --query infra-root-causes
|
|||
|
|
mem ingest --project poimen # all queries in the set
|
|||
|
|
mem ingest --project poimen --limit 50 # first 50 chunks, for iterating
|
|||
|
|
mem ingest --project poimen --resume # skip chunks already in the log
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Progress output, because a run that prints nothing cannot be distinguished from
|
|||
|
|
one that has hung:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
[ 17/412] t=17 update=yes mem=142tok 1.9s
|
|||
|
|
[ 18/412] t=18 update=no mem=142tok 0.8s
|
|||
|
|
...
|
|||
|
|
run 01HXYZ chunks 412 used 17 update-rate 4.1% memory 142tok elapsed 6m12s
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Update-rate is the headline number.** Tool results are ~43% of records and
|
|||
|
|
mostly evidence-free; a correct gate rejects the large majority of chunks. A rate
|
|||
|
|
above ~30% means the gate is not discriminating and the run is an expensive
|
|||
|
|
summarizer — that is the paper's memory-explosion failure and it is what M1.8
|
|||
|
|
gates on.
|
|||
|
|
|
|||
|
|
Runs are long. 412 chunks at ~1–2s each is 6–14 minutes per query, and every
|
|||
|
|
chunk costs a model call, so `--resume` is not a nicety.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. Wire adapters (M0.5/M0.6) → chunker (M0.3) → loop (M1.5) → log (M1.6).
|
|||
|
|
2. Per-chunk progress line to stderr; summary to stdout so it pipes cleanly.
|
|||
|
|
3. Report update-rate in the summary and as `--format json`.
|
|||
|
|
4. `--resume` reads the existing log, finds the highest `t` with a `gate` record,
|
|||
|
|
and restarts from `t+1` with that turn's memory.
|
|||
|
|
5. `--limit` caps chunks processed.
|
|||
|
|
6. Exit non-zero if the run did not reach `run_end`.
|
|||
|
|
7. Ctrl-C finishes the in-flight turn, writes `run_end`, exits — no half-turn.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
- A real project produces a complete log with `run_end`.
|
|||
|
|
- Reported update-rate equals the value computed independently from the log.
|
|||
|
|
- `--resume` on a complete log is a no-op; on a partial one it continues.
|
|||
|
|
- Interrupt produces a valid log.
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
**Harness:** scripted client for determinism, plus one live `#[ignore]` run.
|
|||
|
|
|
|||
|
|
**Integration test** — `tests/it_ingest.rs`:
|
|||
|
|
1. `a1_produces_complete_log` — scripted run; assert `run_end` present and event
|
|||
|
|
counts match the script.
|
|||
|
|
2. `a2_update_rate_matches_log` — compare the reported rate to
|
|||
|
|
`LogWriter::stats()` recomputed from the file.
|
|||
|
|
3. `a3_resume_is_noop_when_complete` — run, resume, assert zero additional model
|
|||
|
|
calls.
|
|||
|
|
4. `a4_resume_continues_partial` — truncate a log after t=10, resume, assert the
|
|||
|
|
next call is t=11 and memory at t=11 equals the replayed memory at t=10.
|
|||
|
|
5. `a5_interrupt_is_clean` — send SIGINT mid-run; assert the log parses, has
|
|||
|
|
`run_end`, and the last `gate` has a matching `memory`-or-not decision.
|
|||
|
|
6. `a6_limit_respected` — `--limit 5` produces exactly 5 gate records.
|
|||
|
|
7. `a7_live_smoke` — `#[ignore]`; real gateway, `--limit 20` on a real project;
|
|||
|
|
assert `run_end` and **print** the update-rate for a human to read.
|
|||
|
|
|
|||
|
|
**Command:** `cargo test -p mem-cli ingest` (add `-- --ignored` for a7)
|
|||
|
|
|
|||
|
|
**False pass:**
|
|||
|
|
- Asserting only that the command exits 0. A run whose gate always answers `no`
|
|||
|
|
exits 0, writes a valid log, and has learned nothing — the update-rate is the
|
|||
|
|
only thing that distinguishes it, which is why a7 prints it rather than
|
|||
|
|
merely asserting a run happened.
|
|||
|
|
- Resuming by counting lines rather than reading the highest `t` with a `gate`
|
|||
|
|
record. Line counts break the moment an `evidence` record is present, i.e. as
|
|||
|
|
soon as the gate ever opened.
|
|||
|
|
|
|||
|
|
## Traps
|
|||
|
|
|
|||
|
|
- No progress output. A 14-minute run that prints nothing is indistinguishable
|
|||
|
|
from a hang, and the first instinct will be to kill it.
|
|||
|
|
- Resume that replays from `t=1` with the old memory. It costs a full run and
|
|||
|
|
produces a log with duplicate turns that `mem verify` will reject.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Background: [DESIGN.md](../DESIGN.md) — Verification, P2
|