121 lines
4.8 KiB
Markdown
121 lines
4.8 KiB
Markdown
# M1.3 — GRU-Mem prompt template
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | M1 — Gated loop at L1 |
|
||
| Size | M — 1–3 days |
|
||
| Status | ⬜ Not started |
|
||
| Flags | — |
|
||
| Spec | inlined below |
|
||
| Blocks | M1.2 |
|
||
|
||
## Goal
|
||
|
||
Assemble the memory-agent prompt exactly as the paper specifies, because the
|
||
model's ability to emit parseable gates depends on the format it was aligned to.
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
Paper Figure 10a, reproduced verbatim — this is the contract, not a starting
|
||
point to improvise on:
|
||
|
||
```
|
||
You are presented with a problem, a section of an article that may contain the
|
||
answer to the problem, and a previous memory. Please read the provided section
|
||
carefully. You should reason about whether the new section contains useful
|
||
information about the problem, and then update the memory with the new
|
||
information that helps to answer the problem.
|
||
Be sure to retain all relevant details from the previous memory while adding any
|
||
new, useful information. You should also carefully judge whether you have
|
||
collected enough information to answer the problem.
|
||
You should reason about whether the new section contains useful information, what
|
||
to update, and what to do next first between <think> and </think>.
|
||
If the new section contains useful information about the problem, you should
|
||
first generate <check>yes</check>. After that, update the new memory between
|
||
<update> and </update>.
|
||
If the new section does not contain useful information about the problem, you
|
||
should first generate <check>no</check>. After that, you should keep the previous
|
||
memory unchanged between <update> and </update>.
|
||
In the end, if you haven't collected enough information for the problem, return
|
||
<next>continue</next>. ONLY when enough information is collected, return
|
||
<next>end</next>.
|
||
<problem> {prompt} </problem>
|
||
<memory> {memory} </memory>
|
||
<section> {chunk} </section>
|
||
```
|
||
|
||
Substitutions for this system: `{prompt}` = the standing question, `{memory}` =
|
||
`M_{t-1}` or the literal `No previous memory` at `t=1` (the paper's own case
|
||
studies show that exact string), `{chunk}` = the rendered chunk.
|
||
|
||
Budget, against the 32768 cap:
|
||
|
||
```
|
||
system + template ~400
|
||
question ~100
|
||
memory <=1024
|
||
chunk <=5000
|
||
response 2048
|
||
------
|
||
~8600 headroom is comfortable
|
||
```
|
||
|
||
Chunk rendering: each record as `[role] text`, records separated by a blank line.
|
||
Role labels matter — the model uses them to tell a tool result from a decision.
|
||
|
||
## Steps
|
||
|
||
1. `PromptBuilder` in `mem-core` producing `(system, user)`.
|
||
2. Template verbatim as above. Any deviation gets a comment saying why.
|
||
3. `t=1` renders `No previous memory` — not empty, not `null`.
|
||
4. Render chunk records as `[role] text`, blank-line separated.
|
||
5. Assert the assembled prompt fits the budget before sending; over budget is an
|
||
error naming the component that overflowed, not a truncation.
|
||
6. `mem prompt --project P --query Q --chunk N` prints the exact prompt, for
|
||
eyeballing what the model actually sees.
|
||
|
||
## Acceptance
|
||
|
||
- Assembled prompt matches a committed golden file byte for byte.
|
||
- `t=1` contains `No previous memory`.
|
||
- Over-budget assembly errors and names the offending component.
|
||
|
||
## Verify
|
||
|
||
**Harness:** golden-file comparison. The prompt is a contract; a diff in it is a
|
||
change to the contract.
|
||
|
||
**Integration test** — `tests/it_prompt.rs`:
|
||
1. `a1_golden_t1` — first turn against `expected/prompt-t1.txt`, exact match.
|
||
2. `a2_golden_tn` — turn with a prior memory against `expected/prompt-tn.txt`.
|
||
3. `a3_no_previous_memory_literal` — assert the exact string at `t=1`.
|
||
4. `a4_all_tags_present` — `<problem>`, `<memory>`, `<section>` each appear
|
||
exactly once.
|
||
5. `a5_role_labels_rendered` — a chunk with all four roles renders all four
|
||
labels.
|
||
6. `a6_over_budget_errors` — a 20000-token chunk errors, message contains
|
||
`section`.
|
||
7. `a7_budget_headroom` — for the real fixture corpus, assert every assembled
|
||
prompt is under 32768 minus 2048.
|
||
|
||
**Command:** `cargo test -p mem-core prompt`
|
||
|
||
**False pass:**
|
||
- Asserting the prompt "contains" the question. A template that dropped the
|
||
`<check>` instructions still contains it, and the model then emits prose the
|
||
parser cannot read. Golden-file equality is the assertion that holds.
|
||
- Skipping assertion 7 by testing only small fixtures. Budget overflow appears
|
||
at p95 chunk size, not at the median.
|
||
|
||
## Traps
|
||
|
||
- Improving the wording. The 3B model's gate reliability comes from this exact
|
||
format; a cleaner rewrite is an unmeasured change to the one thing M1.8 gates on.
|
||
- Rendering an empty `<memory></memory>` at `t=1`. The paper's traces show
|
||
`No previous memory`, and an empty tag reads to the model as "memory exists and
|
||
is empty", which is a different claim.
|
||
|
||
---
|
||
|
||
Background: [DESIGN.md](../DESIGN.md) — Standing queries · paper Fig 10a
|