Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)
ci / markdown (push) Waiting to run
ci / markdown (push) Waiting to run
This commit is contained in:
Story Crater Bot
@@ -0,0 +1,178 @@
|
||||
# M3.5.9 — Git-aware memory references: lookup by code location
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | M3.5 — Distributed API Layer |
|
||||
| Size | M — 1–3 days |
|
||||
| Status | ⬜ Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
| Depends | M3.5.2 (git enrichment in ingest), M3.5.3 (query endpoint) |
|
||||
|
||||
## Goal
|
||||
|
||||
Enable agents to find and cite memory entries by code location (file:line, commit, author). Unifies memory log with git history. Agents reference: `"Per src/kong/buffer.rs:42 (commit abc123)..."` → lookup via git blame, return L0 evidence + L1 memory.
|
||||
|
||||
## Design
|
||||
|
||||
**New database columns** (extend `memory_node` from M2.3):
|
||||
```sql
|
||||
ALTER TABLE memory_node ADD COLUMN git_context JSONB;
|
||||
-- {file, line, commit_sha, commit_msg, author, author_date}
|
||||
|
||||
-- Index for git-based lookup
|
||||
CREATE INDEX ON memory_node USING GIN (git_context);
|
||||
```
|
||||
|
||||
**Three lookup modes:**
|
||||
|
||||
1. **By git location (file:line):**
|
||||
```
|
||||
POST /memory/nodes/by-git
|
||||
{
|
||||
"repo": "github.com/org/poimen",
|
||||
"file": "src/kong/buffer.rs",
|
||||
"line": 42,
|
||||
"project": "poimen"
|
||||
}
|
||||
→ 200 {
|
||||
"nodes": [
|
||||
{
|
||||
"sha256": "...",
|
||||
"level": "L0",
|
||||
"text": "Kong body buffer raised to 16MB...",
|
||||
"git_context": {commit_sha, commit_msg, author},
|
||||
"created_at": "2026-08-20T..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
2. **By commit (evidence from this commit):**
|
||||
```
|
||||
POST /memory/nodes/by-commit
|
||||
{
|
||||
"repo": "github.com/org/poimen",
|
||||
"commit_sha": "abc123def",
|
||||
"project": "poimen"
|
||||
}
|
||||
→ nodes from this commit + parent L1/L2 memories
|
||||
```
|
||||
|
||||
3. **By author (what did person X discover):**
|
||||
```
|
||||
POST /memory/nodes/by-author
|
||||
{
|
||||
"author": "[email protected]",
|
||||
"project": "poimen"
|
||||
}
|
||||
→ L0 nodes created during commits from alice
|
||||
```
|
||||
|
||||
**Query endpoint extension** (M3.5.3):
|
||||
Add optional `git_repo` param:
|
||||
```
|
||||
GET /memory/query?query=Kong&git_repo=github.com/org/poimen&project=poimen
|
||||
→ results enriched with git_context (file, commit, author)
|
||||
```
|
||||
|
||||
**Response format (all modes):**
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{
|
||||
"sha256": "abc...",
|
||||
"level": "L0|L1|L2",
|
||||
"brief": "Kong body buffer raised",
|
||||
"git_ref": "src/kong/buffer.rs:42",
|
||||
"git_commit": {
|
||||
"sha": "abc123def",
|
||||
"message": "Increase body buffer to 16MB",
|
||||
"author": "[email protected]",
|
||||
"date": "2026-08-15T10:30:00Z"
|
||||
},
|
||||
"parents": [...]
|
||||
}
|
||||
],
|
||||
"repo": "github.com/org/poimen"
|
||||
}
|
||||
```
|
||||
|
||||
## Steps
|
||||
|
||||
1. `POST /memory/nodes/by-git` handler:
|
||||
- Parse `file`, `line`, `project`
|
||||
- Query: `SELECT * FROM memory_node WHERE project = $1 AND git_context->>'file' = $2 AND (git_context->>'line')::int = $3`
|
||||
- Walk edges to include parent L1/L2 nodes
|
||||
- Sort by created_at desc
|
||||
|
||||
2. `POST /memory/nodes/by-commit` handler:
|
||||
- Parse `commit_sha`, `project`
|
||||
- Query: `SELECT * FROM memory_node WHERE project = $1 AND git_context->>'commit_sha' = $2`
|
||||
- Include all L0 from this commit + transitive parents (L1/L2)
|
||||
|
||||
3. `POST /memory/nodes/by-author` handler:
|
||||
- Parse `author`, `project`
|
||||
- Query: `SELECT * FROM memory_node WHERE project = $1 AND git_context->>'author' = $2 AND level = 'L0'`
|
||||
- Walk edges to L1 parents
|
||||
|
||||
4. Extend M3.5.3 query handler:
|
||||
- Add optional `git_repo` query param
|
||||
- If provided, enrich response with git_context from each result node
|
||||
- Include `git_ref` in brief (file:line) for agent citation
|
||||
|
||||
5. Deduplication by git:
|
||||
- L0 evidence from same (file, line, commit) = same memory entry
|
||||
- Idempotency: ingesting same commit twice doesn't duplicate L0 nodes
|
||||
- Check: `(file, line, commit_sha)` tuple uniqueness constraint
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `by-git` lookup returns correct L0 evidence + parent memories
|
||||
- `by-commit` returns all evidence from that commit
|
||||
- `by-author` returns all discoveries by that author
|
||||
- Query results enriched with git_context when repo provided
|
||||
- Same evidence never duplicated (idempotent by git tuple)
|
||||
- Agents can cite by code location: "src/kong/buffer.rs:42 (commit abc123)"
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** Integration tests with git history fixture.
|
||||
|
||||
**Setup:** Create test repo with commits:
|
||||
- commit abc123: modify src/kong/buffer.rs:42 (message: "Increase buffer")
|
||||
- commit def456: modify src/kong/handler.rs:10 (message: "Handle large bodies")
|
||||
- Create memory nodes with git_context from these commits
|
||||
|
||||
**Integration test** — `tests/it_git_references.rs`:
|
||||
1. `a1_by_git_lookup` — POST /nodes/by-git with file=buffer.rs, line=42 returns L0 from commit abc123.
|
||||
2. `a2_by_commit_lookup` — POST /nodes/by-commit with abc123 returns both L0 + parent L1/L2.
|
||||
3. `a3_by_author_lookup` — POST /nodes/by-author with alice@org returns all L0 from alice's commits.
|
||||
4. `a4_query_enriched_with_git` — GET /query?query=buffer&git_repo=... returns results with git_context populated.
|
||||
5. `a5_git_ref_in_brief` — result.git_ref = "src/kong/buffer.rs:42" (human-readable).
|
||||
6. `a6_idempotent_by_git_tuple` — ingest same commit twice, L0 nodes count stays 1 (no duplicates).
|
||||
7. `a7_edge_walk_preserves_git` — L1 parent of L0 node includes L0's git_context in parents array.
|
||||
8. `a8_cross_commit_correlation` — two commits affecting same file, both return from by-git lookup (line=0 or range?).
|
||||
9. `a9_author_query_filters_correctly` — two authors, by-author for alice returns only alice's L0.
|
||||
10. `a10_missing_git_context_graceful` — old L0 nodes without git_context (from before M3.5.2) still return but git_ref is null.
|
||||
|
||||
**Command:** `cargo test -p mem-cli git_references`
|
||||
|
||||
**False pass:**
|
||||
- Git context populated in fixture but never actually extracted from repo.git during ingest (M3.5.2). Test only checks stored data, not enrichment.
|
||||
- `by-git` returns results but never walks edges to L1. Parent L1 discoveries are invisible.
|
||||
- Query enrichment tested only with one repo. Multiple repos with overlapping filenames may return wrong results.
|
||||
- Idempotency tested with same commit but different git_repo URLs (github.com vs gitlab.com). Should be treated differently but test may not catch it.
|
||||
|
||||
## Traps
|
||||
|
||||
- Git blame is expensive. Caching blames by (file, commit_sha) pair is necessary for repeated queries.
|
||||
- Line numbers shift with edits. Reference to "line 42" in commit ABC may not match "line 42" in HEAD. Store commit hash, not line number, as primary key.
|
||||
- JSONB queries in PostgreSQL are slower than indexed columns. Consider denormalizing `git_file`, `git_commit`, `git_author` as separate columns if query volume is high.
|
||||
- Author name varies (alice@org vs alice.smith@org). Normalize email in ingest or handle fuzzy matching in by-author.
|
||||
- Cross-repo scenarios: same code in two repos (fork, mirror). git_repo must be part of uniqueness constraint.
|
||||
|
||||
---
|
||||
|
||||
Background: [DESIGN.md § Distributed API Layer](../DESIGN.md#distributed-api-layer-homelab-frontend)
|
||||
Reference in New Issue
Block a user