Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

tasks/M2.3-schema-and-migrations.md

@@ -19,15 +19,14 @@ malformed graph impossible rather than merely unlikely.
 ```sql
 CREATE TABLE memory_node (
   id        BIGSERIAL PRIMARY KEY,
-  level     TEXT NOT NULL CHECK (level IN ('L0','L1','L2')),
+  level     TEXT NOT NULL CHECK (level IN ('L0','L1','L2','R')),
   project   TEXT NOT NULL,
-  query_id  TEXT,                       -- NULL at L2
+  query_id  TEXT,                       -- NULL at L2 and R
   run_id    TEXT NOT NULL,
   t         INT  NOT NULL,
-  source    TEXT,                       -- set at L0
+  source    TEXT,                       -- set at L0; source URI at R
   text      TEXT NOT NULL,
   sha256    TEXT NOT NULL UNIQUE,       -- content identity, from M0.2
-  embedding vector(768) NOT NULL,
   created_at TIMESTAMPTZ NOT NULL DEFAULT now()
 );
 CREATE TABLE memory_edge (
@@ -35,10 +34,53 @@ CREATE TABLE memory_edge (
   parent_sha TEXT NOT NULL REFERENCES memory_node(sha256) ON DELETE CASCADE,
   PRIMARY KEY (child_sha, parent_sha)
 );
-CREATE INDEX ON memory_node USING hnsw (embedding vector_cosine_ops);
 CREATE INDEX ON memory_node (project, level);
+
+-- Vectors live outside the node: one node carries several, and a symptom
+-- projection (M3.7.8) is what makes an answer findable from an error message.
+CREATE TABLE memory_vector (
+  node_sha  TEXT NOT NULL REFERENCES memory_node(sha256) ON DELETE CASCADE,
+  kind      TEXT NOT NULL CHECK (kind IN ('text','symptom')),
+  embedding vector(768) NOT NULL,
+  PRIMARY KEY (node_sha, kind)
+);
+-- Partial index per kind. One index over mixed kinds forces post-filtering,
+-- which starves recall exactly the way M3.6.5 describes.
+CREATE INDEX ON memory_vector USING hnsw (embedding vector_cosine_ops) WHERE kind = 'text';
+CREATE INDEX ON memory_vector USING hnsw (embedding vector_cosine_ops) WHERE kind = 'symptom';
+
+-- Exact-match tier. Failures repeat verbatim; prose does not.
+CREATE TABLE failure_signature (
+  sig_sha    TEXT PRIMARY KEY,          -- hash of the NORMALISED signature (M3.7.7)
+  node_sha   TEXT NOT NULL REFERENCES memory_node(sha256) ON DELETE CASCADE,
+  tool       TEXT NOT NULL,             -- 'github-actions' | 'kubectl' | 'npm'
+  raw        TEXT NOT NULL,             -- pre-normalisation, for display
+  seen_count INT  NOT NULL DEFAULT 1,
+  last_seen  TIMESTAMPTZ NOT NULL
+);
+CREATE INDEX ON failure_signature (tool);
+
+-- A lesson about Kong is actively harmful now that Kong is retired.
+CREATE TABLE memory_supersede (
+  old_sha TEXT NOT NULL REFERENCES memory_node(sha256) ON DELETE CASCADE,
+  new_sha TEXT NOT NULL REFERENCES memory_node(sha256) ON DELETE CASCADE,
+  reason  TEXT,
+  PRIMARY KEY (old_sha, new_sha)
+);
 ```
 
+**`embedding` is deliberately not a column on `memory_node`.** A node needs more
+than one vector: the memory text as written, and a generated *symptom* projection
+describing the errors it would explain. An L1 reads like an answer and a query
+reads like a stack trace, and cosine between those two registers is mediocre —
+the second vector is what closes that gap. One column cannot hold both, and
+bolting on `embedding_2` later is worse than a junction table now.
+
+**`seen_count` and `last_seen` are mutable and that does not break the authority
+rule.** Occurrences are append-only records in the JSONL log; these two fields are
+a fold over them, recomputed by `mem rebuild --from-log` like every other
+projected value.
+
 `sha256 UNIQUE` is what makes rebuild idempotent — re-inserting identical content
 is a conflict to ignore, not a duplicate row. It is also why the hash must exclude
 run ids and timestamps (M0.2).
@@ -47,8 +89,13 @@ run ids and timestamps (M0.2).
 dangling edges. Rebuild drops everything anyway, but a partial cleanup should not
 be able to corrupt the graph.
 
-`query_id` is NULL at L2 by design — L2 spans queries. Enforce it:
-`CHECK ((level = 'L2') = (query_id IS NULL))`.
+`query_id` is NULL at L2 **and at R** — L2 spans queries, R answers none. Enforce
+it: `CHECK ((level IN ('L2','R')) = (query_id IS NULL))`.
+
+The `'R'` level ships here rather than arriving as an `ALTER` from M3.6. Nothing
+is built yet, so widening a constraint that has never existed wrong is free, and
+a migration that exists only because an earlier migration was knowingly
+incomplete is debt taken on for no reason.
 
 Cosine distance, not L2: these are normalised text embeddings and cosine is what
 the model was trained for. `vector_cosine_ops` must match the operator the query
@@ -69,8 +116,9 @@ uses (`<=>`), or the index is silently ignored and every query is a seq scan.
 
 - Migrations apply to a clean database and are idempotent.
 - Inserting a duplicate `sha256` conflicts rather than duplicating.
-- An `L2` row with a non-null `query_id` is rejected by the CHECK.
-- The HNSW index is used by a cosine-distance query.
+- An `L2` or `R` row with a non-null `query_id` is rejected by the CHECK.
+- Both partial HNSW indexes are used by a kind-filtered cosine query.
+- A node can carry a `text` and a `symptom` vector simultaneously.
 
 ## Verify
 
@@ -81,12 +129,21 @@ image tag as production, `ghcr.io/cloudnative-pg/postgresql:16.2`.
 1. `a1_migrate_clean` — apply to an empty database, assert both tables exist.
 2. `a2_migrate_idempotent` — apply twice, assert no error.
 3. `a3_sha_unique` — insert the same sha twice, assert a unique violation.
-4. `a4_level_check` — `level='L3'` rejected; `level='L2'` with a `query_id`
-   rejected; `level='L1'` without one rejected.
+4. `a4_level_check` — `level='L3'` rejected; `level='L2'` and `level='R'` with a
+   `query_id` rejected; `level='L1'` without one rejected.
 5. `a5_edge_fk` — an edge referencing a missing sha is rejected.
-6. `a6_cascade` — delete a node, assert its edges are gone.
-7. `a7_hnsw_is_used` — `EXPLAIN` a `ORDER BY embedding <=> $1 LIMIT 10` query and
-   assert the plan contains `Index Scan` on the HNSW index, not `Seq Scan`.
+6. `a6_cascade` — delete a node, assert its edges, vectors and signatures are
+   gone.
+7. `a7_hnsw_is_used` — `EXPLAIN` a `WHERE kind='text' ORDER BY embedding <=> $1
+   LIMIT 10` query; assert the plan contains an `Index Scan` on the **partial**
+   index, not `Seq Scan` and not a filter applied above a full-index scan.
+8. `a8_symptom_index_separate` — same for `kind='symptom'`; assert the plan names
+   the other index, proving both were created and are distinguishable.
+9. `a9_two_vectors_per_node` — insert both kinds for one node; assert both
+   persist and the composite primary key rejects a third of the same kind.
+10. `a10_signature_unique` — inserting the same `sig_sha` twice conflicts.
+11. `a11_supersede_pair` — a supersede row survives, and deleting either endpoint
+    cascades it away.
 
 **Command:** `cargo test -p mem-store schema`