(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,103 @@
+# T7.1 — Postgres `EventLog`
+
+| Field | Value |
+|---|---|
+| Phase | P7 — Distribution |
+| Size | L — over 3 days |
+| Status | Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | — |
+
+## Goal
+
+The same `EventLog` port over Postgres, partitioned by tenant and time. Not a
+second port — the same one.
+
+## Facts (inlined — no spec read needed)
+
+- Two deployment modes, **one set of ports**:
+
+| Mode | Log + state | Blobs | Coordination |
+|---|---|---|---|
+| **Embedded** — single binary, no services | `redb` | `redb` table | in-process |
+| **Distributed** — multi-node, multi-tenant | Postgres | S3-compatible | Postgres advisory locks, or Redis if leases dominate |
+
+- Postgres transactions supply crash-atomicity exactly as `redb`'s shadow paging
+  does; `commit`'s four writes stay in one transaction.
+- This is why the port was async from the first line: a sync signature over a
+  network call is impossible, and the local implementation paid only a negligible
+  poll for the compatibility.
+- **Every port test runs against both backends or the abstraction is not real.**
+  The conformance suite from T0.5/T0.7 is the deliverable being reused here.
+- Open at this scale: partitioning by tenant and time, index strategy for branch
+  scans, and whether the outbox is a table or a logical replication slot. Decide
+  and record the decision; it is not pre-settled.
+
+## Steps
+
+1. Schema: log table keyed `(tenant, run, branch, lsn)`; state, consumer position
+   and outbox tables mirroring the embedded key shapes.
+2. Partition the log by tenant and by time. Choose declarative partitioning and
+   document the retention interaction with T8.4's tiering.
+3. Index for the access patterns that exist: branch scans from an LSN, recent
+   runs by `RunId` prefix (ULIDs sort by creation time), outbox drain per
+   `BranchKey` ascending `Lsn`.
+4. Implement `commit` as one transaction with all four writes. LSN allocation
+   stays inside the transaction — a sequence is not sufficient, since LSNs are
+   per branch and must be gap-free.
+5. Implement the checkpoint and outbox methods against the same schema.
+6. Run the **unmodified** P0 conformance suite against this backend in CI.
+7. Record the outbox decision (table vs replication slot) with its reasoning.
+
+## Acceptance
+
+- The P0 conformance suite passes **unmodified** against Postgres.
+- Every port test runs against both backends.
+
+## Verify
+
+**Harness:** the **same** conformance function from T0.5, instantiated with a
+Postgres factory. A real Postgres in CI (testcontainers or a service container) —
+not an in-memory fake, which would test nothing about the backend.
+
+**Integration test** — `tests/it_postgres_conformance.rs`:
+1. Call `conformance(|| PostgresEventLog::new(...))` with **zero modifications**
+   to the suite. Any needed change is a finding about the port, not the test.
+2. Run the identical suite against `redb` in the same CI job, so a divergence
+   between backends fails immediately rather than months later.
+3. **Concurrent LSN allocation** across separate **connections** — the redb test
+   used tasks; here contention is across processes and is the real case. Assert
+   `1..=n` with no gaps or repeats.
+4. Assert `commit`'s four writes are in one transaction: kill the connection
+   mid-commit and assert all-or-nothing on reconnect.
+5. Partitioning: insert across several tenants and time ranges; assert a branch
+   scan hits the expected partitions (check the query plan, not just the result).
+6. Outbox: assert `drain_outbox` returns `(BranchKey, Lsn)` ascending across
+   partition boundaries.
+7. Record the outbox decision (table vs replication slot) and assert the chosen
+   one is the one under test.
+
+**Command:** `cargo test -p storage-postgres --test it_postgres_conformance`
+
+**False pass:**
+- A modified copy of the conformance suite. The moment it is edited "just for
+  Postgres", the two backends are no longer proven equivalent — which is the
+  entire acceptance criterion.
+- Step 3 with a single connection, where the sequence is uncontended.
+- A `SERIAL`/sequence-backed LSN. It passes casual reads and is neither per-branch
+  nor gap-free under rollback.
+- Running against SQLite or an in-process fake for CI speed.
+
+## Traps
+
+- A Postgres-only method added to the trait "temporarily". The embedded backend
+  then has a stub and the abstraction is fiction.
+- A global sequence for LSNs. It serializes every run through one atomic and
+  breaks per-branch gap-freedom.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §7, §8.3, §18 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)