98 lines
4.0 KiB
Markdown
98 lines
4.0 KiB
Markdown
# T0.7 — `BlobStore` port + `redb` implementation
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Phase | P0 — Foundations |
|
|
| Size | M — 1 to 3 days |
|
|
| Status | Not started |
|
|
| Flags | — |
|
|
| Spec | inlined below |
|
|
| Blocks | — |
|
|
|
|
## Goal
|
|
|
|
Content-addressed blob storage, deduplicated **within** a tenant and never
|
|
across. `delete` ships now, not later.
|
|
|
|
## Facts (inlined — no spec read needed)
|
|
|
|
```rust
|
|
#[async_trait]
|
|
pub trait BlobStore: Send + Sync {
|
|
async fn put(&self, tenant: TenantId, content: Bytes) -> Result<BlobRef>;
|
|
async fn get(&self, tenant: TenantId, r: &BlobRef) -> Result<Option<Bytes>>;
|
|
/// Reduction and tenant deletion both require this. A store that cannot
|
|
/// delete cannot honour either, and both are obligations.
|
|
async fn delete(&self, tenant: TenantId, r: &BlobRef) -> Result<()>;
|
|
}
|
|
```
|
|
|
|
- **Blobs are namespaced per tenant even though they are content-addressed.**
|
|
Global dedup of prompt and output blobs is tempting — identical system prompts
|
|
across tenants are common — and it is a leak: a shared blob makes one tenant's
|
|
storage accounting depend on another's, and a hash becomes an oracle for "does
|
|
anyone else have this content".
|
|
- `get` returning `None` is a normal outcome, not an error. After reduction
|
|
(T8.4) the original body is deleted and the log carries a
|
|
`Reduced { original, summary }` event explaining what stands in its place.
|
|
- Content addressing means the ref is a hash of the body. A body must never be
|
|
rewritten under an existing ref — that makes the ref a lie.
|
|
|
|
## Steps
|
|
|
|
1. Define `BlobRef` as a Blake3 hash newtype. Define the trait as above.
|
|
2. `redb` table keyed on `Scoped<BlobRef>` — tenant in the key is what enforces
|
|
per-tenant namespacing structurally rather than by convention.
|
|
3. `put`: hash the content, insert if absent, return the ref. Insert-if-absent is
|
|
the dedup, and it is scoped by the key, so no cross-tenant path exists.
|
|
4. `get`: point lookup, `Ok(None)` on miss.
|
|
5. `delete`: remove the key. Deleting a ref one tenant holds must not touch
|
|
another tenant's identical content — which follows from the key shape, and the
|
|
test below is what proves it.
|
|
6. Write the conformance suite generic over `impl BlobStore` so T7.2's
|
|
object-store backend reuses it unchanged.
|
|
|
|
## Acceptance
|
|
|
|
- Identical content put under two tenants produces two independent blobs.
|
|
- Deleting tenant A's blob leaves tenant B's readable.
|
|
|
|
## Verify
|
|
|
|
**Harness:** conformance suite generic over `impl BlobStore`, reused verbatim by
|
|
T7.2's object store.
|
|
|
|
**Integration test** — `tests/it_blobstore_conformance.rs`:
|
|
1. `put` identical bytes under tenant A and tenant B. Assert the returned
|
|
`BlobRef`s are equal (same content hash) **and** that a raw table scan shows
|
|
**two** stored entries.
|
|
2. `delete` tenant A's ref. Assert `get(A, ref)` is `Ok(None)` and
|
|
`get(B, ref)` still returns the bytes.
|
|
3. `put` the same bytes twice under one tenant; assert one stored entry (dedup
|
|
within the tenant) and a stable ref.
|
|
4. `get` a ref that was never written: `Ok(None)`, not an error.
|
|
5. `delete` a ref that does not exist: `Ok(())`, so reduction can run twice.
|
|
6. Re-hash every stored body and assert it matches its key.
|
|
|
|
**Command:** `cargo test -p storage --test it_blobstore_conformance`
|
|
|
|
**False pass:**
|
|
- Step 1 asserting only that both `get`s return the right bytes. A single shared
|
|
blob passes that. The **raw scan showing two entries** is the isolation proof.
|
|
- Step 2 passing because `delete` is a no-op stub. Assert the `None` on A
|
|
explicitly, not just B's survival.
|
|
|
|
## Traps
|
|
|
|
- A global content-addressed table with tenant tracked in a side index. It
|
|
dedups across tenants by construction, which is the leak.
|
|
- Deferring `delete` because nothing calls it yet. Reduction and tenant deletion
|
|
both need it, and retrofitting a delete path into a store designed without one
|
|
is a rewrite.
|
|
|
|
---
|
|
|
|
Background (not required to do this task):
|
|
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §3, §7, §8.6 ·
|
|
[rust-agentic-task.md](../../../rust-agentic-task.md)
|