poimen-memory/tasks/M3.5.7-rate-limiting.md

# M3.5.7 — Rate limiting (per-apikey) and idempotency by sha256

| Field | Value |
|---|---|
| Phase | M3.5 — Distributed API Layer |
| Size | M — 1–3 days |
| Status | ⬜ Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | M3.5.8 |
| Depends | M3.5.2, M3.5.3 (ingest and query endpoints exist) |

## Goal

Rate limiting prevents abusive load; idempotency ensures retry safety. Both are per-apikey and per-endpoint.

## Design

**Rate limits (defaults, configurable via env):**
- `POST /memory/ingest`: 100 jobs/hour per apikey
- `GET /memory/query`: 1000 requests/hour per apikey
- `GET /memory/skills`: unlimited
- `GET /memory/projects`: 100 requests/hour per apikey

**Burst allowance:** 10 requests/second (hard burst cap, then 429).

**Response on rate limit:**
```
HTTP 429 Too Many Requests
Retry-After: 47
{
  "error": "rate_limit_exceeded",
  "reason": "100 requests/hour for POST /memory/ingest",
  "retry_after_seconds": 47,
  "limit_window": "3600s"
}
```

**Idempotency:** 
- `POST /memory/ingest` uses `ingest_id` (SHA256 of batch content) as idempotency key
- Same `ingest_id` resubmitted within 24 hours returns same `job_id`, no re-enqueue
- Idempotency key extracted from request body (not header)

## Implementation

**Rate limiting strategy:** Token bucket per apikey per endpoint. Track in memory (not Redis yet).
```rust
pub struct RateLimiter {
    buckets: Arc<Mutex<HashMap<String, Vec<RateBucket>>>>,  // apikey -> [one per endpoint]
}

pub struct RateBucket {
    tokens: f64,
    last_refill: Instant,
    capacity: f64,
    refill_rate: f64,  // tokens/sec
}
```

**Token refill:** On each request, add `(now - last_refill) * refill_rate` tokens (cap at capacity).

**Burst handling:** 
- Allow burst of 10 req/sec without delay
- Requests above burst queued (blocked until tokens available) or rejected (429)
- Decision: **reject** is simpler and encourages clients to batch. Implement rejection.

**Idempotency:**
- Extract `ingest_id` from request body (JSON key or computed if omitted)
- Check against recent idempotency store (memory, 24h TTL)
- If found, return cached response (job_id)
- If not found, process normally and store (ingest_id → response)

## Steps

1. Create `RateLimiter` struct:
   ```rust
   impl RateLimiter {
     fn new() -> Self { /* init empty */ }
     fn check(&mut self, apikey: &str, endpoint: &str) -> Result<(), RateLimitError> {
       // refill, check capacity, return Ok or Err with Retry-After
     }
   }
   ```

2. Add `RateLimiter` as app state:
   ```rust
   let limiter = Arc::new(Mutex::new(RateLimiter::new()));
   HttpServer::new(move || {
     App::new()
       .app_data(Data::new(limiter.clone()))
   })
   ```

3. Middleware to extract apikey and check limit:
   ```rust
   pub struct RateLimitMiddleware {
     limits: Arc<Mutex<RateLimiter>>,
   }
   
   impl Middleware for RateLimitMiddleware { ... }
   ```
   - Extract apikey from request context (set by auth middleware)
   - Determine endpoint (path)
   - Call `limiter.check(apikey, endpoint)`
   - If Err, return 429 with Retry-After

4. Idempotency store:
   ```rust
   pub struct IdempotencyStore {
     cache: Arc<Mutex<HashMap<String, (HttpResponse, Instant)>>>,
   }
   
   impl IdempotencyStore {
     fn get(&self, key: &str) -> Option<HttpResponse> { /* if not expired */ }
     fn set(&mut self, key: String, response: HttpResponse) { }
   }
   ```

5. `POST /ingest` handler:
   - Parse request body to extract `ingest_id`
   - Query idempotency store for `ingest_id`
   - If found and not expired (24h), return cached response
   - If not found, process normally:
     - Enqueue ingest
     - Cache the 202 response with ingest_id as key
     - Return response

6. Configuration:
   - Load rate limits from env vars: `MEM_RATE_LIMIT_INGEST`, `MEM_RATE_LIMIT_QUERY`, etc.
   - Load burst cap from env: `MEM_RATE_LIMIT_BURST` (default 10 req/sec)
   - Load idempotency TTL from env: `MEM_IDEMPOTENCY_TTL_SECS` (default 86400)

## Acceptance

- Requests within limit succeed (200 or 202)
- Requests at burst cap (10/sec) blocked immediately
- Rate limit reset after time window (test with mocked time)
- Same ingest_id resubmitted returns same job_id (idempotent)
- Different ingest_id queued separately
- Retry-After header correct

## Verify

**Harness:** Integration tests + time mocking.

**Integration test** — `tests/it_rate_limiting.rs`:
1. `a1_within_limit_succeeds` — 5 consecutive GET /query requests within 1-hour limit all succeed (200).
2. `a2_at_burst_cap_429` — 11 GET /query requests in 1 second, 11th returns 429.
3. `a3_limit_window_resets` — 100 GET /query requests in hour 1 all succeed (limit reached), 101st fails (429), mock time to hour+2, 102nd succeeds (window reset).
4. `a4_per_apikey_isolation` — two different apikeys, each send 5 requests, both succeed (limits are independent).
5. `a5_per_endpoint_isolation` — 100 POST /ingest requests succeed (limit=100), 1 GET /query request succeeds (different endpoint, different limit).
6. `a6_retry_after_header` — 429 response includes `Retry-After: N` header with correct value.
7. `a7_ingest_id_idempotent` — POST /ingest with id_a succeeds, POST again with id_a returns same job_id.
8. `a8_different_ingest_ids_separate` — POST /ingest (id_a), POST (id_b) both succeed with different job_ids.
9. `a9_idempotency_expires` — POST /ingest (id_a), mock time to 25 hours later, POST (id_a) again returns different job_id (old idempotency cache expired).
10. `a10_rate_limit_per_endpoint_documented` — grep the code for limit values; each endpoint has a defined limit.

**Command:** `cargo test -p mem-cli rate_limiting -- --nocapture`

**False pass:**
- Burst cap tested with 10 requests but timing is imprecise (some reqs slow, burst calc off).
- Rate limit window reset never tested with time mock. Limits always work within a short test window.
- Idempotency key never actually extracted from body; hardcoded in test.
- Per-apikey isolation not tested with two keys.

## Traps

- Token bucket refill at Instant::now() is wall-clock time; in tests, use a mock clock (or avoid time-dependent tests).
- Burst cap as "10 req/sec" is naive if requests take 100ms each (effectively 10 concurrent). Real burst is 10 within the same millisecond. Better: track request arrival rate over a sliding window.
- Idempotency cache unbounded growth. Must evict expired entries (implement on-read eviction or background sweep).
- Rate limit math: capacity=100 tokens/hour, refill=100/3600 tokens/sec. A request at t=0 uses 1 token (99 left). At t=36s, 1 token is refilled (100 left) — this is correct. Watch for off-by-one.

---

Background: [DESIGN.md § Distributed API Layer § Auth & rate limits](../DESIGN.md#scaling-constraints)