poimen-memory/tasks/M3.5.7-rate-limiting.md

M3.5.7 — Rate limiting (per-apikey) and idempotency by sha256

Field	Value
Phase	M3.5 — Distributed API Layer
Size	M — 1–3 days
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M3.5.8
Depends	M3.5.2, M3.5.3 (ingest and query endpoints exist)

Goal

Rate limiting prevents abusive load; idempotency ensures retry safety. Both are per-apikey and per-endpoint.

Design

Rate limits (defaults, configurable via env):

• POST /memory/ingest: 100 jobs/hour per apikey
• GET /memory/query: 1000 requests/hour per apikey
• GET /memory/skills: unlimited
• GET /memory/projects: 100 requests/hour per apikey

Burst allowance: 10 requests/second (hard burst cap, then 429).

Response on rate limit:

HTTP 429 Too Many Requests
Retry-After: 47
{
  "error": "rate_limit_exceeded",
  "reason": "100 requests/hour for POST /memory/ingest",
  "retry_after_seconds": 47,
  "limit_window": "3600s"
}

Idempotency:

• POST /memory/ingest uses ingest_id (SHA256 of batch content) as idempotency key
• Same ingest_id resubmitted within 24 hours returns same job_id, no re-enqueue
• Idempotency key extracted from request body (not header)

Implementation

Rate limiting strategy: Token bucket per apikey per endpoint. Track in memory (not Redis yet).

pub struct RateLimiter {
    buckets: Arc<Mutex<HashMap<String, Vec<RateBucket>>>>,  // apikey -> [one per endpoint]
}

pub struct RateBucket {
    tokens: f64,
    last_refill: Instant,
    capacity: f64,
    refill_rate: f64,  // tokens/sec
}

Token refill: On each request, add (now - last_refill) * refill_rate tokens (cap at capacity).

Burst handling:

• Allow burst of 10 req/sec without delay
• Requests above burst queued (blocked until tokens available) or rejected (429)
• Decision: reject is simpler and encourages clients to batch. Implement rejection.

Idempotency:

• Extract ingest_id from request body (JSON key or computed if omitted)
• Check against recent idempotency store (memory, 24h TTL)
• If found, return cached response (job_id)
• If not found, process normally and store (ingest_id → response)

Steps

1. Create RateLimiter struct:

   impl RateLimiter {
     fn new() -> Self { /* init empty */ }
     fn check(&mut self, apikey: &str, endpoint: &str) -> Result<(), RateLimitError> {
       // refill, check capacity, return Ok or Err with Retry-After
     }
   }
   

2. Add RateLimiter as app state:

   let limiter = Arc::new(Mutex::new(RateLimiter::new()));
   HttpServer::new(move || {
     App::new()
       .app_data(Data::new(limiter.clone()))
   })
   

3. Middleware to extract apikey and check limit:

   pub struct RateLimitMiddleware {
     limits: Arc<Mutex<RateLimiter>>,
   }
   
   impl Middleware for RateLimitMiddleware { ... }
   

   • Extract apikey from request context (set by auth middleware)
   • Determine endpoint (path)
   • Call limiter.check(apikey, endpoint)
   • If Err, return 429 with Retry-After

4. Idempotency store:

   pub struct IdempotencyStore {
     cache: Arc<Mutex<HashMap<String, (HttpResponse, Instant)>>>,
   }
   
   impl IdempotencyStore {
     fn get(&self, key: &str) -> Option<HttpResponse> { /* if not expired */ }
     fn set(&mut self, key: String, response: HttpResponse) { }
   }
   

5. POST /ingest handler:

   • Parse request body to extract ingest_id
   • Query idempotency store for ingest_id
   • If found and not expired (24h), return cached response
   • If not found, process normally:
     • Enqueue ingest
     • Cache the 202 response with ingest_id as key
     • Return response

6. Configuration:

   • Load rate limits from env vars: MEM_RATE_LIMIT_INGEST, MEM_RATE_LIMIT_QUERY, etc.
   • Load burst cap from env: MEM_RATE_LIMIT_BURST (default 10 req/sec)
   • Load idempotency TTL from env: MEM_IDEMPOTENCY_TTL_SECS (default 86400)

Acceptance

• Requests within limit succeed (200 or 202)
• Requests at burst cap (10/sec) blocked immediately
• Rate limit reset after time window (test with mocked time)
• Same ingest_id resubmitted returns same job_id (idempotent)
• Different ingest_id queued separately
• Retry-After header correct

Verify

Harness: Integration tests + time mocking.

Integration test — tests/it_rate_limiting.rs:

1. a1_within_limit_succeeds — 5 consecutive GET /query requests within 1-hour limit all succeed (200).
2. a2_at_burst_cap_429 — 11 GET /query requests in 1 second, 11th returns 429.
3. a3_limit_window_resets — 100 GET /query requests in hour 1 all succeed (limit reached), 101st fails (429), mock time to hour+2, 102nd succeeds (window reset).
4. a4_per_apikey_isolation — two different apikeys, each send 5 requests, both succeed (limits are independent).
5. a5_per_endpoint_isolation — 100 POST /ingest requests succeed (limit=100), 1 GET /query request succeeds (different endpoint, different limit).
6. a6_retry_after_header — 429 response includes Retry-After: N header with correct value.
7. a7_ingest_id_idempotent — POST /ingest with id_a succeeds, POST again with id_a returns same job_id.
8. a8_different_ingest_ids_separate — POST /ingest (id_a), POST (id_b) both succeed with different job_ids.
9. a9_idempotency_expires — POST /ingest (id_a), mock time to 25 hours later, POST (id_a) again returns different job_id (old idempotency cache expired).
10. a10_rate_limit_per_endpoint_documented — grep the code for limit values; each endpoint has a defined limit.

Command: cargo test -p mem-cli rate_limiting -- --nocapture

False pass:

• Burst cap tested with 10 requests but timing is imprecise (some reqs slow, burst calc off).
• Rate limit window reset never tested with time mock. Limits always work within a short test window.
• Idempotency key never actually extracted from body; hardcoded in test.
• Per-apikey isolation not tested with two keys.

Traps

• Token bucket refill at Instant::now() is wall-clock time; in tests, use a mock clock (or avoid time-dependent tests).
• Burst cap as "10 req/sec" is naive if requests take 100ms each (effectively 10 concurrent). Real burst is 10 within the same millisecond. Better: track request arrival rate over a sliding window.
• Idempotency cache unbounded growth. Must evict expired entries (implement on-read eviction or background sweep).
• Rate limit math: capacity=100 tokens/hour, refill=100/3600 tokens/sec. A request at t=0 uses 1 token (99 left). At t=36s, 1 token is refilled (100 left) — this is correct. Watch for off-by-one.

---

Background: DESIGN.md § Distributed API Layer § Auth & rate limits