poimen-memory/tasks/M1.1-llm-chat-client.md

# M1.1 — `mem-llm` chat client

| Field | Value |
|---|---|
| Phase | M1 — Gated loop at L1 |
| Size | M — 1–3 days |
| Status | ✅ Done |
| Flags | — |
| Spec | inlined below |
| Blocks | M0.1 |

## Goal

Talk to the homelab gateway, with the two non-obvious details that cost a day to
find already baked in.

## Files

| Action | Path |
|---|---|
| Create | `crates/mem-llm/src/chat.rs` — `ChatClient`, `Completion`, `Usage` |
| Replace | `crates/mem-llm/src/lib.rs` — replace `pub mod placeholder {}` with `pub mod chat;` + re-exports |
| Create | `tests/it_chat_client.rs` — integration tests (workspace root, matches existing convention) |
| Modify | `Cargo.toml` root — add `wiremock = "0.6"` to `[dev-dependencies]` |

## Dependencies

| Crate | Where | Already present? |
|---|---|---|
| `reqwest` (json feature) | `crates/mem-llm/Cargo.toml` | ✅ yes |
| `serde`, `serde_json` | `crates/mem-llm/Cargo.toml` | ✅ yes |
| `tokio` | `crates/mem-llm/Cargo.toml` | ✅ yes |
| `anyhow`, `thiserror` | `crates/mem-llm/Cargo.toml` | ✅ yes |
| `wiremock = "0.6"` | root `Cargo.toml` `[dev-dependencies]` | ❌ add |

## Existing code

- `crates/mem-llm/src/lib.rs` is an empty placeholder — replace entirely
- No existing HTTP client code to reuse; build from scratch
- `crates/mem-core/src/lesson.rs` has an unrelated events JSONL writer — ignore it here

## API shape

```http
POST https://api.riotpiao.com/v1/qwen/chat/completions

Headers:
  apikey: <value of MEM_API_KEY env var>
  Content-Type: application/json

Body (note: NO "tools" key — not even an empty array):
{
  "model": "qwen2.5:3b-instruct",
  "messages": [
    {"role": "system", "content": "<system prompt>"},
    {"role": "user", "content": "<user prompt>"}
  ],
  "max_tokens": 2048
}

Response 200:
{
  "choices": [
    {"message": {"role": "assistant", "content": "<model output>"}}
  ],
  "usage": {
    "prompt_tokens": 1234,
    "completion_tokens": 567,
    "total_tokens": 1801
  }
}

Response 401 (wrong auth header):
{"message": "Unauthorized"}

Response 400 (body too large or malformed):
{"error": {"message": "[] is too short - 'messages'"}}
```

## Facts (inlined — no spec read needed)

```
base   https://api.riotpiao.com/v1
route  POST /v1/qwen/chat/completions      qwen2.5:3b-instruct
       POST /v1/ornith/chat/completions    ornith:35b
       POST /v1/reasoning/chat/completions DeepSeek-R1-Distill-32B (no tools)
```

**Auth is `apikey:`, not `Authorization: Bearer`.** Kong's `key-auth` compares
the whole header value against the stored key, so the OpenAI SDK convention
returns 401. Verified:

```
-H "apikey: $KEY"                 -> 200
-H "Authorization: $KEY"          -> 200
-H "Authorization: Bearer $KEY"   -> 401
```

**One provider per route.** `baseUrl` is per-route and the model id is `ornith:35b`
with the tag, not `ornith`. `/v1/models` advertises `/v1/score` which does not
work — do not trust that list as a capability probe.

Request bodies above ~10.6 KB used to fail with
`{"error":{"message":"[] is too short - 'messages'"}}`; the Kong body buffer was
raised to 16m and it is fixed. If that error ever reappears, it is the buffer,
not the client.

Send **no `tools` array**. The controller needs none, and the reasoning route
rejects any request carrying one.

## Steps

1. `ChatClient::new(base_url, api_key, model)` in `mem-llm`.
2. Send the `apikey` header. Read the key from `MEM_API_KEY`, never from a
   committed file.
3. `complete(system, user, max_tokens) -> Completion { text, usage, latency }`.
4. Timeout default 300s — local models are slow to first token and a cold load
   can take minutes.
5. Retry on 5xx and timeout with exponential backoff, max 3. **Do not retry 4xx**
   — a 400 is a malformed request and retrying it just costs three times as much.
6. On any error, include the response body in the error. The useful information
   is always in the body, never the status.
7. `MEM_LLM_RECORD=<dir>` writes every request/response pair to disk, for
   building fixtures without hand-writing them.

## Acceptance

- A real completion round-trips against the gateway.
- A 401 is reported as an auth error naming the header convention.
- A 4xx is not retried; a 5xx is.

## Verify

**Harness:** `wiremock` for the offline tests; one `#[ignore]` test against the
live gateway.

**Integration test** — `tests/it_chat_client.rs`:
1. `a1_sends_apikey_header` — assert the mock received `apikey` and **no**
   `authorization` header.
2. `a2_no_tools_field` — assert the serialized body has no `tools` key at all,
   not merely an empty array.
3. `a3_retries_5xx` — mock 503 twice then 200; assert 3 requests and success.
4. `a4_does_not_retry_4xx` — mock 400; assert exactly 1 request and an error
   carrying the body text.
5. `a5_timeout_is_configurable` — mock a 2s delay with a 1s timeout; assert a
   timeout error.
6. `a6_live_smoke` — `#[ignore]`; real gateway, `qwen2.5:3b-instruct`, prompt
   "reply with exactly: pong", assert the text contains `pong`.

**Command:** `cargo test --test it_chat_client` (add `-- --ignored` for a6)

**False pass:**
- Testing only against the mock. The mock accepts whatever header you send it;
  assertion 6 against the live gateway is the only thing that proves the auth
  convention is right.
- Asserting `tools: []` is absent by checking `body.tools.is_empty()`. An empty
  array serialized into the request is still a `tools` key, and that is what the
  reasoning route rejects. Assert on the raw JSON.

## Traps

- Using `Authorization: Bearer`. It is what every SDK does and it 401s here.
- Retrying 400s. The body-buffer bug produced a 400 for a whole day; retrying it
  tripled the load and produced identical failures more slowly.
- A 60s timeout. That is the value that made `ornith` look broken when it was
  merely cold.

---

Background: [DESIGN.md](../DESIGN.md) — Verified facts