27 lines
1.4 KiB
Markdown
27 lines
1.4 KiB
Markdown
# 1.2 — Streaming passthrough (GREEN)
|
|
|
|
Phase: 1 — Proxy core
|
|
Stage: RED
|
|
Depends on: [1.1](1.1-reverse-proxy.md)
|
|
|
|
- [ ] An SSE response from an upstream reaches the client unbuffered: each `data:` event is readable by the client before the upstream has sent the next one
|
|
- [ ] A chunked response reaches the client chunk by chunk, not accumulated and flushed at completion
|
|
- [ ] Response headers reach the client before the first body byte, not after
|
|
- [ ] `Content-Type: text/event-stream` and the upstream's `Cache-Control` and `Connection` semantics survive the proxy
|
|
- [ ] No response body is written to memory or disk in full before forwarding
|
|
- [ ] The terminating `data: [DONE]` sentinel and the final zero-length chunk pass through
|
|
- [ ] A test asserts wall-clock ordering: the Nth event is observed at the client before the upstream emits the N+1th
|
|
|
|
The whole product is token streaming. If the gateway buffers, a caller waits minutes
|
|
for a response that should have started in seconds, and the user-visible behaviour of
|
|
the model API regresses versus Kong. nginx in front is already configured with
|
|
`proxy-buffering: off`; the gateway must not reintroduce buffering behind it.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
go test ./internal/proxy/... -run TestSSEUnbuffered -v
|
|
# expected: passes — client observes each of 5 stub-emitted SSE events with the
|
|
# upstream still open, and total observed inter-event gaps match the stub's emit delays
|
|
```
|