3 Commits

Include renames

Author	SHA1	Message	Date
3 people	30e0a83a50	feat(network): SSE optimization for local LLM streaming (#31 #32 #33) (#26) ... Addresses three critical network issues for LLM streaming performance: **#33 Disable proxy buffering for SSE** - Add X-Accel-Buffering: no header to response - Tells nginx/Ingress to stream events immediately instead of buffering **#32 HTTP/2 multiplexing for concurrent streams** - Enable HTTP/2 in server config via http2.ConfigureServer() - Increase MaxConnsPerHost to 10 for better concurrency - Allows multiple concurrent LLM requests without blocking **#31 TCP backpressure for streaming LLM responses** - Set TCP_NODELAY on dialer to disable Nagle's algorithm - Reduces latency by sending small packets immediately - Critical for low TTFT (time-to-first-token) under load **Tests added:** - TestTCPBackpressure: Verifies TCP backpressure handling with slow client - TestConcurrentSSEStreams: Confirms HTTP/2 multiplexing works correctly --------- Co-authored-by: poison <[email protected]> Reviewed-on: #26 Co-authored-by: poimen <[email protected]>	2026-09-13 23:37:33 +00:00
Story Crater Bot	6cf5c3e5a9	fix(server): stop WriteTimeout from killing in-progress LLM streams ... http.Server.WriteTimeout is an absolute deadline over the whole response, not an inactivity timeout -- 15s was cutting off SSE streams from the reasoning model mid-generation, surfacing to clients as a "terminated" error well before the model finished. Switch to ReadHeaderTimeout (protects against slow headers without capping body duration) and raise WriteTimeout to match the edge nginx Ingress's proxy-read/send-timeout of 3600s.	2026-08-21 20:06:10 -07:00
Story Crater BotandClaude Opus 5	058f11cf2b	chore: initial commit of Go API gateway ... Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>	2026-08-19 20:54:34 -07:00