fix(server): stop WriteTimeout from killing in-progress LLM streams
Build / Build and push image (push) Successful in 34s
CI / Test, vet, build (push) Successful in 2m16s

http.Server.WriteTimeout is an absolute deadline over the whole
response, not an inactivity timeout -- 15s was cutting off SSE streams
from the reasoning model mid-generation, surfacing to clients as a
"terminated" error well before the model finished. Switch to
ReadHeaderTimeout (protects against slow headers without capping body
duration) and raise WriteTimeout to match the edge nginx Ingress's
proxy-read/send-timeout of 3600s.
This commit is contained in:
Story Crater Bot
2026-08-21 20:06:10 -07:00
parent 0aaf4116f1
commit 6cf5c3e5a9
+11 -5
View File
@@ -21,11 +21,17 @@ type Server struct {
func New(listenAddr string, shutdownTimeout time.Duration, handler http.Handler) *Server {
return &Server{
httpServer: &http.Server{
Addr: listenAddr,
Handler: handler,
ReadTimeout: 15 * time.Second,
WriteTimeout: 15 * time.Second,
IdleTimeout: 60 * time.Second,
Addr: listenAddr,
Handler: handler,
// ReadHeaderTimeout (not ReadTimeout) and a long WriteTimeout: both
// ReadTimeout and WriteTimeout are absolute deadlines covering the
// whole request/response body, not inactivity timeouts -- a 15s
// WriteTimeout here was killing in-progress LLM SSE streams (proxy.go's
// outbound transport deliberately avoids this same mistake). Mirrors
// the edge nginx Ingress's proxy-read/send-timeout of 3600s.
ReadHeaderTimeout: 15 * time.Second,
WriteTimeout: 1 * time.Hour,
IdleTimeout: 60 * time.Second,
},
shutdownTimeout: shutdownTimeout,
healthChecker: NewHealthChecker(false, false),