feat(network): SSE optimization for local LLM streaming (#31 #32 #33) #26

Merged

rock merged 4 commits from feat/sse-optimization-31-32-33 into main 2026-09-13 23:37:34 +00:00

Conversation 0 Commits 4 Files Changed 23

+1606 -797

4 Commits

Author	SHA1	Message	Date
Admin Bot	6608f1a8d5	test: add workflow visibility tests for poimen-harness namespace ... Verify that WorkflowAdapter provides visibility into terminated workflows in the poimen-harness namespace. This ensures namespace pass-down feature is working correctly and users can specify different domains/namespaces via X-Service: workflow requests. Tests added: 1. integration-test.sh: Added workflow visibility tests - List workflows in poimen-harness namespace - Verify terminated/completed workflows are visible - Validate namespace parameter requirement - Check auth enforcement 2. workflow-visibility-test.sh: NEW dedicated workflow test script - Tests WorkflowAdapter namespace pass-down - Verifies list, describe, and auth enforcement - Specific focus on poimen-harness namespace - Looks for 4 terminated workflows 3. task-workflow-visibility.yaml: NEW Tekton task - Runs workflow visibility tests against live gateway - Sidecar deployment pattern - Publishes result + summary + workflow-count metrics 4. pipeline-sse-optimization.yaml: Updated - Added workflow-visibility-tests stage (runs after integration-tests) - Updated report-results to include workflow test results - Full pipeline now: integration → workflow-visibility → load → report 5. kustomization.yaml: Updated - Added task-workflow-visibility.yaml - Added workflow-visibility-test-script ConfigMap This ensures that the deprecated /workflows endpoint replacement correctly supports multi-tenant access via namespace specification in request payload.	2026-09-14 08:24:42 +09:00
Admin Bot	c6cd41fd4b	feat: implement WorkflowAdapter with namespace pass-down support ... Enable X-Service: workflow routing to Temporal via ServiceAdapter. Users can now specify namespace/domain in request payload for multi-tenant workflow access. Changes: - Implement WorkflowAdapter in serviceadapter/workflow_adapter.go * Defines 10 workflow resources: start, describe, list, history, terminate, cancel, signal, query, reset, update * Each resource validates namespace parameter in payload * Forwards requests to Temporal gRPC handler - Add GetWorkflowSpec() to define ServiceAdapter spec with: * Upstream: grpc://temporal:7233 * Auth requirements per operation (execute, read, signal, query) * Request/response schemas for validation - Wire WorkflowAdapter into main.go: * Register workflow adapter in serviceadapter registry * Initialize with temporal handler for gRPC forwarding - Remove old empty WorkflowAdapter stub from adapters.go Usage: curl -X POST https://api.riotpiao.com/ \ -H 'X-Service: workflow' \ -H 'X-Resource: start' \ -H 'Authorization: Bearer TOKEN' \ -d '{ "namespace": "default", "workflow_id": "my-workflow", "workflow_type": "MyWorkflow", "task_queue": "default" }' Namespace is required in all workflow operations and must be specified by the client in the request payload. This enables multi-tenant support where different teams access their own Temporal namespaces.	2026-09-14 08:21:49 +09:00
Admin Bot	b0f608f145	refactor: retire /workflows endpoint, use X-Service: workflow instead ... Remove deprecated /workflows HTTP endpoint in favor of unified X-Service header routing. All workflow operations now route through: X-Service: workflow X-Resource: {action} (start, describe, signal, query, etc) This consolidates routing patterns and allows users to specify domain/ namespace via request payload instead of path prefixes. Changes: - Remove internal/proxy/workflows.go (484 lines of predefined workflows) - Remove internal/proxy/workflows_test.go - Remove /workflows handler from proxy.ServeHTTP() - Update README.md to document X-Service routing pattern - Add migration note: use X-Service: workflow instead of /workflows WorkflowAdapter in serviceadapter/ handles X-Service: workflow requests and forwards to Temporal gRPC API. Users can now specify domain/namespace in request payload for multi-tenant workflow access. Related: #26	2026-09-14 08:18:36 +09:00
Admin Bot	cc9a32f53a	feat(network): SSE optimization for local LLM streaming (#31 #32 #33) ... Addresses three critical network issues for LLM streaming performance: **#33 Disable proxy buffering for SSE** - Add X-Accel-Buffering: no header to response - Tells nginx/Ingress to stream events immediately instead of buffering - Paired with ResponseController.Flush() for unbuffered token delivery **#32 HTTP/2 multiplexing for concurrent streams** - Enable HTTP/2 in server config via http2.ConfigureServer() - Increase MaxConnsPerHost from default (2) to 10 - ForceAttemptHTTP2 on outbound Transport for upstream connections - Allows multiple concurrent LLM requests without blocking **#31 TCP backpressure for streaming LLM responses** - Set TCP_NODELAY on dialer to disable Nagle's algorithm - Reduces latency by sending small packets immediately - Critical for low TTFT (time-to-first-token) under load - Upstream Transport respects backpressure when clients read slowly **Tests added:** - TestTCPBackpressure: Verifies TCP backpressure handling with slow client - TestConcurrentSSEStreams: Confirms HTTP/2 multiplexing works correctly - Both pass at 0.11s and 0.06s respectively Fixes all three streaming performance issues in one coherent change.	2026-09-14 08:14:04 +09:00