Addresses three critical network issues for LLM streaming performance: **#33 Disable proxy buffering for SSE** - Add X-Accel-Buffering: no header to response - Tells nginx/Ingress to stream events immediately instead of buffering **#32 HTTP/2 multiplexing for concurrent streams** - Enable HTTP/2 in server config via http2.ConfigureServer() - Increase MaxConnsPerHost to 10 for better concurrency - Allows multiple concurrent LLM requests without blocking **#31 TCP backpressure for streaming LLM responses** - Set TCP_NODELAY on dialer to disable Nagle's algorithm - Reduces latency by sending small packets immediately - Critical for low TTFT (time-to-first-token) under load **Tests added:** - TestTCPBackpressure: Verifies TCP backpressure handling with slow client - TestConcurrentSSEStreams: Confirms HTTP/2 multiplexing works correctly --------- Co-authored-by: poison <[email protected]> Reviewed-on: #26 Co-authored-by: poimen <[email protected]>
Implement Kubernetes-native CI/CD with Tekton Pipelines for pre-merge integration testing. ## What This Does Adds Tekton Pipelines to orchestrate integration tests before deployment: 1. **Tekton Task** (task-integration-test.yaml) - Runs integration tests in container - Tests memory, S3, SQS, workflow, IAM services - Reports pass/fail results 2. **Tekton Pipeline** (pipeline-integration-test.yaml) - Parameterized pipeline for reusability - Takes image tag as input - Outputs test results 3. **ArgoCD Application** (k8s/argocd-apps/tekton.yaml) - Manages Tekton installation - Tekton controller watches and executes PipelineRuns - No manual kubectl applies needed 4. **Updated CI** (.gitea/workflows/ci.yaml) - Build image with commit SHA - Create PipelineRun to test image - Wait for Tekton to complete tests - Only promote to :latest if tests pass - ArgoCD detects :latest and deploys ## Architecture git push → CI builds image:sha → Create PipelineRun → Tekton runs tests → Results to CI → Promote :latest → ArgoCD deploys ## Code Quality ✓ DRY: Parameterized, reusable Task and Pipeline ✓ SOLID: Single responsibility, clean interfaces ✓ GitOps: Everything in git, managed by ArgoCD ✓ Security: Non-root containers, resource limits ✓ Observable: Logs, status, results tracking ## Files Changed - k8s/tekton/task-integration-test.yaml - Task definition - k8s/tekton/pipeline-integration-test.yaml - Pipeline definition - k8s/tekton/kustomization.yaml - Kustomize for management - k8s/tekton/base/tekton-release.yaml - Release reference - k8s/tekton/README.md - Documentation - k8s/argocd-apps/tekton.yaml - ArgoCD Application - .gitea/workflows/ci.yaml - Updated CI workflow ## Review Checklist - [ ] Tekton manifests are clean and parameterized - [ ] ArgoCD Application properly configured - [ ] CI workflow correctly triggers PipelineRun - [ ] Error handling for test failures - [ ] Logs and status properly captured - [ ] Documentation is clear ## Testing After merge: 1. ArgoCD syncs and installs Tekton Pipelines 2. Next git push triggers CI 3. CI creates PipelineRun 4. Tekton runs integration tests 5. Results show in CI workflow --------- Co-authored-by: poimen <[email protected]> Reviewed-on: #25 Co-authored-by: poimen <[email protected]>
