Admin Bot
cc9a32f53a
CI / CI (pull_request) Successful in 3m11s
Addresses three critical network issues for LLM streaming performance: **#33 Disable proxy buffering for SSE** - Add X-Accel-Buffering: no header to response - Tells nginx/Ingress to stream events immediately instead of buffering - Paired with ResponseController.Flush() for unbuffered token delivery **#32 HTTP/2 multiplexing for concurrent streams** - Enable HTTP/2 in server config via http2.ConfigureServer() - Increase MaxConnsPerHost from default (2) to 10 - ForceAttemptHTTP2 on outbound Transport for upstream connections - Allows multiple concurrent LLM requests without blocking **#31 TCP backpressure for streaming LLM responses** - Set TCP_NODELAY on dialer to disable Nagle's algorithm - Reduces latency by sending small packets immediately - Critical for low TTFT (time-to-first-token) under load - Upstream Transport respects backpressure when clients read slowly **Tests added:** - TestTCPBackpressure: Verifies TCP backpressure handling with slow client - TestConcurrentSSEStreams: Confirms HTTP/2 multiplexing works correctly - Both pass at 0.11s and 0.06s respectively Fixes all three streaming performance issues in one coherent change.
37 lines
807 B
Bash
37 lines
807 B
Bash
#!/bin/bash
|
|
# Example: Send email via notification/sendMsg endpoint
|
|
|
|
BASE_URL="${1:-https://api.riotpiao.com}"
|
|
AUTH_TOKEN="${2:-}" # Optional JWT token if auth required
|
|
|
|
PAYLOAD=$(cat <<'EOF'
|
|
{
|
|
"format": "smtp",
|
|
"title": "System Alert",
|
|
"message": "CPU usage exceeded 90% threshold",
|
|
"priority": 7,
|
|
"extras": {
|
|
"to_email": "[email protected]",
|
|
"cc": "[email protected]"
|
|
}
|
|
}
|
|
EOF
|
|
)
|
|
|
|
if [ -n "$AUTH_TOKEN" ]; then
|
|
curl -X POST "$BASE_URL" \
|
|
-H "X-Service: notification" \
|
|
-H "X-Resource: sendMsg" \
|
|
-H "Content-Type: application/json" \
|
|
-H "Authorization: Bearer $AUTH_TOKEN" \
|
|
-d "$PAYLOAD"
|
|
else
|
|
curl -X POST "$BASE_URL" \
|
|
-H "X-Service: notification" \
|
|
-H "X-Resource: sendMsg" \
|
|
-H "Content-Type: application/json" \
|
|
-d "$PAYLOAD"
|
|
fi
|
|
|
|
echo ""
|