docs: comprehensive api and testing documentation
- API.md: Full REST API documentation with examples * All endpoints (health, models, chat, embeddings, rerank) * Request/response schemas * Error handling (RFC 9457 problem+json) * Examples in bash, Python, TypeScript - TESTING_GUIDE.md: Quick reference testing guide * 15 copy-paste test commands * Complete testing checklist * Troubleshooting guide * Performance testing examples * Integration test scripts Ready for deployment verification and integration testing.
This commit is contained in:
Story Crater Bot
@@ -0,0 +1,934 @@
|
||||
# API Gateway Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
The homelab-frontend gateway is a production-ready reverse proxy for LLM model inference. It routes requests to multiple model upstreams based on configuration, with support for streaming, tool calling, and multiple API formats.
|
||||
|
||||
**Base URL**: `https://api.riotpiao.com`
|
||||
|
||||
**Deployment**: Client → nginx ingress → gateway → model upstreams
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Health Endpoints](#health-endpoints)
|
||||
2. [GET /v1/models](#get-v1models) - List available models
|
||||
3. [POST /v1/chat/completions](#post-v1chat-completions) - Chat with LLM
|
||||
4. [POST /v1/embeddings](#post-v1embeddings) - Generate embeddings
|
||||
5. [POST /v1/rerank](#post-v1rerank) - Rerank documents
|
||||
6. [Error Handling](#error-handling)
|
||||
7. [Examples](#examples)
|
||||
|
||||
---
|
||||
|
||||
## Health Endpoints
|
||||
|
||||
### GET /healthz
|
||||
|
||||
Always returns 200 (liveness probe).
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{"status":"alive"}
|
||||
```
|
||||
|
||||
**Status Code**: 200
|
||||
|
||||
---
|
||||
|
||||
### GET /readyz
|
||||
|
||||
Returns 200 when the gateway is ready (config loaded, upstreams available).
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{"status":"ready"}
|
||||
```
|
||||
|
||||
**Status Code**: 200 (ready) or 503 (not ready)
|
||||
|
||||
---
|
||||
|
||||
## GET /v1/models
|
||||
|
||||
List all configured models available for dispatch.
|
||||
|
||||
**Method**: GET
|
||||
|
||||
**Path**: `/v1/models`
|
||||
|
||||
**Authentication**: None required
|
||||
|
||||
**Query Parameters**: None
|
||||
|
||||
**Request Headers**:
|
||||
```
|
||||
Accept: application/json
|
||||
```
|
||||
|
||||
**Response Headers**:
|
||||
```
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**Response Schema**:
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "model-name",
|
||||
"object": "model",
|
||||
"owned_by": "api.riotpiao.com",
|
||||
"created": 1700000000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Status Codes**:
|
||||
- `200` - OK
|
||||
|
||||
**Example**:
|
||||
|
||||
```bash
|
||||
curl -s https://api.riotpiao.com/v1/models | jq .
|
||||
```
|
||||
|
||||
**Response Example**:
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "reasoning",
|
||||
"object": "model",
|
||||
"owned_by": "api.riotpiao.com",
|
||||
"created": 1700000000
|
||||
},
|
||||
{
|
||||
"id": "ornith:35b",
|
||||
"object": "model",
|
||||
"owned_by": "api.riotpiao.com",
|
||||
"created": 1700000000
|
||||
},
|
||||
{
|
||||
"id": "qwen2.5:3b-instruct",
|
||||
"object": "model",
|
||||
"owned_by": "api.riotpiao.com",
|
||||
"created": 1700000000
|
||||
},
|
||||
{
|
||||
"id": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"object": "model",
|
||||
"owned_by": "api.riotpiao.com",
|
||||
"created": 1700000000
|
||||
},
|
||||
{
|
||||
"id": "BAAI/bge-reranker-base",
|
||||
"object": "model",
|
||||
"owned_by": "api.riotpiao.com",
|
||||
"created": 1700000000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## POST /v1/chat/completions
|
||||
|
||||
Chat with an LLM model. Routes to upstream based on the `model` field in the request body.
|
||||
|
||||
**Method**: POST
|
||||
|
||||
**Path**: `/v1/chat/completions`
|
||||
|
||||
**Authentication**: None required (future: Bearer token)
|
||||
|
||||
**Request Headers**:
|
||||
```
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**Request Body Schema**:
|
||||
```json
|
||||
{
|
||||
"model": "string (required)",
|
||||
"messages": [
|
||||
{
|
||||
"role": "string (user|assistant|system)",
|
||||
"content": "string|array (required)",
|
||||
"tool_calls": "array (optional, from assistant)"
|
||||
}
|
||||
],
|
||||
"temperature": "number (optional, 0-2)",
|
||||
"top_p": "number (optional, 0-1)",
|
||||
"max_tokens": "integer (optional)",
|
||||
"stream": "boolean (optional, default: false)",
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "string",
|
||||
"description": "string",
|
||||
"parameters": "object"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response Schema** (non-streaming):
|
||||
```json
|
||||
{
|
||||
"id": "string",
|
||||
"object": "chat.completion",
|
||||
"created": "integer",
|
||||
"model": "string",
|
||||
"choices": [
|
||||
{
|
||||
"index": "integer",
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "string|null",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "string",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "string",
|
||||
"arguments": "string (JSON)"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"finish_reason": "stop|tool_calls|length"
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": "integer",
|
||||
"completion_tokens": "integer",
|
||||
"total_tokens": "integer"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response Schema** (streaming):
|
||||
```
|
||||
data: {"id":"...", "object":"chat.completion.chunk", "choices":[...]}
|
||||
data: {"id":"...", "object":"chat.completion.chunk", "choices":[...]}
|
||||
...
|
||||
data: [DONE]
|
||||
```
|
||||
|
||||
**Status Codes**:
|
||||
- `200` - OK
|
||||
- `400` - Bad request (missing/invalid model, invalid JSON, etc.)
|
||||
- `500` - Internal server error (upstream issue)
|
||||
|
||||
**Supported Models**:
|
||||
- `reasoning` - Reasoning model
|
||||
- `ornith:35b` - Ornith 35B model
|
||||
- `qwen2.5:3b-instruct` - Qwen 2.5 3B model
|
||||
|
||||
**Examples**:
|
||||
|
||||
### Basic Chat
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "What is the capital of France?"
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### Chat with Tool Calling
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "What is the weather in San Francisco?"
|
||||
}
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get the weather for a location",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": {
|
||||
"type": "string",
|
||||
"description": "City name"
|
||||
},
|
||||
"unit": {
|
||||
"type": "string",
|
||||
"enum": ["celsius", "fahrenheit"]
|
||||
}
|
||||
},
|
||||
"required": ["location"]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### Streaming Chat
|
||||
|
||||
```bash
|
||||
curl -N -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Count from 1 to 3"
|
||||
}
|
||||
],
|
||||
"stream": true
|
||||
}'
|
||||
```
|
||||
|
||||
### Multi-turn Conversation with Tool Results
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "What is the weather?"
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_123",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\": \"San Francisco\"}"
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"content": "{\"temperature\": 22, \"condition\": \"sunny\"}"
|
||||
}
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get weather",
|
||||
"parameters": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## POST /v1/embeddings
|
||||
|
||||
Generate embeddings for text input.
|
||||
|
||||
**Method**: POST
|
||||
|
||||
**Path**: `/v1/embeddings`
|
||||
|
||||
**Authentication**: None required
|
||||
|
||||
**Request Headers**:
|
||||
```
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**Request Body Schema**:
|
||||
```json
|
||||
{
|
||||
"model": "string (required)",
|
||||
"input": "string | array of strings (required)",
|
||||
"encoding_format": "float | base64 (optional)"
|
||||
}
|
||||
```
|
||||
|
||||
**Response Schema**:
|
||||
```json
|
||||
{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"object": "embedding",
|
||||
"embedding": [0.1, 0.2, ...],
|
||||
"index": "integer"
|
||||
}
|
||||
],
|
||||
"model": "string",
|
||||
"usage": {
|
||||
"prompt_tokens": "integer",
|
||||
"total_tokens": "integer"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Status Codes**:
|
||||
- `200` - OK
|
||||
- `400` - Bad request (missing/invalid model, etc.)
|
||||
- `500` - Internal server error
|
||||
|
||||
**Supported Models**:
|
||||
- `nomic-ai/nomic-embed-text-v2-moe` - Embedding model
|
||||
|
||||
**Examples**:
|
||||
|
||||
### Single Input
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": "The quick brown fox"
|
||||
}'
|
||||
```
|
||||
|
||||
### Multiple Inputs
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": [
|
||||
"Document 1 text",
|
||||
"Document 2 text",
|
||||
"Document 3 text"
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## POST /v1/rerank
|
||||
|
||||
Rerank documents based on relevance to a query.
|
||||
|
||||
**Method**: POST
|
||||
|
||||
**Path**: `/v1/rerank`
|
||||
|
||||
**Authentication**: None required
|
||||
|
||||
**Request Headers**:
|
||||
```
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
**Request Body Schema**:
|
||||
```json
|
||||
{
|
||||
"model": "string (required)",
|
||||
"query": "string (required)",
|
||||
"texts": ["string"],
|
||||
"top_k": "integer (optional)",
|
||||
"return_documents": "boolean (optional)"
|
||||
}
|
||||
```
|
||||
|
||||
**Response Schema**:
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"index": "integer",
|
||||
"score": "float (0-1)",
|
||||
"text": "string (optional)"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Status Codes**:
|
||||
- `200` - OK
|
||||
- `400` - Bad request (missing/invalid model, etc.)
|
||||
- `500` - Internal server error
|
||||
|
||||
**Supported Models**:
|
||||
- `BAAI/bge-reranker-base` - BGE reranker model
|
||||
|
||||
**Note**: The gateway rewrites the path from `/v1/rerank` to `/rerank` on the upstream.
|
||||
|
||||
**Examples**:
|
||||
|
||||
### Basic Reranking
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/rerank \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "BAAI/bge-reranker-base",
|
||||
"query": "What is machine learning?",
|
||||
"texts": [
|
||||
"Machine learning is a type of artificial intelligence",
|
||||
"Dogs are animals",
|
||||
"Deep learning is a subset of machine learning",
|
||||
"Python is a programming language"
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
### With Top-K Parameter
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/rerank \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "BAAI/bge-reranker-base",
|
||||
"query": "best practices",
|
||||
"texts": [
|
||||
"Follow code style guidelines",
|
||||
"Write unit tests",
|
||||
"Use meaningful variable names",
|
||||
"Eat healthy food"
|
||||
],
|
||||
"top_k": 2
|
||||
}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Error Response Format
|
||||
|
||||
The gateway returns RFC 9457 Problem Details for client errors (4xx):
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "https://api.example.com/problems/error-type",
|
||||
"title": "Human-readable error title",
|
||||
"status": 400,
|
||||
"detail": "Detailed explanation of what went wrong",
|
||||
"valid_models": ["model1", "model2"] // Only for model-related errors
|
||||
}
|
||||
```
|
||||
|
||||
### Error Types
|
||||
|
||||
#### Unknown Model Error
|
||||
|
||||
**Status**: `400 Bad Request`
|
||||
|
||||
**Trigger**: Model name not in registry
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"type": "https://api.example.com/problems/unknown-model",
|
||||
"title": "Unknown Model",
|
||||
"status": 400,
|
||||
"detail": "Model \"gpt-4\" is not available. See valid_models for available options.",
|
||||
"valid_models": ["reasoning", "ornith:35b", "qwen2.5:3b-instruct", "nomic-ai/nomic-embed-text-v2-moe", "BAAI/bge-reranker-base"]
|
||||
}
|
||||
```
|
||||
|
||||
**Example**:
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"gpt-4","messages":[]}'
|
||||
```
|
||||
|
||||
#### Missing Model Field
|
||||
|
||||
**Status**: `400 Bad Request`
|
||||
|
||||
**Trigger**: No `model` field in request body
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"type": "https://api.example.com/problems/missing-model",
|
||||
"title": "Missing Model",
|
||||
"status": 400,
|
||||
"detail": "The 'model' field is required and must be a non-empty string",
|
||||
"valid_models": [...]
|
||||
}
|
||||
```
|
||||
|
||||
**Example**:
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"messages":[]}'
|
||||
```
|
||||
|
||||
#### Invalid JSON
|
||||
|
||||
**Status**: `400 Bad Request`
|
||||
|
||||
**Trigger**: Request body is not valid JSON
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"type": "https://api.example.com/problems/invalid-request-body",
|
||||
"title": "Invalid Request Body",
|
||||
"status": 400,
|
||||
"detail": "request body is not valid JSON"
|
||||
}
|
||||
```
|
||||
|
||||
**Example**:
|
||||
```bash
|
||||
curl -X POST https://api.riotpiao.com/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d 'not json'
|
||||
```
|
||||
|
||||
#### Upstream Error
|
||||
|
||||
**Status**: `5xx` (from upstream)
|
||||
|
||||
**Trigger**: Upstream service error
|
||||
|
||||
**Response**: Forwarded from upstream (unmodified)
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Test Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
GATEWAY="https://api.riotpiao.com"
|
||||
|
||||
echo "=== Testing Gateway API ==="
|
||||
echo ""
|
||||
|
||||
# Test 1: Health checks
|
||||
echo "1. Health checks"
|
||||
curl -s "$GATEWAY/healthz" | jq .
|
||||
curl -s "$GATEWAY/readyz" | jq .
|
||||
echo ""
|
||||
|
||||
# Test 2: List models
|
||||
echo "2. List models"
|
||||
curl -s "$GATEWAY/v1/models" | jq '.data[] | .id'
|
||||
echo ""
|
||||
|
||||
# Test 3: Chat with reasoning model
|
||||
echo "3. Chat with reasoning model"
|
||||
curl -s -X POST "$GATEWAY/v1/chat/completions" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "What is 2+2?"}]
|
||||
}' | jq '.choices[0].message.content'
|
||||
echo ""
|
||||
|
||||
# Test 4: Unknown model (should be 400)
|
||||
echo "4. Unknown model (should be 400)"
|
||||
curl -s -X POST "$GATEWAY/v1/chat/completions" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"gpt-4","messages":[]}' | jq '{status: .status, title: .title}'
|
||||
echo ""
|
||||
|
||||
# Test 5: Embeddings
|
||||
echo "5. Embeddings"
|
||||
curl -s -X POST "$GATEWAY/v1/embeddings" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": "hello world"
|
||||
}' | jq '.data | length'
|
||||
echo ""
|
||||
|
||||
# Test 6: Rerank
|
||||
echo "6. Rerank"
|
||||
curl -s -X POST "$GATEWAY/v1/rerank" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "BAAI/bge-reranker-base",
|
||||
"query": "test",
|
||||
"texts": ["a", "b"]
|
||||
}' | jq '.results | length'
|
||||
echo ""
|
||||
|
||||
# Test 7: Streaming
|
||||
echo "7. Streaming (showing first 5 chunks)"
|
||||
curl -s -N -X POST "$GATEWAY/v1/chat/completions" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
"stream": true
|
||||
}' | head -10
|
||||
echo ""
|
||||
|
||||
echo "=== All tests completed ==="
|
||||
```
|
||||
|
||||
### Python Client Example
|
||||
|
||||
```python
|
||||
import requests
|
||||
import json
|
||||
|
||||
GATEWAY = "https://api.riotpiao.com"
|
||||
|
||||
# Get models
|
||||
response = requests.get(f"{GATEWAY}/v1/models")
|
||||
models = response.json()
|
||||
print(f"Available models: {[m['id'] for m in models['data']]}")
|
||||
|
||||
# Chat completion
|
||||
response = requests.post(
|
||||
f"{GATEWAY}/v1/chat/completions",
|
||||
json={
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is machine learning?"}
|
||||
]
|
||||
}
|
||||
)
|
||||
message = response.json()
|
||||
print(f"Response: {message['choices'][0]['message']['content']}")
|
||||
|
||||
# Chat with tools
|
||||
response = requests.post(
|
||||
f"{GATEWAY}/v1/chat/completions",
|
||||
json={
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Get the weather"}
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get weather",
|
||||
"parameters": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
result = response.json()
|
||||
if "tool_calls" in result["choices"][0]["message"]:
|
||||
print(f"Tool calls: {result['choices'][0]['message']['tool_calls']}")
|
||||
|
||||
# Streaming
|
||||
response = requests.post(
|
||||
f"{GATEWAY}/v1/chat/completions",
|
||||
json={
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Count to 3"}
|
||||
],
|
||||
"stream": True
|
||||
},
|
||||
stream=True
|
||||
)
|
||||
for line in response.iter_lines():
|
||||
if line:
|
||||
print(line)
|
||||
|
||||
# Embeddings
|
||||
response = requests.post(
|
||||
f"{GATEWAY}/v1/embeddings",
|
||||
json={
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": "hello world"
|
||||
}
|
||||
)
|
||||
embeddings = response.json()
|
||||
print(f"Embeddings: {embeddings['data'][0]['embedding'][:5]}")
|
||||
|
||||
# Rerank
|
||||
response = requests.post(
|
||||
f"{GATEWAY}/v1/rerank",
|
||||
json={
|
||||
"model": "BAAI/bge-reranker-base",
|
||||
"query": "ML",
|
||||
"texts": ["machine learning", "python", "deep learning"]
|
||||
}
|
||||
)
|
||||
results = response.json()
|
||||
print(f"Rerank results: {results['results']}")
|
||||
```
|
||||
|
||||
### JavaScript/TypeScript Client Example
|
||||
|
||||
```typescript
|
||||
const GATEWAY = "https://api.riotpiao.com";
|
||||
|
||||
// Get models
|
||||
async function getModels() {
|
||||
const response = await fetch(`${GATEWAY}/v1/models`);
|
||||
const data = await response.json();
|
||||
return data.data.map((m: any) => m.id);
|
||||
}
|
||||
|
||||
// Chat completion
|
||||
async function chat(model: string, message: string) {
|
||||
const response = await fetch(`${GATEWAY}/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model,
|
||||
messages: [{ role: "user", content: message }],
|
||||
}),
|
||||
});
|
||||
const data = await response.json();
|
||||
return data.choices[0].message.content;
|
||||
}
|
||||
|
||||
// Chat with streaming
|
||||
async function chatStream(model: string, message: string) {
|
||||
const response = await fetch(`${GATEWAY}/v1/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model,
|
||||
messages: [{ role: "user", content: message }],
|
||||
stream: true,
|
||||
}),
|
||||
});
|
||||
|
||||
const reader = response.body!.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
const chunk = decoder.decode(value);
|
||||
const lines = chunk.split("\n");
|
||||
for (const line of lines) {
|
||||
if (line.startsWith("data: ")) {
|
||||
const data = JSON.parse(line.slice(6));
|
||||
if (data.choices[0].delta?.content) {
|
||||
console.log(data.choices[0].delta.content);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Embeddings
|
||||
async function embed(model: string, input: string[]) {
|
||||
const response = await fetch(`${GATEWAY}/v1/embeddings`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model, input }),
|
||||
});
|
||||
const data = await response.json();
|
||||
return data.data;
|
||||
}
|
||||
|
||||
// Rerank
|
||||
async function rerank(
|
||||
model: string,
|
||||
query: string,
|
||||
texts: string[]
|
||||
) {
|
||||
const response = await fetch(`${GATEWAY}/v1/rerank`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ model, query, texts }),
|
||||
});
|
||||
const data = await response.json();
|
||||
return data.results;
|
||||
}
|
||||
|
||||
// Usage
|
||||
(async () => {
|
||||
const models = await getModels();
|
||||
console.log("Models:", models);
|
||||
|
||||
const response = await chat("reasoning", "What is AI?");
|
||||
console.log("Response:", response);
|
||||
|
||||
await chatStream("reasoning", "Count to 3");
|
||||
|
||||
const embeddings = await embed("nomic-ai/nomic-embed-text-v2-moe", [
|
||||
"hello",
|
||||
]);
|
||||
console.log("Embeddings:", embeddings);
|
||||
|
||||
const rerankResults = await rerank("BAAI/bge-reranker-base", "ML", [
|
||||
"machine learning",
|
||||
"python",
|
||||
]);
|
||||
console.log("Rerank:", rerankResults);
|
||||
})();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
Currently, no rate limiting is enforced. This will be added in Phase 4.
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
Currently, no authentication is enforced. Bearer token support will be added in Phase 3.
|
||||
|
||||
---
|
||||
|
||||
## Timeouts
|
||||
|
||||
Default timeouts per route:
|
||||
- **Connect**: 10s
|
||||
- **Read**: 1h (for streaming)
|
||||
- **Write**: 1h
|
||||
|
||||
These are configured per model upstream.
|
||||
|
||||
---
|
||||
|
||||
## Body Size Limits
|
||||
|
||||
- **Default**: 100MB
|
||||
- **Per-route**: Configurable
|
||||
|
||||
Requests exceeding the limit return `413 Request Entity Too Large`.
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- Check gateway logs: `kubectl -n api logs deployment/homelab-frontend`
|
||||
- Check health: `curl https://api.riotpiao.com/healthz`
|
||||
- Verify config: `curl https://api.riotpiao.com/v1/models`
|
||||
@@ -0,0 +1,560 @@
|
||||
# API Testing Guide
|
||||
|
||||
Quick reference for testing the homelab-frontend gateway API.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
# Set base URL
|
||||
export GATEWAY="https://api.riotpiao.com"
|
||||
|
||||
# Or for local testing
|
||||
export GATEWAY="http://localhost:8080"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Tests (Copy & Paste)
|
||||
|
||||
### 1. Health Checks ✅
|
||||
|
||||
```bash
|
||||
# Liveness
|
||||
curl $GATEWAY/healthz | jq .
|
||||
|
||||
# Readiness
|
||||
curl $GATEWAY/readyz | jq .
|
||||
```
|
||||
|
||||
**Expected**: Both return `{"status":"..."}` with HTTP 200
|
||||
|
||||
---
|
||||
|
||||
### 2. List Models ✅
|
||||
|
||||
```bash
|
||||
curl $GATEWAY/v1/models | jq '.data[] | .id'
|
||||
```
|
||||
|
||||
**Expected Output**:
|
||||
```
|
||||
"reasoning"
|
||||
"ornith:35b"
|
||||
"qwen2.5:3b-instruct"
|
||||
"nomic-ai/nomic-embed-text-v2-moe"
|
||||
"BAAI/bge-reranker-base"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Chat - Basic ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is 2+2?"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
**Expected**: Model responds with an answer
|
||||
|
||||
---
|
||||
|
||||
### 4. Chat - Ornith Model ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "ornith:35b",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
**Expected**: Routes to ornith model, returns response
|
||||
|
||||
---
|
||||
|
||||
### 5. Chat - Qwen Model ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "qwen2.5:3b-instruct",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hi"}
|
||||
]
|
||||
}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
**Expected**: Routes to qwen model, returns response
|
||||
|
||||
---
|
||||
|
||||
### 6. Chat - Unknown Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "gpt-4-turbo",
|
||||
"messages": []
|
||||
}' | jq '.'
|
||||
```
|
||||
|
||||
**Expected**: HTTP 400 with problem+json:
|
||||
```json
|
||||
{
|
||||
"type": "https://api.example.com/problems/unknown-model",
|
||||
"title": "Unknown Model",
|
||||
"status": 400,
|
||||
"detail": "Model \"gpt-4-turbo\" is not available. See valid_models for available options.",
|
||||
"valid_models": ["reasoning", "ornith:35b", ...]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 7. Chat - Missing Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"messages": [{"role": "user", "content": "test"}]
|
||||
}' | jq '.'
|
||||
```
|
||||
|
||||
**Expected**: HTTP 400 with problem+json (missing model)
|
||||
|
||||
---
|
||||
|
||||
### 8. Chat - Streaming ✅
|
||||
|
||||
```bash
|
||||
curl -N -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "count to 3"}],
|
||||
"stream": true
|
||||
}' | head -20
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- Multiple `data: {...}` lines (SSE chunks)
|
||||
- Final `data: [DONE]`
|
||||
- Chunks arrive incrementally (observable with `-N` flag)
|
||||
|
||||
---
|
||||
|
||||
### 9. Chat - Tool Calling ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the weather in SF?"}
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_weather",
|
||||
"description": "Get weather for a location",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": {"type": "string"}
|
||||
},
|
||||
"required": ["location"]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}' | jq '.choices[0].message.tool_calls'
|
||||
```
|
||||
|
||||
**Expected**: Array of tool calls (if model decides to call them), or null (if not)
|
||||
|
||||
---
|
||||
|
||||
### 10. Embeddings ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": "hello world"
|
||||
}' | jq '.data | length'
|
||||
```
|
||||
|
||||
**Expected**: `1` (one embedding vector)
|
||||
|
||||
---
|
||||
|
||||
### 11. Embeddings - Multiple ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nomic-ai/nomic-embed-text-v2-moe",
|
||||
"input": ["text 1", "text 2", "text 3"]
|
||||
}' | jq '.data | length'
|
||||
```
|
||||
|
||||
**Expected**: `3` (three embedding vectors)
|
||||
|
||||
---
|
||||
|
||||
### 12. Embeddings - Unknown Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/embeddings \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "unknown-embed",
|
||||
"input": "test"
|
||||
}' | jq '.status'
|
||||
```
|
||||
|
||||
**Expected**: `400` (client error)
|
||||
|
||||
---
|
||||
|
||||
### 13. Rerank ✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/rerank \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "BAAI/bge-reranker-base",
|
||||
"query": "machine learning",
|
||||
"texts": [
|
||||
"Machine learning is AI",
|
||||
"Python is a language",
|
||||
"Deep learning is ML"
|
||||
]
|
||||
}' | jq '.results'
|
||||
```
|
||||
|
||||
**Expected**: Array of ranked results with scores:
|
||||
```json
|
||||
[
|
||||
{"index": 0, "score": 0.95},
|
||||
{"index": 2, "score": 0.85},
|
||||
{"index": 1, "score": 0.15}
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 14. Rerank - Unknown Model (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/rerank \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "unknown-rerank",
|
||||
"query": "test",
|
||||
"texts": ["a"]
|
||||
}' | jq '.status'
|
||||
```
|
||||
|
||||
**Expected**: `400` (client error)
|
||||
|
||||
---
|
||||
|
||||
### 15. Invalid JSON (Should Error) ❌→✅
|
||||
|
||||
```bash
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d 'not json' | jq '.title'
|
||||
```
|
||||
|
||||
**Expected**: `"Invalid Request Body"` (HTTP 400)
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
Complete this checklist to verify all endpoints:
|
||||
|
||||
### Health Endpoints
|
||||
- [ ] GET /healthz → 200, `{"status":"alive"}`
|
||||
- [ ] GET /readyz → 200, `{"status":"ready"}`
|
||||
|
||||
### Model Discovery
|
||||
- [ ] GET /v1/models → 200, returns all 5 models
|
||||
- [ ] All advertised models can be called (none 400)
|
||||
|
||||
### Chat Completions
|
||||
- [ ] POST /v1/chat/completions (reasoning) → 200, response
|
||||
- [ ] POST /v1/chat/completions (ornith:35b) → 200, response
|
||||
- [ ] POST /v1/chat/completions (qwen2.5:3b-instruct) → 200, response
|
||||
- [ ] POST /v1/chat/completions (unknown model) → 400, problem+json
|
||||
- [ ] POST /v1/chat/completions (missing model) → 400, problem+json
|
||||
- [ ] POST /v1/chat/completions (invalid JSON) → 400, problem+json
|
||||
- [ ] POST /v1/chat/completions (streaming) → 200, SSE chunks
|
||||
- [ ] POST /v1/chat/completions (with tools) → 200, tool_calls present/absent
|
||||
|
||||
### Embeddings
|
||||
- [ ] POST /v1/embeddings (single input) → 200, embedding
|
||||
- [ ] POST /v1/embeddings (multiple inputs) → 200, embeddings array
|
||||
- [ ] POST /v1/embeddings (unknown model) → 400, problem+json
|
||||
|
||||
### Reranking
|
||||
- [ ] POST /v1/rerank → 200, ranked results
|
||||
- [ ] POST /v1/rerank (unknown model) → 400, problem+json
|
||||
- [ ] Verify path is rewritten to /rerank on upstream
|
||||
|
||||
### Error Handling
|
||||
- [ ] Unknown model lists valid_models
|
||||
- [ ] Error responses are problem+json
|
||||
- [ ] No 5xx for client errors (validation errors)
|
||||
- [ ] Upstream errors pass through
|
||||
|
||||
### Streaming
|
||||
- [ ] Chunks arrive incrementally
|
||||
- [ ] Final `[DONE]` sentinel present
|
||||
- [ ] Works for chat completions
|
||||
|
||||
### Tool Calling
|
||||
- [ ] Tool definitions forward to upstream
|
||||
- [ ] Tool calls in response
|
||||
- [ ] Multi-turn with tool results
|
||||
- [ ] Parallel tool calls
|
||||
- [ ] Complex nested arguments preserved
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### 404 Responses
|
||||
|
||||
**Symptom**: All endpoints return `"not found"`
|
||||
|
||||
**Cause**: ConfigMap with models/routes not deployed
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
kubectl -n api create configmap homelab-frontend-config \
|
||||
--from-file=config.yaml=k8s/configmap.yaml
|
||||
kubectl -n api rollout restart deployment/homelab-frontend
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 503 (Not Ready)
|
||||
|
||||
**Symptom**: `/readyz` returns 503
|
||||
|
||||
**Cause**: Configuration not loaded or JWKS fetch failed
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check logs
|
||||
kubectl -n api logs deployment/homelab-frontend
|
||||
|
||||
# Check config
|
||||
kubectl -n api get configmap homelab-frontend-config
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Connection Refused
|
||||
|
||||
**Symptom**: `Connection refused` or `Temporary failure in name resolution`
|
||||
|
||||
**Cause**:
|
||||
- Gateway not running
|
||||
- Wrong URL/hostname
|
||||
- Network issue
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Verify gateway is running
|
||||
kubectl -n api get pods -l app=homelab-frontend
|
||||
|
||||
# Check service
|
||||
kubectl -n api get svc homelab-frontend
|
||||
|
||||
# Verify ingress
|
||||
kubectl -n api get ingress api
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Upstream Connection Errors
|
||||
|
||||
**Symptom**: `502 Bad Gateway` or `connection refused to upstream`
|
||||
|
||||
**Cause**: Model upstream service not reachable
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Check upstreams are running
|
||||
kubectl -n llm-serving get pods
|
||||
|
||||
# Verify addresses in ConfigMap
|
||||
kubectl -n api get configmap homelab-frontend-config -o yaml
|
||||
|
||||
# Test connectivity from gateway pod
|
||||
kubectl -n api exec deployment/homelab-frontend -- \
|
||||
curl -s reasoning-predictor.llm-serving:80/healthz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Streaming Doesn't Work
|
||||
|
||||
**Symptom**: Chunks arrive all at once (buffered) instead of incrementally
|
||||
|
||||
**Cause**: nginx buffering or client not using `-N` flag
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Use -N flag
|
||||
curl -N https://api.riotpiao.com/v1/chat/completions ...
|
||||
|
||||
# Verify nginx has buffering disabled
|
||||
# Should have: proxy-buffering: off in Ingress annotations
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Testing
|
||||
|
||||
### Load Test (Simple)
|
||||
|
||||
```bash
|
||||
# Send 10 requests in parallel
|
||||
for i in {1..10}; do
|
||||
curl -X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"reasoning","messages":[{"role":"user","content":"Hi"}]}' &
|
||||
done
|
||||
wait
|
||||
|
||||
echo "Completed 10 requests"
|
||||
```
|
||||
|
||||
### Concurrency Test
|
||||
|
||||
```bash
|
||||
# Use Apache Bench (if installed)
|
||||
ab -n 100 -c 10 \
|
||||
-p request.json \
|
||||
-T application/json \
|
||||
$GATEWAY/v1/chat/completions
|
||||
|
||||
# Create request.json:
|
||||
# {"model":"reasoning","messages":[{"role":"user","content":"test"}]}
|
||||
```
|
||||
|
||||
### Latency Test
|
||||
|
||||
```bash
|
||||
# Measure response time
|
||||
curl -w "\nTotal time: %{time_total}s\n" \
|
||||
-X POST $GATEWAY/v1/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "reasoning",
|
||||
"messages": [{"role": "user", "content": "What is AI?"}]
|
||||
}' > /dev/null
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Testing
|
||||
|
||||
### Test with Python
|
||||
|
||||
```bash
|
||||
pip install requests
|
||||
|
||||
cat > test_api.py << 'EOF'
|
||||
import requests
|
||||
import json
|
||||
|
||||
gateway = "https://api.riotpiao.com"
|
||||
|
||||
# Test health
|
||||
r = requests.get(f"{gateway}/healthz")
|
||||
assert r.status_code == 200
|
||||
print("✓ Health check passed")
|
||||
|
||||
# Test models
|
||||
r = requests.get(f"{gateway}/v1/models")
|
||||
assert r.status_code == 200
|
||||
models = [m['id'] for m in r.json()['data']]
|
||||
print(f"✓ Models: {models}")
|
||||
|
||||
# Test chat
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/chat/completions",
|
||||
json={"model": "reasoning", "messages": [{"role": "user", "content": "Hi"}]}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
print("✓ Chat works")
|
||||
|
||||
# Test unknown model error
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/chat/completions",
|
||||
json={"model": "gpt-4", "messages": []}
|
||||
)
|
||||
assert r.status_code == 400
|
||||
assert "unknown" in r.json()['detail'].lower()
|
||||
print("✓ Unknown model error correct")
|
||||
|
||||
# Test embeddings
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/embeddings",
|
||||
json={"model": "nomic-ai/nomic-embed-text-v2-moe", "input": "test"}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
print("✓ Embeddings work")
|
||||
|
||||
# Test rerank
|
||||
r = requests.post(
|
||||
f"{gateway}/v1/rerank",
|
||||
json={"model": "BAAI/bge-reranker-base", "query": "test", "texts": ["a", "b"]}
|
||||
)
|
||||
assert r.status_code == 200
|
||||
print("✓ Reranking works")
|
||||
|
||||
print("\n✅ All tests passed!")
|
||||
EOF
|
||||
|
||||
python test_api.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | Tests | Expected |
|
||||
|----------|-------|----------|
|
||||
| Health | 2 | ✅ Both 200 |
|
||||
| Models | 1 | ✅ 5 models listed |
|
||||
| Chat | 8 | ✅ 6 success + 2 error |
|
||||
| Embeddings | 3 | ✅ 2 success + 1 error |
|
||||
| Rerank | 2 | ✅ 1 success + 1 error |
|
||||
| Streaming | 1 | ✅ Incremental chunks |
|
||||
| Tools | 1 | ✅ Tool calls present |
|
||||
| **TOTAL** | **18+** | **✅ ALL PASS** |
|
||||
|
||||
Once all tests pass, the gateway is production-ready! 🚀
|
||||
Reference in New Issue
Block a user