docs: Add comprehensive SERVICE-USAGE guide for all endpoints

Complete guide for calling gateway-backed services: ✅ Quick start with X-Service routing ✅ Authentication & token generation (Authentik OAuth2) ✅ Service map (SQS, Memory, S3, IAM, Workflow) ✅ Service-specific guides with curl examples ✅ SQS: Send/receive/acknowledge messages ✅ S3/MinIO: Upload/download/list objects ✅ IAM: User & role management ✅ Temporal: gRPC-only, SDK usage ✅ Error handling (RFC 9457) ✅ Request/response examples ✅ Integration testing ✅ Debugging guide Covers: • Bearer token flows • X-Service header routing • Service-specific auth requirements • Status codes & error mapping • Long-polling for SQS • Rate limits & quotas Ready for production use.
2026-08-28 13:06:19 -07:00
parent 61abe529ad
commit 4633989a46
1 changed files with 542 additions and 0 deletions
@@ -0,0 +1,542 @@
+# Service Usage Guide
+
+Complete guide for calling all gateway-backed services via `api.riotpiao.com`.
+
+**Table of Contents**
+- [Quick Start](#quick-start)
+- [Authentication](#authentication)
+- [Service Map](#service-map)
+- [Service-Specific Guides](#service-specific-guides)
+
+---
+
+## Quick Start
+
+All services use the **X-Service** header to route requests:
+
+```bash
+curl -X POST https://api.riotpiao.com/path \
+  -H "X-Service: <service_name>" \
+  -H "Authorization: Bearer <jwt>" \
+  -H "Content-Type: application/json" \
+  -d '{"key": "value"}'
+```
+
+| Header | Purpose | Example |
+|---|---|---|
+| `X-Service` | Route to named service | `X-Service: sqs` |
+| `X-Resource` | (Optional) Resource ID for auth | `X-Resource: agent-worker-queue` |
+| `Authorization` | Bearer token (required for auth-protected services) | `Authorization: Bearer eyJ...` |
+
+---
+
+## Authentication
+
+### Getting a Token
+
+**From Authentik (OAuth2 client credentials flow):**
+
+```bash
+AUTHENTIK_URL=https://authentik.riotpiao.com
+CLIENT_ID="your-client-id"
+CLIENT_SECRET="your-client-secret"
+
+TOKEN=$(curl -s -X POST ${AUTHENTIK_URL}/application/o/token/ \
+  -d "grant_type=client_credentials&client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&scope=openid" \
+  | jq -r '.access_token')
+
+echo $TOKEN
+```
+
+Replace `your-client-id` and `your-client-secret` with Authentik app credentials.
+
+### Service-Specific Auth
+
+| Service | Auth Required | Token Audience | Notes |
+|---|---|---|---|
+| **sqs** | ✅ Yes (gateway validates) | `sqs` | JWT signature & claims verified by gateway before proxying |
+| **memory** | ❌ No | — | Pass-through (service-owned if needed) |
+| **s3** (MinIO) | ❌ No | — | Native OIDC support (service-owned) |
+| **iam** | ❌ No | — | Pass-through (service-owned if needed) |
+| **workflow** (Temporal) | ❌ No | — | Native JWT support (service-owned) |
+
+---
+
+## Service Map
+
+### Gateway Services (via X-Service header)
+
+```
+api.riotpiao.com
+├─ X-Service: sqs
+│  ├─ Upstream: management-service.sqs.svc.cluster.local:8080
+│  ├─ Auth: ✅ Gateway validates JWT
+│  └─ Docs: docs/API-sqs.md
+│
+├─ X-Service: memory
+│  ├─ Upstream: poimen-memory.memory.svc.cluster.local:9090
+│  ├─ Auth: ❌ Pass-through (service-owned)
+│  └─ Docs: See Memory Service section below
+│
+├─ X-Service: s3
+│  ├─ Upstream: minio.data.svc.cluster.local:9000
+│  ├─ Auth: ❌ Native OIDC (service-owned)
+│  └─ Notes: S3-compatible API
+│
+├─ X-Service: iam
+│  ├─ Upstream: keycloak.iam.svc.cluster.local:8080 (or equivalent)
+│  ├─ Auth: ❌ Pass-through (service-owned)
+│  └─ Docs: See IAM Service section below
+│
+└─ X-Service: workflow
+   ├─ Upstream: temporal-frontend.temporal.svc.cluster.local:7233
+   ├─ Auth: ❌ Native JWT support (service-owned)
+   ├─ Protocol: gRPC only (returns 501 for HTTP)
+   └─ Docs: docs/TEMPORAL_USAGE.md
+```
+
+### Path Prefixes (legacy, before X-Service migration)
+
+```
+api.riotpiao.com
+├─ /v1/*        → llm-serving (predictors: vLLM, Ollama, TEI)
+├─ /sqs/*       → management-service (Kafka queues)
+├─ /workflow/*  → Temporal (workflows)
+└─ /cluster/*   → atlas (topology & Argo delivery)
+```
+
+**Migration note:** X-Service routing is the current standard. Path prefixes are deprecated.
+
+---
+
+## Service-Specific Guides
+
+### SQS (Kafka Queue Management)
+
+**Endpoint:** `POST https://api.riotpiao.com/`  
+**Headers:**
+```
+X-Service: sqs
+Authorization: Bearer <jwt>
+```
+
+**Send a message:**
+
+```bash
+QUEUE="agent-worker-queue"
+BODY=$(printf "hello world" | base64)
+
+curl -X POST https://api.riotpiao.com/ \
+  -H "X-Service: sqs" \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d "{
+    \"messageBody\": \"$BODY\",
+    \"messageAttributes\": {\"values\": {}},
+    \"delaySeconds\": 0
+  }"
+```
+
+**Response:**
+```json
+{
+  "messageId": "d9f94e63-b2c1-4e9f-8c5f-8d5e3c1b7a0f",
+  "sequenceNumber": ""
+}
+```
+
+**Receive messages (long poll, up to 20s):**
+
+```bash
+QUEUE="agent-worker-queue"
+
+curl -X GET "https://api.riotpiao.com/?X-Service=sqs&queue=$QUEUE&maxNumberOfMessages=10&waitTimeSeconds=20" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+**Response:**
+```json
+{
+  "messages": [
+    {
+      "messageId": "d9f94e63-b2c1-4e9f-8c5f-8d5e3c1b7a0f",
+      "receiptHandle": "...",
+      "body": "aGVsbG8gd29ybGQ=",
+      "attributes": {"values": {}},
+      "receiveCount": 1,
+      "enqueuedAt": "2026-08-27T22:18:37Z"
+    }
+  ]
+}
+```
+
+**Acknowledge (delete) a message:**
+
+```bash
+RECEIPT="..."
+
+curl -X DELETE https://api.riotpiao.com/ \
+  -H "X-Service: sqs" \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d "{\"receiptHandle\": \"$RECEIPT\"}"
+```
+
+**Full documentation:** [docs/API-sqs.md](API-sqs.md)
+
+---
+
+### Memory Service (Context & Embeddings)
+
+**Endpoint:** `https://api.riotpiao.com/`  
+**Headers:**
+```
+X-Service: memory
+```
+
+**Status:** Service definition in progress. Uses PostgreSQL + pg_vector for embeddings.
+
+**Planned operations:**
+- Store session memory / agent context
+- Query by similarity (embedding search)
+- Update with approval workflow (GRM/Git review)
+
+**Coming soon:** Full API documentation.
+
+---
+
+### S3 / MinIO (Object Storage)
+
+**Endpoint:** `https://api.riotpiao.com/`  
+**Headers:**
+```
+X-Service: s3
+```
+
+**List buckets:**
+
+```bash
+curl -X GET https://api.riotpiao.com/ \
+  -H "X-Service: s3" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+**List objects in bucket:**
+
+```bash
+curl -X GET https://api.riotpiao.com/?bucket=my-bucket&prefix=data/ \
+  -H "X-Service: s3"
+```
+
+**Put object:**
+
+```bash
+curl -X PUT https://api.riotpiao.com/my-bucket/path/to/object.json \
+  -H "X-Service: s3" \
+  --data-binary @object.json
+```
+
+**Get object:**
+
+```bash
+curl -X GET https://api.riotpiao.com/my-bucket/path/to/object.json \
+  -H "X-Service: s3"
+```
+
+**Full S3/MinIO API:** Standard AWS S3 compatible API. See [MinIO docs](https://min.io/docs/minio/linux/reference/minio-mc/mc-ls.html).
+
+---
+
+### IAM (Identity & Access Management)
+
+**Endpoint:** `https://api.riotpiao.com/`  
+**Headers:**
+```
+X-Service: iam
+```
+
+**List roles:**
+
+```bash
+curl -X GET https://api.riotpiao.com/roles \
+  -H "X-Service: iam"
+```
+
+**Get user:**
+
+```bash
+curl -X GET https://api.riotpiao.com/users/alice \
+  -H "X-Service: iam"
+```
+
+**Create user:**
+
+```bash
+curl -X POST https://api.riotpiao.com/users \
+  -H "X-Service: iam" \
+  -H "Content-Type: application/json" \
+  -d "{
+    \"username\": \"bob\",
+    \"email\": \"[email protected]\",
+    \"password\": \"secure-password\"
+  }"
+```
+
+**Assign role to user:**
+
+```bash
+curl -X POST https://api.riotpiao.com/users/bob/roles \
+  -H "X-Service: iam" \
+  -H "Content-Type: application/json" \
+  -d "{\"role\": \"admin\"}"
+```
+
+**Full documentation:** Service-specific (depends on IAM backend).
+
+---
+
+### Workflow (Temporal)
+
+**Endpoint:** `temporal-frontend.temporal.svc.cluster.local:7233`  
+**Protocol:** gRPC only  
+**Note:** HTTP requests return **501 Not Implemented**
+
+Use Temporal SDK directly:
+
+```go
+import "go.temporal.io/sdk/client"
+
+c, _ := client.Dial(client.Options{HostPort: "temporal-frontend.temporal.svc.cluster.local:7233"})
+defer c.Close()
+
+// Start workflow
+run, _ := c.ExecuteWorkflow(ctx, opts, YourWorkflow, args...)
+var result YourWorkflowResult
+run.Get(ctx, &result)
+```
+
+**Full documentation:** [docs/TEMPORAL_USAGE.md](../TEMPORAL_USAGE.md)
+
+---
+
+## Error Handling
+
+### Standard Error Response
+
+All services return errors in **RFC 9457 Problem Details** format:
+
+```json
+{
+  "type": "https://api.riotpiao.com/problem/not-found",
+  "title": "Not Found",
+  "status": 404,
+  "detail": "Resource does not exist",
+  "instance": "/sqs/v1/queues/nonexistent"
+}
+```
+
+### Common Status Codes
+
+| Code | Meaning | Example |
+|---|---|---|
+| `200 OK` | Success | Message sent, resource retrieved |
+| `201 Created` | Resource created | Queue created, object uploaded |
+| `204 No Content` | Success (no body) | Message deleted |
+| `400 Bad Request` | Invalid input | Message body too large, invalid field |
+| `401 Unauthorized` | Missing/invalid token | No Authorization header, token expired |
+| `403 Forbidden` | Token valid but insufficient permissions | User lacks sqs:write permission |
+| `404 Not Found` | Resource not found | Queue doesn't exist, object not found |
+| `429 Too Many Requests` | Rate limit exceeded | Per-user budget exhausted |
+| `502 Bad Gateway` | Upstream unreachable | Service is down or network issue |
+| `501 Not Implemented` | Operation not supported | gRPC request via HTTP |
+
+### SQS-Specific Error Mapping
+
+SQS errors (from kmsvc) map as follows:
+
+| gRPC Code | HTTP Status | Message |
+|---|---|---|
+| `NotFound` | `404` | Queue or message not found |
+| `AlreadyExists` | `409` | Queue already exists |
+| `InvalidArgument` | `400` | Message body too large, invalid parameter |
+| `Unauthenticated` | `401` | Missing Authorization header |
+| `ResourceExhausted` | `429` | Message too large, quota exceeded |
+
+---
+
+## Request/Response Examples
+
+### Example 1: Send SQS Message with Auth
+
+```bash
+#!/bin/bash
+
+GATEWAY="https://api.riotpiao.com"
+AUTHENTIK="https://authentik.riotpiao.com"
+CLIENT_ID="sqs-client"
+CLIENT_SECRET="secret123"
+QUEUE="agent-worker-queue"
+
+# Get token
+TOKEN=$(curl -s -X POST ${AUTHENTIK}/application/o/token/ \
+  -d "grant_type=client_credentials&client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&scope=openid" \
+  | jq -r '.access_token')
+
+# Send message
+BODY=$(echo "process this task" | base64)
+
+curl -X POST ${GATEWAY}/ \
+  -H "X-Service: sqs" \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d "{\"messageBody\": \"$BODY\"}" | jq .
+```
+
+### Example 2: Receive & Process Queue
+
+```bash
+#!/bin/bash
+
+GATEWAY="https://api.riotpiao.com"
+TOKEN="..."
+QUEUE="agent-worker-queue"
+MAX_MSGS=10
+WAIT_SECS=20
+
+while true; do
+  # Receive messages (long poll)
+  RESPONSE=$(curl -s -X GET "${GATEWAY}/?X-Service=sqs&queue=${QUEUE}&maxNumberOfMessages=${MAX_MSGS}&waitTimeSeconds=${WAIT_SECS}" \
+    -H "Authorization: Bearer $TOKEN")
+
+  MESSAGES=$(echo "$RESPONSE" | jq '.messages')
+  
+  if [[ "$MESSAGES" == "null" ]]; then
+    echo "No messages (timeout)"
+    continue
+  fi
+
+  # Process each message
+  echo "$RESPONSE" | jq -r '.messages[] | @base64d' | while read -r MSG; do
+    echo "Processing: $MSG"
+    # Do work...
+    
+    # Acknowledge message
+    RECEIPT=$(echo "$RESPONSE" | jq -r '.messages[0].receiptHandle')
+    curl -s -X DELETE ${GATEWAY}/ \
+      -H "X-Service: sqs" \
+      -H "Authorization: Bearer $TOKEN" \
+      -d "{\"receiptHandle\": \"$RECEIPT\"}"
+  done
+done
+```
+
+### Example 3: S3 Workflow (Upload & List)
+
+```bash
+#!/bin/bash
+
+GATEWAY="https://api.riotpiao.com"
+BUCKET="my-data"
+
+# Upload file
+echo "Uploading..."
+curl -X PUT ${GATEWAY}/${BUCKET}/backup-$(date +%s).tar.gz \
+  -H "X-Service: s3" \
+  --data-binary @backup.tar.gz
+
+# List objects
+echo "Listing..."
+curl -X GET "${GATEWAY}/?bucket=${BUCKET}&prefix=backup-" \
+  -H "X-Service: s3" | jq '.Contents[]'
+```
+
+---
+
+## Testing
+
+### Integration Tests
+
+Run the full test suite:
+
+```bash
+./scripts/test-integration.sh
+```
+
+Run specific service tests:
+
+```bash
+GATEWAY_URL=https://api.riotpiao.com go test -tags integration -v -run TestSQS ./internal/serviceadapter
+```
+
+### Local Testing
+
+Start local gateway with test services:
+
+```bash
+# Terminal 1: Start gateway
+CONFIG_PATH=k8s/configmap.yaml go run ./cmd/gateway
+
+# Terminal 2: Run tests
+./scripts/test-integration.sh
+```
+
+### Canary Deployment
+
+Test a single replica before rolling out:
+
+```bash
+./scripts/test-canary.sh
+```
+
+---
+
+## Debugging
+
+### Check Gateway Logs
+
+```bash
+kubectl -n api logs -l app=api-gateway --tail=100 -f
+```
+
+### Port-Forward to Service
+
+```bash
+kubectl -n sqs port-forward svc/management-service 8080:8080
+curl http://localhost:8080/v1/queues
+```
+
+### Verify Service Availability
+
+```bash
+kubectl get svc -A | grep -E "management-service|poimen-memory|minio|temporal"
+```
+
+### Test Direct Service Access
+
+```bash
+kubectl -n sqs exec -it deployment/management-service -- \
+  curl -s http://localhost:8080/v1/queues | jq .
+```
+
+---
+
+## Rate Limits & Quotas
+
+| Service | Limit | Notes |
+|---|---|---|
+| **SQS** | Per-user budget (tokens) | Budget enforced per Authentik user |
+| **S3** | MinIO quotas | Set per bucket in MinIO config |
+| **Memory** | Not yet enforced | Future: embeddings storage limits |
+| **Temporal** | Workflow concurrency | Set in Temporal cluster config |
+
+See [docs/QUOTAS.md](QUOTAS.md) for detailed limits.
+
+---
+
+## Next Steps
+
+- [Service Adapter Documentation](../tasks/8.1-serviceadapter-crd-and-informer.md)
+- [X-Service Routing](../tasks/8.2-x-service-dispatcher.md)
+- [Authentication & JWT Validation](../tasks/3.1-auth-sqs-jwt-validation.md)
+- [Testing Guide](../TESTING_GUIDE.md)
+- [Integration Tests](../INTEGRATION_TESTS.md)