Files
homelab-frontend/TEMPORAL_USAGE.md
T

1211 lines
22 KiB
Markdown
Raw Normal View History

# Temporal Workflow API Gateway - Usage Guide
## Overview
2026-08-29 21:54:16 -07:00
The API Gateway exposes Temporal workflow operations through a unified `/workflow` REST endpoint. Internally uses gRPC to communicate with Temporal server (port 7233), eliminating need for direct gRPC connections.
**Base URL**: `https://api.riotpiao.com/workflow`
2026-08-29 21:54:16 -07:00
**Architecture**:
```
Client (HTTP REST) → Gateway → gRPC → Temporal (port 7233)
```
## Implementation Status
**Phase 1 (✅ Current):** Workflow and Activity operations via WorkflowService
- START_WORKFLOW, DESCRIBE_WORKFLOW, LIST_WORKFLOWS
- GET_WORKFLOW_HISTORY, SIGNAL_WORKFLOW, QUERY_WORKFLOW
- CANCEL_WORKFLOW, TERMINATE_WORKFLOW, RESET_WORKFLOW, UPDATE_WORKFLOW
- HEARTBEAT_ACTIVITY, COMPLETE_ACTIVITY, FAIL_ACTIVITY
**Phase 2 (⏳ Pending):** OperatorService operations
- Namespace management, Search attributes, Task queue monitoring, Cluster ops
- Currently return: `{"error": "NOT_IMPLEMENTED", "message": "... requires OperatorService support"}`
---
## Unified REST API Design
### Request Format
All requests use a unified structure:
```json
{
"action": "operation_name",
"namespace": "default",
"payload": {
"specific": "fields_for_operation"
}
}
```
### Response Format
All successful responses follow:
```json
{
"success": true,
"action": "operation_name",
"namespace": "default",
"data": {
"operation_specific": "fields"
},
"timestamp": "2024-01-15T10:30:00Z"
}
```
### Error Response Format
```json
{
"success": false,
"action": "operation_name",
"error": "error_code",
"message": "Human readable error message",
"details": {
"additional": "context"
},
"timestamp": "2024-01-15T10:30:00Z"
}
```
---
## Operations Reference
### Workflow Execution Operations
#### 1. START_WORKFLOW
**Description**: Start a new workflow execution
**Request**:
```json
{
"action": "START_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"workflow_type": "OrderProcessing",
"task_queue": "main_queue",
"input": {
"order_id": "12345",
"amount": 99.99
},
"options": {
"workflow_execution_timeout": 3600,
"workflow_run_timeout": 1800,
"workflow_task_timeout": 300
},
"memo": {
"user_id": "user_123",
"request_id": "req_456"
}
}
}
```
**Response**:
```json
{
"success": true,
"action": "START_WORKFLOW",
"data": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"start_time": "2024-01-15T10:30:00Z"
}
}
```
**Status Codes**: 200 (Success), 400 (Invalid input), 409 (Duplicate workflow_id)
---
#### 2. DESCRIBE_WORKFLOW
**Description**: Get detailed information about a specific workflow execution
**Request**:
```json
{
"action": "DESCRIBE_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456"
}
}
```
**Response**:
```json
{
"success": true,
"action": "DESCRIBE_WORKFLOW",
"data": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"workflow_type": "OrderProcessing",
"status": "RUNNING",
"start_time": "2024-01-15T10:30:00Z",
"last_update_time": "2024-01-15T10:31:00Z",
"execution_time": 60,
"memo": {...},
"search_attributes": {...},
"pending_activities": 2
}
}
```
---
#### 3. LIST_WORKFLOWS
**Description**: List workflow executions with optional filtering and pagination
**Request**:
```json
{
"action": "LIST_WORKFLOWS",
"namespace": "default",
"payload": {
"status": "RUNNING",
"workflow_type": "OrderProcessing",
"page_size": 50,
"next_page_token": "token_from_previous_response",
"filters": {
"start_time_from": "2024-01-10T00:00:00Z",
"start_time_to": "2024-01-20T23:59:59Z",
"custom_attribute": "value"
}
}
}
```
**Status Filter Values**: `RUNNING`, `COMPLETED`, `FAILED`, `CANCELED`, `TERMINATED`, `CONTINUED_AS_NEW`
**Response**:
```json
{
"success": true,
"action": "LIST_WORKFLOWS",
"data": {
"executions": [
{
"workflow_id": "wf_001",
"run_id": "run_001",
"workflow_type": "OrderProcessing",
"status": "RUNNING",
"start_time": "2024-01-15T10:00:00Z"
}
],
"next_page_token": "token_for_next_page",
"total_count": 150
}
}
```
---
#### 4. GET_WORKFLOW_HISTORY
**Description**: Retrieve workflow execution history (events)
**Request**:
```json
{
"action": "GET_WORKFLOW_HISTORY",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"max_events": 100,
"next_page_token": "token_from_previous"
}
}
```
**Response**:
```json
{
"success": true,
"action": "GET_WORKFLOW_HISTORY",
"data": {
"events": [
{
"event_id": 1,
"timestamp": "2024-01-15T10:30:00Z",
"type": "WorkflowExecutionStarted",
"attributes": {...}
},
{
"event_id": 2,
"timestamp": "2024-01-15T10:30:01Z",
"type": "WorkflowTaskScheduled",
"attributes": {...}
}
],
"next_page_token": "token_for_next_page"
}
}
```
---
#### 5. TERMINATE_WORKFLOW
**Description**: Stop a running workflow execution immediately
**Request**:
```json
{
"action": "TERMINATE_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"reason": "Order cancelled by user",
"details": {
"cancelled_by": "user_123",
"cancellation_code": "USER_REQUEST"
}
}
}
```
**Response**:
```json
{
"success": true,
"action": "TERMINATE_WORKFLOW",
"data": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"terminated_time": "2024-01-15T10:35:00Z"
}
}
```
---
#### 6. CANCEL_WORKFLOW
**Description**: Request graceful cancellation of a workflow (allows cleanup)
**Request**:
```json
{
"action": "CANCEL_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"reason": "Cancel request"
}
}
```
**Response**:
```json
{
"success": true,
"action": "CANCEL_WORKFLOW",
"data": {
"workflow_id": "unique-workflow-id",
"status": "CANCELING"
}
}
```
---
#### 7. SIGNAL_WORKFLOW
**Description**: Send a signal to a running workflow (trigger event handling)
**Request**:
```json
{
"action": "SIGNAL_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"signal_name": "payment_received",
"input": {
"payment_id": "pay_123",
"amount": 99.99
}
}
}
```
**Response**:
```json
{
"success": true,
"action": "SIGNAL_WORKFLOW",
"data": {
"workflow_id": "unique-workflow-id",
"signal_name": "payment_received",
"signaled_time": "2024-01-15T10:32:00Z"
}
}
```
---
#### 8. QUERY_WORKFLOW
**Description**: Query workflow state without modifying it (read-only)
**Request**:
```json
{
"action": "QUERY_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"query_type": "get_order_status",
"args": {
"include_history": false
}
}
}
```
**Response**:
```json
{
"success": true,
"action": "QUERY_WORKFLOW",
"data": {
"query_result": {
"order_status": "payment_pending",
"current_activity": "WaitForPayment",
"started_at": "2024-01-15T10:30:00Z"
}
}
}
```
---
#### 9. RESET_WORKFLOW
**Description**: Reset workflow to a specific point in history
**Request**:
```json
{
"action": "RESET_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"reset_type": "FIRST_DECISION_COMPLETED",
"event_id": 5
}
}
```
**Reset Types**: `FIRST_DECISION_COMPLETED`, `LAST_DECISION_COMPLETED`, `LAST_CONTINUOUS_DECISION_STARTED`, `BAD_BINARY_ID`
---
#### 10. UPDATE_WORKFLOW
**Description**: Update a running workflow (async operation)
**Request**:
```json
{
"action": "UPDATE_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "unique-workflow-id",
"run_id": "abc123def456",
"update_name": "pause_workflow",
"input": {
"reason": "maintenance"
}
}
}
```
---
### Activity Operations
#### 11. HEARTBEAT_ACTIVITY
**Description**: Record activity heartbeat to indicate progress
**Request**:
```json
{
"action": "HEARTBEAT_ACTIVITY",
"namespace": "default",
"payload": {
"task_token": "base64_encoded_token",
"details": {
"processed_items": 100,
"total_items": 500,
"percentage": 20
}
}
}
```
---
#### 12. COMPLETE_ACTIVITY
**Description**: Mark an activity task as completed with result
**Request**:
```json
{
"action": "COMPLETE_ACTIVITY",
"namespace": "default",
"payload": {
"task_token": "base64_encoded_token",
"result": {
"status": "success",
"output_data": {...}
}
}
}
```
---
#### 13. FAIL_ACTIVITY
**Description**: Mark an activity task as failed with error details
**Request**:
```json
{
"action": "FAIL_ACTIVITY",
"namespace": "default",
"payload": {
"task_token": "base64_encoded_token",
"failure": {
"type": "ApplicationError",
"message": "Database connection timeout",
"retry": true
}
}
}
```
---
### Namespace Operations
#### 14. LIST_NAMESPACES
**Description**: List all available namespaces
**Request**:
```json
{
"action": "LIST_NAMESPACES",
"payload": {
"page_size": 50,
"next_page_token": "token"
}
}
```
**Response**:
```json
{
"success": true,
"action": "LIST_NAMESPACES",
"data": {
"namespaces": [
{
"name": "default",
"description": "Default namespace",
"owner_email": "[email protected]",
"state": "ACTIVE",
"created_time": "2023-01-01T00:00:00Z"
}
]
}
}
```
---
#### 15. DESCRIBE_NAMESPACE
**Description**: Get details about a specific namespace
**Request**:
```json
{
"action": "DESCRIBE_NAMESPACE",
"namespace": "default",
"payload": {}
}
```
---
#### 16. CREATE_NAMESPACE
**Description**: Create a new namespace
**Request**:
```json
{
"action": "CREATE_NAMESPACE",
"payload": {
"namespace_name": "production",
"description": "Production workflows",
"owner_email": "[email protected]",
"retention_days": 30,
"data": {
"team": "backend",
"environment": "prod"
}
}
}
```
---
#### 17. UPDATE_NAMESPACE
**Description**: Update namespace configuration
**Request**:
```json
{
"action": "UPDATE_NAMESPACE",
"namespace": "default",
"payload": {
"description": "Updated description",
"retention_days": 45,
"data": {
"team": "backend"
}
}
}
```
---
#### 18. DELETE_NAMESPACE
**Description**: Delete a namespace (irreversible)
**Request**:
```json
{
"action": "DELETE_NAMESPACE",
"namespace": "default",
"payload": {
"reason": "Cleanup old namespace"
}
}
```
---
### Search Attributes Operations
#### 19. LIST_SEARCH_ATTRIBUTES
**Description**: List custom search attributes for a namespace
**Request**:
```json
{
"action": "LIST_SEARCH_ATTRIBUTES",
"namespace": "default",
"payload": {}
}
```
**Response**:
```json
{
"success": true,
"action": "LIST_SEARCH_ATTRIBUTES",
"data": {
"attributes": {
"order_id": "Keyword",
"customer_tier": "Keyword",
"total_amount": "Double",
"created_date": "Datetime"
}
}
}
```
---
#### 20. ADD_SEARCH_ATTRIBUTES
**Description**: Add custom search attributes to a namespace
**Request**:
```json
{
"action": "ADD_SEARCH_ATTRIBUTES",
"namespace": "default",
"payload": {
"search_attributes": {
"shipping_address": "Text",
"priority_level": "Int",
"estimated_delivery": "Datetime"
}
}
}
```
---
### Cluster Operations
#### 21. GET_CLUSTER_INFO
**Description**: Get cluster information and status
**Request**:
```json
{
"action": "GET_CLUSTER_INFO",
"payload": {}
}
```
**Response**:
```json
{
"success": true,
"action": "GET_CLUSTER_INFO",
"data": {
"cluster_name": "temporal-cluster",
"version": "1.24.0",
"members": [
{
"role": "leader",
"address": "temporal-0:7233"
},
{
"role": "member",
"address": "temporal-1:7233"
}
]
}
}
```
---
#### 22. LIST_CLUSTER_MEMBERS
**Description**: List all cluster members
**Request**:
```json
{
"action": "LIST_CLUSTER_MEMBERS",
"payload": {}
}
```
---
#### 23. GET_SYSTEM_INFO
**Description**: Get system information and metrics
**Request**:
```json
{
"action": "GET_SYSTEM_INFO",
"payload": {}
}
```
**Response**:
```json
{
"success": true,
"action": "GET_SYSTEM_INFO",
"data": {
"server_version": "1.24.0",
"capabilities": [
"SIGNAL_WORKFLOW",
"QUERY_WORKFLOW",
"UPDATE_WORKFLOW"
],
"uptime_seconds": 864000
}
}
```
---
### Metrics Endpoint
#### 24. GET_METRICS
**Description**: Get Prometheus metrics
**Request**:
```json
{
"action": "GET_METRICS",
"payload": {
"format": "prometheus",
"filter": "temporal_"
}
}
```
**Or via direct HTTP**:
```bash
GET https://api.riotpiao.com/workflow/metrics
```
**Response**: Prometheus text format metrics
**Key Metrics**:
- `temporal_workflow_execution_started_total` - Total workflows started
- `temporal_workflow_execution_completed_total` - Workflows completed
- `temporal_workflow_execution_failed_total` - Workflows failed
- `temporal_activity_execution_started_total` - Activities started
- `temporal_activity_execution_completed_total` - Activities completed
- `temporal_activity_execution_failed_total` - Activities failed
- `temporal_request_latency_histogram` - Request latencies
---
## HTTP Endpoints
### POST /workflow
Execute Temporal operations with JSON request body (all operations above)
```bash
curl -X POST https://api.riotpiao.com/workflow \
-H 'Content-Type: application/json' \
-d '{
"action": "START_WORKFLOW",
"namespace": "default",
"payload": {...}
}'
```
---
### GET /workflow/metrics
Get Prometheus metrics directly
```bash
curl https://api.riotpiao.com/workflow/metrics
```
---
### GET /workflow/health
Health check endpoint
```bash
curl https://api.riotpiao.com/workflow/health
```
**Response**:
```json
{
"status": "healthy",
"temporal_connected": true,
"latency_ms": 5
}
```
---
## Error Codes
| Code | HTTP | Meaning |
|------|------|---------|
| `INVALID_REQUEST` | 400 | Missing or invalid fields |
| `INVALID_ACTION` | 400 | Unknown action |
| `AUTHENTICATION_FAILED` | 401 | Auth header missing/invalid |
| `PERMISSION_DENIED` | 403 | No permission for action |
| `NAMESPACE_NOT_FOUND` | 404 | Namespace doesn't exist |
| `WORKFLOW_NOT_FOUND` | 404 | Workflow doesn't exist |
| `WORKFLOW_ALREADY_EXISTS` | 409 | Workflow ID duplicate |
| `WORKFLOW_EXECUTING` | 409 | Workflow still running |
| `TEMPORAL_UNAVAILABLE` | 503 | Temporal server unreachable |
| `INTERNAL_ERROR` | 500 | Unexpected server error |
---
## Task Queue Management
### What is a Task Queue?
A **Task Queue** is a queue where:
- **Workflow Tasks** are dispatched to workflow workers
- **Activity Tasks** are dispatched to activity workers
- Multiple workers can listen on the same queue
- Load is distributed among available workers
### Task Queue Attributes
```json
{
"name": "main_queue",
"kind": "WORKFLOW", // or ACTIVITY
"poison_pill_count": 0,
"reader_count": 3,
"ack_level": 1050
}
```
---
## TaskQueue Recommendation & Analysis
### Option 1: Expose TaskQueue as Managed Resource (RECOMMENDED)
**Design**: Allow CRUD operations on task queues via `/workflow` endpoint with new operations:
```json
{
"action": "LIST_TASK_QUEUES",
"payload": {
"namespace": "default",
"queue_type": "WORKFLOW" // or ACTIVITY
}
}
```
**Pros**:
**Visibility** - Know all active queues and their status
**Monitoring** - Track reader count, poison pills, ack level
**Debugging** - Identify stuck queues or low reader count
**Consistent UI** - All operations through same `/workflow` endpoint
**Alerting** - Monitor queue health metrics
**Cons**:
**Limited Utility** - Can't modify task queue behavior (read-only mostly)
**Not Needed for Normal Operations** - Workers create/manage queues automatically
**Complex State** - Queue state changes automatically with worker connections
---
### Option 2: Hidden Implementation Detail (NOT RECOMMENDED)
**Design**: Don't expose TaskQueue operations; let workers manage queues
**Pros**:
**Simpler API** - Fewer operations
**Fewer Bugs** - Less user confusion
**Less Operational Burden** - Workers handle it
**Cons**:
**No Visibility** - Can't see queue status
**Harder Debugging** - No way to verify queue health
**No Alerting** - Can't monitor queue metrics
**Production Risk** - Silent failures if queues stuck
---
### **RECOMMENDATION: Option 1 (Expose as Read-Only Monitor)**
**Implementation**:
```json
{
"action": "LIST_TASK_QUEUES",
"namespace": "default",
"payload": {
"queue_type": "WORKFLOW"
}
}
```
Response:
```json
{
"success": true,
"data": {
"queues": [
{
"name": "main_queue",
"type": "WORKFLOW",
"reader_count": 3,
"poison_pill_count": 0,
"last_activity": "2024-01-15T10:32:00Z"
}
]
}
}
```
**Benefits**:
- Operational visibility without complication
- Enables monitoring dashboards
- Helps debug worker connectivity issues
- No direct manipulation (workers manage lifecycle)
- Completes the REST API for all Temporal concepts
---
## Usage Examples
### Example 1: Start a Workflow and Monitor
```bash
# Start workflow
curl -X POST https://api.riotpiao.com/workflow \
-H 'Content-Type: application/json' \
-d '{
"action": "START_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "order-123",
"workflow_type": "ProcessOrder",
"task_queue": "orders_queue",
"input": {"order_id": "123", "amount": 99.99}
}
}'
# Check workflow status
curl -X POST https://api.riotpiao.com/workflow \
-d '{
"action": "DESCRIBE_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "order-123",
"run_id": "run_abc123"
}
}'
```
### Example 2: Send Signal to Running Workflow
```bash
curl -X POST https://api.riotpiao.com/workflow \
-H 'Content-Type: application/json' \
-d '{
"action": "SIGNAL_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "order-123",
"run_id": "run_abc123",
"signal_name": "payment_completed",
"input": {"amount": 99.99, "payment_id": "pay_456"}
}
}'
```
### Example 3: Query Workflow State
```bash
curl -X POST https://api.riotpiao.com/workflow \
-d '{
"action": "QUERY_WORKFLOW",
"namespace": "default",
"payload": {
"workflow_id": "order-123",
"run_id": "run_abc123",
"query_type": "get_status"
}
}'
```
### Example 4: List Running Workflows
```bash
curl -X POST https://api.riotpiao.com/workflow \
-d '{
"action": "LIST_WORKFLOWS",
"namespace": "default",
"payload": {
"status": "RUNNING",
"page_size": 50
}
}'
```
### Example 5: Monitor Task Queues
```bash
curl -X POST https://api.riotpiao.com/workflow \
-d '{
"action": "LIST_TASK_QUEUES",
"namespace": "default",
"payload": {
"queue_type": "WORKFLOW"
}
}'
```
---
## Authentication & Security
**Planned for Phase 3**:
- Bearer token authentication
- Namespace-based authorization
- Action-based permissions (e.g., can only QUERY, not TERMINATE)
- Audit logging
---
## Rate Limiting
**Planned for Phase 4**:
- Per-namespace rate limits
- Per-action rate limits
- Backoff/retry guidance
---
## Pagination
For list operations supporting pagination:
```json
{
"action": "LIST_WORKFLOWS",
"payload": {
"page_size": 50,
"next_page_token": "token_from_previous_response"
}
}
```
Response includes `next_page_token` for fetching next page:
```json
{
"data": {
"executions": [...],
"next_page_token": "token_for_next_page"
}
}
```
---
## Filtering Syntax
For advanced filtering in LIST operations:
**Time-based**:
- `start_time_from`: ISO 8601 datetime
- `start_time_to`: ISO 8601 datetime
**Status**:
- `status`: RUNNING, COMPLETED, FAILED, CANCELED, TERMINATED, CONTINUED_AS_NEW
**Type**:
- `workflow_type`: Workflow type name
**Custom Attributes**:
- Custom indexed attributes: `"custom_field": "value"`
---
## Timeout Configuration
For long-running operations:
```json
{
"action": "START_WORKFLOW",
"payload": {
"workflow_id": "long_running",
"workflow_type": "LongTask",
"task_queue": "long_queue",
"options": {
"workflow_execution_timeout": 86400, // 24 hours
"workflow_run_timeout": 3600, // 1 hour per run
"workflow_task_timeout": 300 // 5 minutes per task
}
}
}
```
---
## Support & Debugging
### Check Health
```bash
curl https://api.riotpiao.com/workflow/health
```
### View Cluster Info
```bash
curl -X POST https://api.riotpiao.com/workflow \
-d '{"action": "GET_CLUSTER_INFO", "payload": {}}'
```
### Monitor Metrics
```bash
curl https://api.riotpiao.com/workflow/metrics | grep temporal_workflow
```
### Check Namespace Status
```bash
curl -X POST https://api.riotpiao.com/workflow \
-d '{
"action": "DESCRIBE_NAMESPACE",
"namespace": "default"
}'
```
---
## What Changed from Direct Temporal Access
| Operation | Before (Direct gRPC) | After (REST) |
|-----------|----------------------|--------------|
| Start Workflow | `client.StartWorkflowExecution()` | `POST /workflow` with `START_WORKFLOW` action |
| List Workflows | `client.ListWorkflowExecutions()` | `POST /workflow` with `LIST_WORKFLOWS` action |
| Signal Workflow | `client.SignalWorkflowExecution()` | `POST /workflow` with `SIGNAL_WORKFLOW` action |
| Query Workflow | `client.QueryWorkflowExecution()` | `POST /workflow` with `QUERY_WORKFLOW` action |
| Access Metrics | `curl localhost:6933/metrics` | `curl https://api.riotpiao.com/workflow/metrics` |
**Benefits**:
- Single endpoint for all operations
- No port forwarding needed
- Unified authentication/authorization
- Easier monitoring and logging
- RESTful consistency
---
## Next Steps
1. **Implement** gateway integration with Temporal Go SDK
2. **Add** gRPC tunneling through HTTP/2
3. **Deploy** `/workflow` endpoint
4. **Monitor** operations and latencies
5. **Extend** with Phase 3 authentication
6. **Add** Phase 4 rate limiting
---
**Ready to implement?** See implementation guide: `TEMPORAL_IMPLEMENTATION.md`