Test
b2cebe1ba7
- Add LLMBatcher for grouping similar LLM requests - Automatic grouping by request type and model - Enqueue requests with optional result channels - Auto-flush on max batch size - Manual flush on demand - Time-based flush (max batch age) - Result delivery via channels - Batch status tracking and error handling - API cost reduction through request consolidation - 29 LLM batching tests, all passing Features: - Enqueue() for adding LLM requests - Flush() for manual batch creation - GetPendingBatch() for next batch - MarkBatchExecuting/Completed/Failed() - GroupByTypeAndModel() - automatic grouping - ResultDelivery() via channels - GetStats() for batching statistics - Token counting and tracking Performance Benefits: - 3 Implementer requests → 1 API call - N requests in M batches saves N-M API calls - Example: 30 requests in 3 batches saves 27 API calls (90% reduction) - Configurable batch size (default 10) - Configurable max age (default 2s) Grouping Strategy: - Requests grouped by (Type, Model) - Implementer + claude-opus → separate batch from Implementer + gpt-4 - Judge requests grouped separately from Implementer - Enables provider-specific optimizations Result Delivery: - Each request gets async result channel - Results delivered to channels on completion - Error results on batch failure - Non-blocking result delivery Statistics: - Total requests tracked - Total batches created - Average requests per batch - API calls saved calculation - Total tokens used - Total execution time Test Coverage: - 29 LLM batching tests (enqueue, flush, grouping, delivery) - Result delivery verification - Token counting tested - Auto-flush and manual flush - Error handling - Multi-type grouping - Concurrent safety (RWMutex) Next: T2.7 (Workflow history pruning)