Implement LLMInferenceActivity integration for Temporal workflows

Workflow Input Structure:
  ├─ question: User content for reasoning
  ├─ project: Project ID for scoping
  ├─ operations: Flags for link_entities, infer_facts, reason_query, summarize
  └─ llm_activity: Configuration for LLMInferenceActivity
       ├─ model: Selected based on complexity (reasoning|ornith:35b|qwen2.5:3b)
       ├─ system_prompt: Task-specific instruction (Zep-backed)
       ├─ user_prompt: Content to process
       ├─ temperature: 0.7 (reasoning) or 0.5 (validation)
       └─ max_tokens: 2048 (reasoning) or 512 (validation)

Model Selection:
  ├─ reason_query=true, summarize=true → reasoning (DeepSeek-R1, complex)
  ├─ reason_query=true, summarize=false → ornith:35b (medium)
  └─ reason_query=false → qwen2.5:3b (fast, <100ms)

System Prompts (handlers/llm_prompts.rs):
  ├─ entity_extraction_system_prompt(): Extract entities + relationships + facts
  ├─ reasoning_system_prompt(): Step-by-step reasoning + answers
  ├─ agent_capability_validation_prompt(): Validate agent capabilities
  └─ fact_validation_system_prompt(): Detect contradictions

Workflow Activity Execution:
  ├─ Temporal receives workflow input with llm_activity config
  ├─ ReasoningWorkflow orchestrates:
  │  ├─ Activity 1: RetrieveMemory (optional context)
  │  ├─ Activity 2: LLMInferenceActivity (calls /v1/chat/completions via gateway)
  │  │   └─ Retries: 3× with backoff (2s, 4s, 8s)
  │  │   └─ Timeout: 120s
  │  │   └─ JWT propagation: Authorization: Bearer header
  │  ├─ Activity 3: PersistResults (save to memory_entity/memory_edge)
  │  └─ Activity 4: SummarizeFindings (return results)
  ├─ Memory handler polls DESCRIBE_WORKFLOW (30× with 100ms delay, 3s timeout)
  └─ Returns ReasoningResult with answers, confidence, reasoning_steps

Changes:
  ├─ execute_reasoning_workflow(): Build llm_activity config with model selection
  ├─ select_llm_model(): Choose model based on operation complexity
  ├─ build_system_prompt(): Use Zep-inspired prompts for reasoning
  ├─ handlers/llm_prompts.rs: Centralized prompt templates (5 system + 4 user builders)
  ├─ AgentInitialization: Include llm_activity for capability validation
  └─ Fixed duplicate extract_jwt_token call in agent_handler.rs

Activity Contract:
  ├─ Workflow input includes llm_activity block
  ├─ Temporal passes to LLMInferenceActivity
  ├─ Activity substitutes {{ previous_output }} template variables
  ├─ Activity calls POST /v1/chat/completions with JWT header
  ├─ Activity returns { response, model, stop_reason, tokens_used }
  ├─ PersistResults activity stores results to DB
  └─ Workflow returns: question, answers[], confidence, reasoning_steps[]

Tests Added:
  + 14 new tests in llm_prompts.rs (prompt validation, user prompt builders)

Compilation: 
This commit is contained in:
2026-09-05 00:52:30 -07:00
parent b33901aa5b
commit 4c275525e9
5 changed files with 777 additions and 8 deletions
@@ -260,21 +260,68 @@ pub async fn unified_synthesis_handler(
})
}
/// Select LLM model based on operations complexity
fn select_llm_model(operations: &serde_json::Value) -> &'static str {
let reason_query = operations.get("reason_query")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let summarize = operations.get("summarize")
.and_then(|v| v.as_bool())
.unwrap_or(false);
match (reason_query, summarize) {
(true, true) => "reasoning", // Complex: extract + reason + summarize
(true, false) => "ornith:35b", // Medium: extract + reason
(false, _) => "qwen2.5:3b", // Quick: only linking/inference
}
}
/// Build LLM system prompt for entity/fact extraction
fn build_system_prompt(operations: &serde_json::Value) -> String {
let reason_query = operations.get("reason_query")
.and_then(|v| v.as_bool())
.unwrap_or(false);
// Use Zep-inspired reasoning prompt for complex reasoning, entity extraction otherwise
if reason_query {
crate::handlers::llm_prompts::reasoning_system_prompt().to_string()
} else {
crate::handlers::llm_prompts::entity_extraction_system_prompt().to_string()
}
}
/// Execute reasoning workflow via Temporal
/// Returns parsed ReasoningResult from workflow output
async fn execute_reasoning_workflow(
client: &crate::agent::client_sdk::SynthesisClient,
body: &UnifiedSynthesisRequest,
) -> Result<ReasoningResult, String> {
// Build START_WORKFLOW request
// Prepare operations metadata
let operations = serde_json::json!({
"link_entities": body.link_entities,
"infer_facts": body.infer_facts,
"reason_query": body.reason_query,
"summarize": body.summarize
});
// Select model based on complexity
let model = select_llm_model(&operations);
let system_prompt = build_system_prompt(&operations);
// Build START_WORKFLOW request with LLMInferenceActivity inputs
let workflow_input = serde_json::json!({
// Workflow input
"question": body.content,
"project": body.project,
"operations": {
"link_entities": body.link_entities,
"infer_facts": body.infer_facts,
"reason_query": body.reason_query,
"summarize": body.summarize
"operations": operations,
// LLMInferenceActivity inputs (passed to Temporal activity)
"llm_activity": {
"model": model,
"system_prompt": system_prompt,
"user_prompt": body.content,
"temperature": 0.7,
"max_tokens": 2048
}
});