feat: add canvas compatibility checking for connection validation

2026-09-05 01:01:01 -07:00
parent 2a3b080e29
commit 9624f0e18d
5 changed files with 704 additions and 19 deletions
@@ -0,0 +1,322 @@
+# Canvas Reasoner: Auto-Inferring Workflow Connections
+
+## Overview
+
+The **CanvasReasonerActivity** uses LLM reasoning to automatically suggest connections between workflow activities when users drop new nodes onto the canvas. It analyzes input/output compatibility and detects connection problems.
+
+## Connection Logic
+
+### How It Works
+
+1. **Analyze Node Schemas**
+   - Get each activity's input/output fields from knowledge base
+   - Activities are classified as:
+     - **Generators**: No inputs, has outputs (e.g., API call, trigger)
+     - **Processors**: Has inputs and outputs (e.g., analyze code, security scan)
+     - **Sinks/Terminals**: Has inputs, no outputs (e.g., notification, approval)
+
+2. **LLM Reasoning**
+   - Pass all nodes + their schemas to reasoning model
+   - Ask LLM to suggest edges based on:
+     - Type compatibility (string→string, object→object)
+     - Logical execution order
+     - Data flow requirements
+     - Common workflow patterns
+
+3. **Validate Suggestions**
+   - Check all suggested edges exist in node map
+   - Skip self-loops
+   - Remove duplicates
+
+4. **Compatibility Checking**
+   - For each suggested edge: `source → target`
+   - Verify source produces outputs
+   - Verify target accepts inputs
+   - Check output/input type compatibility
+   - Flag incompatible connections
+
+5. **Identify Issues**
+   - Collect all incompatible edges
+   - Identify disconnected nodes (no edges in/out)
+   - Generate user alerts for problems
+
+## Connection Impossibility Detection
+
+### Why Connections Fail
+
+1. **Missing Outputs**
+   ```
+   NotifyStatusActivity → AnalyzeCodeActivity
+   ⚠️ NotifyStatusActivity produces no outputs
+   Reason: Notification is terminal activity (sink)
+   Solution: Add an intermediate processor that has outputs
+   ```
+
+2. **Missing Inputs**
+   ```
+   CloneRepoActivity → ApproveWorkflowActivity
+   ⚠️ ApproveWorkflowActivity accepts no inputs
+   Reason: Approval is a terminal activity (sink)
+   Solution: ApproveWorkflowActivity only works as final step
+   ```
+
+3. **Type Mismatch**
+   ```
+   LLMInferenceActivity (output: string) → DeploymentPreCheckActivity (input: object)
+   ⚠️ String output cannot satisfy object input requirement
+   Reason: Incompatible data types
+   Solution: Use LLM transformation node to convert string→object
+   ```
+
+4. **Semantic Incompatibility**
+   ```
+   NotifyStatusActivity → CloneRepoActivity
+   ⚠️ No logical connection between these activities
+   Reason: Notification cannot be input to clone operation
+   Solution: Ensure data flow makes semantic sense
+   ```
+
+### Incompatibility Data Structure
+
+```json
+{
+  "incompatible_edges": [
+    {
+      "source": "node-1",
+      "target": "node-2",
+      "reason": "Source activity produces no outputs",
+      "source_needs": "any output",
+      "target_needs": "path, depth",
+      "suggestion": "Use LLM transformation to map outputs to inputs"
+    }
+  ],
+  "disconnected_nodes": ["node-5", "node-8"],
+  "user_alerts": [
+    "⚠️  node-1 → node-2: Source activity produces no outputs. Use LLM transformation to map outputs to inputs",
+    "🔌 Node 'NotifyStatus-1' has no connections. Consider adding edges or removing it."
+  ]
+}
+```
+
+## User Alerts
+
+### Alert Types
+
+1. **Incompatibility Warnings** (⚠️)
+   ```
+   ⚠️  source → target: reason. suggestion.
+   ```
+   - Highlighted in red on canvas
+   - Shows in error sidebar
+   - Prevents workflow execution until fixed
+
+2. **Disconnection Warnings** (🔌)
+   ```
+   🔌 Node 'label' has no connections. Consider adding edges or removing it.
+   ```
+   - Highlighted in yellow
+   - Nodes with no input/output edges
+   - May be valid (first step, last step) or indicate design error
+
+3. **Type Mismatch Info** (ℹ️)
+   ```
+   ℹ️  To connect source → target, use transformer to map: {source_outputs} → {target_inputs}
+   ```
+   - Suggestion to use intermediate LLM node
+   - Provides mapping information
+
+## Frontend Integration
+
+### Canvas UI Feedback
+
+When CanvasReasonerActivity returns incompatibilities:
+
+1. **Visual Markers**
+   - Incompatible suggested edges: ❌ red dashed line (don't auto-add)
+   - Disconnected nodes: ⚠️ yellow border
+
+2. **Sidebar Alerts**
+   ```
+   🚨 Connection Issues (3)
+   
+   ⚠️  CloneRepo → ApproveWorkflow
+      Reason: ApproveWorkflow is terminal (no outputs)
+      Suggestion: Place ApproveWorkflow at end of workflow
+   
+   ⚠️  LLMInference → DeploymentPreCheck
+      Reason: Type mismatch (string ≠ object)
+      Suggestion: Add LLM transformation node
+   
+   🔌 SecurityScan-1 has no incoming edges
+      Suggestion: Connect CloneRepo → SecurityScan
+   ```
+
+3. **User Actions**
+   - ✅ Accept suggestions (green edges)
+   - ❌ Reject incompatible edges
+   - 🔧 Add transformer nodes
+   - 🗑️ Remove disconnected nodes
+
+### API Response Example
+
+```json
+{
+  "suggested_edges": [
+    {"source": "clone-1", "target": "analyze-1"},
+    {"source": "analyze-1", "target": "security-1"},
+    {"source": "security-1", "target": "report-1"}
+  ],
+  "reasoning": "Standard code review workflow: clone → analyze → scan → report",
+  "confidence": 0.92,
+  "incompatible_edges": [
+    {
+      "source": "report-1",
+      "target": "approve-1",
+      "reason": "ReportGenerator has no outputs (terminal activity)",
+      "suggestion": "ApproveWorkflow can only be a final step"
+    }
+  ],
+  "disconnected_nodes": [],
+  "user_alerts": [
+    "⚠️  report-1 → approve-1: ReportGenerator has no outputs (terminal activity). ApproveWorkflow can only be a final step"
+  ]
+}
+```
+
+## Knowledge Base Schema
+
+Each activity in `activity_knowledge_base.json` defines:
+
+```json
+{
+  "name": "CloneRepoActivity",
+  "inputs": {
+    "repo": {"type": "string", "required": true},
+    "branch": {"type": "string", "required": false}
+  },
+  "outputs": {
+    "path": {"type": "string"},
+    "commit": {"type": "string"}
+  }
+}
+```
+
+### Classification Rules
+
+- **Generator** (0 inputs): trigger, API call, schedule
+- **Processor** (1+ inputs, 1+ outputs): analysis, transformation, scan
+- **Sink** (1+ inputs, 0 outputs): notification, approval, archive
+- **Bypass** (0 inputs, 0 outputs): rare - usually error
+
+## Common Patterns
+
+### ✅ Valid Chains
+
+```
+CloneRepo → Analyze → SecurityScan → Report
+(generator) → (processor) → (processor) → (sink)
+```
+
+```
+Trigger → LLMInference → Decision → (Branch: Notify OR Approve)
+(gen) → (processor) → (processor) → (sink)
+```
+
+### ❌ Invalid Chains
+
+```
+Notify → CloneRepo  ❌
+(sink) → (generator) - backward flow
+
+CloneRepo → CloneRepo → Analyze  ❌
+self-loop - no benefit
+
+Analyze → Approve → Notify  ❌
+Approve is terminal (sink), can't output to Notify
+```
+
+## Edge Cases
+
+### Multiple Outputs → Single Input
+```
+SecurityScan → Report
+SecurityScan outputs: [issues, metrics, severity]
+Report inputs: [report_data]
+
+LLM must infer: bundle all outputs into single report_data object
+Confidence: 0.7 (requires transformation)
+```
+
+### Terminal Activities
+- **ApproveWorkflowActivity**: Must be last (blocks workflow)
+- **NotifyStatusActivity**: Can be mid-workflow (async notify)
+- **ArchiveResultsActivity**: Should be last (persistence)
+
+### Data Transformation
+When source outputs don't match target inputs:
+
+```python
+# User can insert transformer node:
+LLMInference → [LLMTransformer] → DeploymentPreCheck
+
+# Transformer:
+# - Input: LLMInference.output (string)
+# - Output: DeploymentPreCheck.requirements (object)
+# - Action: Call LLM to convert format
+```
+
+## Testing Incompatibility Detection
+
+### Test Case 1: Terminal Activity as Source
+```go
+source := db.WorkflowNode{ID: "n1", Type: "notify-status", Label: "Notify"}
+target := db.WorkflowNode{ID: "n2", Type: "clone-repo", Label: "Clone"}
+
+warnings := CheckConnectionCompatibility(source, target)
+// Should warn: NotifyStatusActivity produces no outputs
+```
+
+### Test Case 2: Type Mismatch
+```go
+source := db.WorkflowNode{ID: "n1", Type: "llm-inference", ...}
+target := db.WorkflowNode{ID: "n2", Type: "deployment-check", ...}
+
+warnings := CheckConnectionCompatibility(source, target)
+// Should warn: string output ≠ object input
+```
+
+### Test Case 3: Disconnected Node
+```go
+nodes := []db.WorkflowNode{n1, n2, n3}
+edges := []db.WorkflowEdge{{Source: "n1", Target: "n2"}}
+
+disconnected := IdentifyDisconnectedNodes(nodes, edges)
+// Should return ["n3"]
+```
+
+## Future Enhancements
+
+1. **Automatic Transformer Insertion**
+   - Detect incompatibilities
+   - Auto-suggest LLM transformer nodes
+   - Chain transformers if needed
+
+2. **Confidence Scoring**
+   - Increase when types match perfectly
+   - Decrease for semantic mismatches
+   - Factor in activity dependencies
+
+3. **Learning from History**
+   - Track successful workflows
+   - Remember user edits to suggestions
+   - Improve LLM prompts over time
+
+4. **Multi-Path Analysis**
+   - Suggest multiple connection topologies
+   - Show cost/efficiency of each
+   - Rank by execution time/cost
+
+5. **Dry-Run Validation**
+   - Execute suggested workflow in simulation
+   - Catch runtime errors early
+   - Show data flow through each node