poimen-workflows/docs/CANVAS_REASONER_LOGIC.md

# Canvas Reasoner: Auto-Inferring Workflow Connections

## Overview

The **CanvasReasonerActivity** uses LLM reasoning to automatically suggest connections between workflow activities when users drop new nodes onto the canvas. It analyzes input/output compatibility and detects connection problems.

## Connection Logic

### How It Works

1. **Analyze Node Schemas**
   - Get each activity's input/output fields from knowledge base
   - Activities are classified as:
     - **Generators**: No inputs, has outputs (e.g., API call, trigger)
     - **Processors**: Has inputs and outputs (e.g., analyze code, security scan)
     - **Sinks/Terminals**: Has inputs, no outputs (e.g., notification, approval)

2. **LLM Reasoning**
   - Pass all nodes + their schemas to reasoning model
   - Ask LLM to suggest edges based on:
     - Type compatibility (string→string, object→object)
     - Logical execution order
     - Data flow requirements
     - Common workflow patterns

3. **Validate Suggestions**
   - Check all suggested edges exist in node map
   - Skip self-loops
   - Remove duplicates

4. **Compatibility Checking**
   - For each suggested edge: `source → target`
   - Verify source produces outputs
   - Verify target accepts inputs
   - Check output/input type compatibility
   - Flag incompatible connections

5. **Identify Issues**
   - Collect all incompatible edges
   - Identify disconnected nodes (no edges in/out)
   - Generate user alerts for problems

## Connection Impossibility Detection

### Why Connections Fail

1. **Missing Outputs**
   ```
   NotifyStatusActivity → AnalyzeCodeActivity
   ⚠️ NotifyStatusActivity produces no outputs
   Reason: Notification is terminal activity (sink)
   Solution: Add an intermediate processor that has outputs
   ```

2. **Missing Inputs**
   ```
   CloneRepoActivity → ApproveWorkflowActivity
   ⚠️ ApproveWorkflowActivity accepts no inputs
   Reason: Approval is a terminal activity (sink)
   Solution: ApproveWorkflowActivity only works as final step
   ```

3. **Type Mismatch**
   ```
   LLMInferenceActivity (output: string) → DeploymentPreCheckActivity (input: object)
   ⚠️ String output cannot satisfy object input requirement
   Reason: Incompatible data types
   Solution: Use LLM transformation node to convert string→object
   ```

4. **Semantic Incompatibility**
   ```
   NotifyStatusActivity → CloneRepoActivity
   ⚠️ No logical connection between these activities
   Reason: Notification cannot be input to clone operation
   Solution: Ensure data flow makes semantic sense
   ```

### Incompatibility Data Structure

```json
{
  "incompatible_edges": [
    {
      "source": "node-1",
      "target": "node-2",
      "reason": "Source activity produces no outputs",
      "source_needs": "any output",
      "target_needs": "path, depth",
      "suggestion": "Use LLM transformation to map outputs to inputs"
    }
  ],
  "disconnected_nodes": ["node-5", "node-8"],
  "user_alerts": [
    "⚠️  node-1 → node-2: Source activity produces no outputs. Use LLM transformation to map outputs to inputs",
    "🔌 Node 'NotifyStatus-1' has no connections. Consider adding edges or removing it."
  ]
}
```

## User Alerts

### Alert Types

1. **Incompatibility Warnings** (⚠️)
   ```
   ⚠️  source → target: reason. suggestion.
   ```
   - Highlighted in red on canvas
   - Shows in error sidebar
   - Prevents workflow execution until fixed

2. **Disconnection Warnings** (🔌)
   ```
   🔌 Node 'label' has no connections. Consider adding edges or removing it.
   ```
   - Highlighted in yellow
   - Nodes with no input/output edges
   - May be valid (first step, last step) or indicate design error

3. **Type Mismatch Info** (ℹ️)
   ```
   ℹ️  To connect source → target, use transformer to map: {source_outputs} → {target_inputs}
   ```
   - Suggestion to use intermediate LLM node
   - Provides mapping information

## Frontend Integration

### Canvas UI Feedback

When CanvasReasonerActivity returns incompatibilities:

1. **Visual Markers**
   - Incompatible suggested edges: ❌ red dashed line (don't auto-add)
   - Disconnected nodes: ⚠️ yellow border

2. **Sidebar Alerts**
   ```
   🚨 Connection Issues (3)
   
   ⚠️  CloneRepo → ApproveWorkflow
      Reason: ApproveWorkflow is terminal (no outputs)
      Suggestion: Place ApproveWorkflow at end of workflow
   
   ⚠️  LLMInference → DeploymentPreCheck
      Reason: Type mismatch (string ≠ object)
      Suggestion: Add LLM transformation node
   
   🔌 SecurityScan-1 has no incoming edges
      Suggestion: Connect CloneRepo → SecurityScan
   ```

3. **User Actions**
   - ✅ Accept suggestions (green edges)
   - ❌ Reject incompatible edges
   - 🔧 Add transformer nodes
   - 🗑️ Remove disconnected nodes

### API Response Example

```json
{
  "suggested_edges": [
    {"source": "clone-1", "target": "analyze-1"},
    {"source": "analyze-1", "target": "security-1"},
    {"source": "security-1", "target": "report-1"}
  ],
  "reasoning": "Standard code review workflow: clone → analyze → scan → report",
  "confidence": 0.92,
  "incompatible_edges": [
    {
      "source": "report-1",
      "target": "approve-1",
      "reason": "ReportGenerator has no outputs (terminal activity)",
      "suggestion": "ApproveWorkflow can only be a final step"
    }
  ],
  "disconnected_nodes": [],
  "user_alerts": [
    "⚠️  report-1 → approve-1: ReportGenerator has no outputs (terminal activity). ApproveWorkflow can only be a final step"
  ]
}
```

## Knowledge Base Schema

Each activity in `activity_knowledge_base.json` defines:

```json
{
  "name": "CloneRepoActivity",
  "inputs": {
    "repo": {"type": "string", "required": true},
    "branch": {"type": "string", "required": false}
  },
  "outputs": {
    "path": {"type": "string"},
    "commit": {"type": "string"}
  }
}
```

### Classification Rules

- **Generator** (0 inputs): trigger, API call, schedule
- **Processor** (1+ inputs, 1+ outputs): analysis, transformation, scan
- **Sink** (1+ inputs, 0 outputs): notification, approval, archive
- **Bypass** (0 inputs, 0 outputs): rare - usually error

## Common Patterns

### ✅ Valid Chains

```
CloneRepo → Analyze → SecurityScan → Report
(generator) → (processor) → (processor) → (sink)
```

```
Trigger → LLMInference → Decision → (Branch: Notify OR Approve)
(gen) → (processor) → (processor) → (sink)
```

### ❌ Invalid Chains

```
Notify → CloneRepo  ❌
(sink) → (generator) - backward flow

CloneRepo → CloneRepo → Analyze  ❌
self-loop - no benefit

Analyze → Approve → Notify  ❌
Approve is terminal (sink), can't output to Notify
```

## Edge Cases

### Multiple Outputs → Single Input
```
SecurityScan → Report
SecurityScan outputs: [issues, metrics, severity]
Report inputs: [report_data]

LLM must infer: bundle all outputs into single report_data object
Confidence: 0.7 (requires transformation)
```

### Terminal Activities
- **ApproveWorkflowActivity**: Must be last (blocks workflow)
- **NotifyStatusActivity**: Can be mid-workflow (async notify)
- **ArchiveResultsActivity**: Should be last (persistence)

### Data Transformation
When source outputs don't match target inputs:

```python
# User can insert transformer node:
LLMInference → [LLMTransformer] → DeploymentPreCheck

# Transformer:
# - Input: LLMInference.output (string)
# - Output: DeploymentPreCheck.requirements (object)
# - Action: Call LLM to convert format
```

## Testing Incompatibility Detection

### Test Case 1: Terminal Activity as Source
```go
source := db.WorkflowNode{ID: "n1", Type: "notify-status", Label: "Notify"}
target := db.WorkflowNode{ID: "n2", Type: "clone-repo", Label: "Clone"}

warnings := CheckConnectionCompatibility(source, target)
// Should warn: NotifyStatusActivity produces no outputs
```

### Test Case 2: Type Mismatch
```go
source := db.WorkflowNode{ID: "n1", Type: "llm-inference", ...}
target := db.WorkflowNode{ID: "n2", Type: "deployment-check", ...}

warnings := CheckConnectionCompatibility(source, target)
// Should warn: string output ≠ object input
```

### Test Case 3: Disconnected Node
```go
nodes := []db.WorkflowNode{n1, n2, n3}
edges := []db.WorkflowEdge{{Source: "n1", Target: "n2"}}

disconnected := IdentifyDisconnectedNodes(nodes, edges)
// Should return ["n3"]
```

## Future Enhancements

1. **Automatic Transformer Insertion**
   - Detect incompatibilities
   - Auto-suggest LLM transformer nodes
   - Chain transformers if needed

2. **Confidence Scoring**
   - Increase when types match perfectly
   - Decrease for semantic mismatches
   - Factor in activity dependencies

3. **Learning from History**
   - Track successful workflows
   - Remember user edits to suggestions
   - Improve LLM prompts over time

4. **Multi-Path Analysis**
   - Suggest multiple connection topologies
   - Show cost/efficiency of each
   - Rank by execution time/cost

5. **Dry-Run Validation**
   - Execute suggested workflow in simulation
   - Catch runtime errors early
   - Show data flow through each node