fbc1f6122f
Implemented workflow templates based on Anthropic research: - parallel-review.md: Parallel execution of security + performance reviews - evaluator-optimizer.md: Iterative improvement loop (code-skeptic → the-fixer) - chain-of-thought.md: Sequential step decomposition with gates Each template includes: - Pattern overview and implementation - Execution flow diagram - Benefits and configuration - Usage examples Related: Issues #21, #22 - Patterns from research Milestone: #47 Cognitive Enhancement
104 lines
2.5 KiB
Markdown
104 lines
2.5 KiB
Markdown
# Evaluator-Optimizer Workflow
|
|
|
|
Implements the Evaluator-Optimizer pattern from Anthropic research.
|
|
|
|
## Overview
|
|
|
|
Iterative improvement loop: evaluator reviews, optimizer improves, repeat until convergence.
|
|
|
|
## Pattern
|
|
|
|
```python
|
|
def evaluator_optimizer_loop(code, max_iterations=3):
|
|
"""
|
|
Evaluator-Optimizer pattern for code review.
|
|
From Anthropic: "One LLM call generates a response while
|
|
another provides evaluation and feedback in a loop."
|
|
"""
|
|
|
|
for iteration in range(max_iterations):
|
|
# Evaluator reviews
|
|
evaluation = Task(
|
|
subagent_type="code-skeptic",
|
|
prompt=code
|
|
)
|
|
|
|
if evaluation.verdict == "APPROVED":
|
|
return {"success": True, "iterations": iteration}
|
|
|
|
# Optimizer fixes
|
|
fixes = Task(
|
|
subagent_type="the-fixer",
|
|
issues=evaluation.issues,
|
|
code=code
|
|
)
|
|
|
|
code = apply_fixes(code, fixes)
|
|
|
|
return {"success": False, "iterations": max_iterations}
|
|
```
|
|
|
|
## Execution Flow
|
|
|
|
```
|
|
[Code Submitted]
|
|
↓
|
|
[Evaluator: code-skeptic]
|
|
↓
|
|
┌─────────────────────┐
|
|
│ │
|
|
│ [APPROVED] ──Yes──→ [PASS]
|
|
│ │
|
|
│ [CHANGES] ──No───→ [Optimizer: the-fixer]
|
|
│ │
|
|
│ ↓
|
|
│ [Apply Fixes]
|
|
│ │
|
|
│ ↓
|
|
│ [Iterations < Max?]
|
|
│ Yes No
|
|
│ ↓ ↓
|
|
│ [Loop] [Escalate]
|
|
└─────────────────────┘
|
|
```
|
|
|
|
## Benefits
|
|
|
|
- **Convergence Guarantee**: Maximum iterations prevent infinite loops
|
|
- **Quality Improvement**: Each iteration improves code
|
|
- **Iterative Feedback**: Evaluators teach optimizers
|
|
|
|
## Configuration
|
|
|
|
```yaml
|
|
# .kilo/config/evaluator-optimizer.yaml
|
|
|
|
iteration_loops:
|
|
code_review:
|
|
evaluator: code-skeptic
|
|
optimizer: the-fixer
|
|
max_iterations: 3
|
|
convergence: all_issues_resolved
|
|
|
|
security_review:
|
|
evaluator: security-auditor
|
|
optimizer: the-fixer
|
|
max_iterations: 2
|
|
convergence: no_critical_vulnerabilities
|
|
```
|
|
|
|
## Usage
|
|
|
|
```
|
|
/workflow evaluator-optimizer --issue 42 --max-iterations 3
|
|
```
|
|
|
|
## Scoring
|
|
|
|
| Iterations | Score Impact |
|
|
|------------|--------------|
|
|
| 1 | Perfect (10/10) |
|
|
| 2 | Good (8/10) |
|
|
| 3 | Acceptable (7/10) |
|
|
| >3 | Poor (triggers prompt-optimizer) |
|