Merge remote-tracking branch 'origin/agent-sync-features'

This commit is contained in:
¨NW¨
2026-04-24 07:21:39 +01:00
19 changed files with 3340 additions and 791 deletions

View File

@@ -1,26 +1,549 @@
# Docker Reference
# Docker & Containerization Rules
Quick reference for Docker, Compose, Swarm. Detailed patterns in `.kilo/skills/docker-*`.
Essential rules for Docker, Docker Compose, Docker Swarm, and container technologies.
## Checklist
## Dockerfile Best Practices
- [ ] Multi-stage builds; order layers least→most frequently changing
- [ ] Run as non-root user; specific image versions (never `latest`)
- [ ] COPY package*.json before COPY . for cache; clean package manager caches
- [ ] Compose 3.8+; environment variables; resource limits; health checks
- [ ] .env for local secrets (gitignored); Docker secrets for Swarm
- [ ] Separated networks (frontend/backend); internal for DB
- [ ] Named volumes with labels; init scripts read-only
- [ ] Swarm: replicated services, rollback config, placement constraints
- [ ] Scan images: `trivy image` or `docker scout vulnerabilities`
- [ ] Logging: json-file driver, max-size/max-file limits
### Layer Optimization
## Common Commands
- Minimize layers by combining commands
- Order layers from least to most frequently changing
- Use multi-stage builds to reduce image size
- Clean up package manager caches
```dockerfile
# ✅ Good: Multi-stage build with layer optimization
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
USER node
EXPOSE 3000
CMD ["node", "server.js"]
# ❌ Bad: Single stage, many layers
FROM node:20
RUN npm install -g nodemon
WORKDIR /app
COPY . .
RUN npm install
EXPOSE 3000
CMD ["nodemon", "server.js"]
```
### Security
- Run as non-root user
- Use specific image versions, not `latest`
- Scan images for vulnerabilities
- Don't store secrets in images
```dockerfile
# ✅ Good
FROM node:20-alpine
RUN addgroup -g 1001 appgroup && \
adduser -u 1001 -G appgroup -D appuser
WORKDIR /app
COPY --chown=appuser:appgroup . .
USER appuser
CMD ["node", "server.js"]
# ❌ Bad
FROM node:latest # Unpredictable version
# Running as root (default)
COPY . .
CMD ["node", "server.js"]
```
### Caching Strategy
```dockerfile
# ✅ Good: Dependencies cached separately
COPY package*.json ./
RUN npm ci
COPY . .
# ❌ Bad: All code copied before dependencies
COPY . .
RUN npm install
```
## Docker Compose
### Service Structure
- Use version 3.8+ for modern features
- Define services in logical order
- Use environment variables for configuration
- Set resource limits
```yaml
# ✅ Good
version: '3.8'
services:
app:
image: myapp:latest
build:
context: .
dockerfile: Dockerfile
environment:
- NODE_ENV=production
- DATABASE_URL=postgres://db:5432/app
depends_on:
db:
condition: service_healthy
networks:
- app-network
deploy:
resources:
limits:
cpus: '0.5'
memory: 512M
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
db:
image: postgres:15-alpine
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: app
POSTGRES_USER: ${DB_USER}
POSTGRES_PASSWORD: ${DB_PASSWORD}
networks:
- app-network
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $POSTGRES_USER"]
interval: 10s
timeout: 5s
retries: 5
networks:
app-network:
driver: bridge
volumes:
postgres-data:
```
### Environment Variables
- Use `.env` files for local development
- Never commit `.env` files with secrets
- Use Docker secrets for sensitive data in Swarm
```bash
docker-compose logs -f app # View logs
docker exec -it app sh # Shell into container
docker stats # Resource usage
docker system prune -a # Clean unused
docker scout vulnerabilities img # Scan
# .env (gitignored)
NODE_ENV=production
DB_PASSWORD=secure_password_here
JWT_SECRET=your_jwt_secret_here
```
```yaml
# docker-compose.yml
services:
app:
env_file:
- .env
# OR explicit for non-sensitive
environment:
- NODE_ENV=production
# Secrets for sensitive data in Swarm
secrets:
- db_password
```
### Network Patterns
```yaml
# ✅ Good: Separated networks for security
networks:
frontend:
driver: bridge
backend:
driver: bridge
internal: true # No external access
services:
web:
networks:
- frontend
- backend
api:
networks:
- backend
db:
networks:
- backend
```
### Volume Management
```yaml
# ✅ Good: Named volumes with labels
volumes:
postgres-data:
driver: local
labels:
- "app=myapp"
- "type=database"
services:
db:
volumes:
- postgres-data:/var/lib/postgresql/data
- ./init-scripts:/docker-entrypoint-initdb.d:ro
```
## Docker Swarm
### Service Deployment
```yaml
# docker-compose.yml (Swarm compatible)
version: '3.8'
services:
api:
image: myapp/api:latest
deploy:
mode: replicated
replicas: 3
update_config:
parallelism: 1
delay: 10s
failure_action: rollback
rollback_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
placement:
constraints:
- node.role == worker
preferences:
- spread: node.id
resources:
limits:
cpus: '0.5'
memory: 512M
reservations:
cpus: '0.25'
memory: 256M
networks:
- app-network
secrets:
- db_password
- jwt_secret
configs:
- app_config
networks:
app-network:
driver: overlay
attachable: true
secrets:
db_password:
external: true
jwt_secret:
external: true
configs:
app_config:
external: true
```
### Stack Deployment
```bash
# Deploy stack
docker stack deploy -c docker-compose.yml mystack
# List services
docker stack services mystack
# Scale service
docker service scale mystack_api=5
# Update service
docker service update --image myapp/api:v2 mystack_api
# Rollback
docker service rollback mystack_api
```
### Health Checks
```yaml
services:
api:
# Health check in Dockerfile
healthcheck:
test: ["CMD", "node", "healthcheck.js"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
# Or in compose
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
interval: 30s
timeout: 10s
retries: 3
```
### Secrets Management
```bash
# Create secret
echo "my_secret_password" | docker secret create db_password -
# Create secret from file
docker secret create jwt_secret ./jwt_secret.txt
# List secrets
docker secret ls
# Use in compose
secrets:
db_password:
external: true
```
### Config Management
```bash
# Create config
docker config create app_config ./config.json
# Use in compose
configs:
app_config:
external: true
services:
api:
configs:
- app_config
```
## Container Security
### Image Security
```bash
# Scan image for vulnerabilities
docker scout vulnerabilities myapp:latest
trivy image myapp:latest
# Check image for secrets
gitleaks --image myapp:latest
```
### Runtime Security
```dockerfile
# ✅ Good: Security measures
FROM node:20-alpine
# Create non-root user
RUN addgroup -g 1001 appgroup && \
adduser -u 1001 -G appgroup -D appuser
# Set read-only filesystem
RUN chmod -R 755 /app && \
chown -R appuser:appgroup /app
WORKDIR /app
COPY --chown=appuser:appgroup . .
# Drop all capabilities
USER appuser
VOLUME ["/tmp"]
CMD ["node", "server.js"]
```
### Network Security
```yaml
# ✅ Good: Limited network access
services:
api:
networks:
- backend
# No ports exposed to host
db:
networks:
- backend
# Internal network only
networks:
backend:
internal: true # No internet access
```
### Resource Limits
```yaml
services:
api:
deploy:
resources:
limits:
cpus: '1.0'
memory: 1G
reservations:
cpus: '0.5'
memory: 512M
```
## Common Patterns
### Development Setup
```yaml
# docker-compose.dev.yml
version: '3.8'
services:
app:
build:
context: .
dockerfile: Dockerfile.dev
volumes:
- .:/app
- /app/node_modules
environment:
- NODE_ENV=development
ports:
- "3000:3000"
command: npm run dev
```
### Production Setup
```yaml
# docker-compose.prod.yml
version: '3.8'
services:
app:
image: myapp:${VERSION}
environment:
- NODE_ENV=production
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
healthcheck:
test: ["CMD", "node", "healthcheck.js"]
interval: 30s
timeout: 10s
retries: 3
```
### Multi-Environment
```bash
# Override files
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
### Logging
```yaml
services:
app:
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
labels: "app,environment"
```
## CI/CD Integration
### Build Pipeline
```yaml
# .github/workflows/docker.yml
name: Docker Build
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Build image
run: docker build -t myapp:${{ github.sha }} .
- name: Scan image
run: trivy image myapp:${{ github.sha }}
- name: Push to registry
run: |
echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USER }} --password-stdin
docker push myapp:${{ github.sha }}
```
## Troubleshooting
### Common Commands
```bash
# View logs
docker-compose logs -f app
# Execute in container
docker-compose exec app sh
# Check health
docker inspect --format='{{.State.Health.Status}}' <container>
# View resource usage
docker stats
# Remove unused resources
docker system prune -a
# Debug network
docker network inspect app-network
# Swarm diagnostics
docker node ls
docker service ps mystack_api
```
## Prohibitions
- DO NOT run containers as root
- DO NOT use `latest` tag in production
- DO NOT expose unnecessary ports
- DO NOT store secrets in images
- DO NOT use privileged mode unnecessarily
- DO NOT mount host directories without restrictions
- DO NOT skip health checks in production
- DO NOT ignore vulnerability scans

View File

@@ -1,283 +1,115 @@
# Evolutionary Sync Rules
# Evolutionary Mode Rules
Rules for synchronizing agent evolution data automatically.
When agents are modified, created, or updated during evolutionary improvement, this rule ensures all related files stay synchronized.
## When to Sync
## Source of Truth
### Automatic Sync Triggers
**`kilo.json`** is the single source of truth for:
- Agent definitions (models, modes, descriptions)
- Command definitions (models, descriptions)
- Categories and groupings
1. **After each completed issue**
- When agent completes task and posts Gitea comment
- Extract performance metrics from comment
## Files to Synchronize
2. **On model change**
- When agent model is updated in kilo.jsonc
- When capability-index.yaml is modified
When agents change, update ALL of these files:
3. **On agent file change**
- When .kilo/agents/*.md files are modified
- On create/delete of agent files
| File | What to Update |
|------|----------------|
| `kilo.json` | Models, modes, descriptions (source of truth) |
| `.kilo/agents/*.md` | Model in YAML frontmatter |
| `.kilo/KILO_SPEC.md` | Pipeline Agents table, Workflow Commands table |
| `AGENTS.md` | Pipeline Agents tables by category |
| `.kilo/agents/orchestrator.md` | Task Tool Invocation table |
4. **On prompt update**
- When agent receives prompt optimization
- Track optimization improvements
## Sync Checklist
### Manual Sync Triggers
```bash
# Sync from all sources
bun run sync:evolution
# Sync specific source
bun run agent-evolution/scripts/sync-agent-history.ts --source git
bun run agent-evolution/scripts/sync-agent-history.ts --source gitea
# Open dashboard
bun run evolution:dashboard
bun run evolution:open
```
## Data Flow
When modifying agents:
```
┌─────────────────────────────────────────────────────────────┐
│ Data Sources │
├─────────────────────────────────────────────────────────────┤
│ .kilo/agents/*.md ──► Parse frontmatter, model │
│ .kilo/kilo.jsonc ──► Model assignments │
│ .kilo/capability-index.yaml ──► Capabilities, routing │
│ Git History ──► Change timeline │
│ Gitea Issue Comments ──► Performance scores │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ agent-evolution/data/ │
│ agent-versions.json │
├─────────────────────────────────────────────────────────────┤
│ { │
│ "agents": { │
│ "lead-developer": { │
│ "current": { model, provider, fit_score, ... }, │
│ "history": [ { model_change, ... } ], │
│ "performance_log": [ { score, issue, ... } ] │
│ } │
│ } │
│ } │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ agent-evolution/index.html │
│ Interactive Dashboard │
├─────────────────────────────────────────────────────────────┤
│ • Overview - Stats, recent changes, recommendations │
│ • All Agents - Filterable cards with history │
│ • Timeline - Full evolution history │
│ • Recommendations - Export, priority-based view │
│ • Model Matrix - Agent × Model mapping │
└─────────────────────────────────────────────────────────────┘
□ Update kilo.json with new model/description
□ Update agent .md file frontmatter
□ Update KILO_SPEC.md Pipeline Agents table
□ Update AGENTS.md category tables
□ Update orchestrator.md subagent_type mappings (if new agent)
□ Run scripts/sync-agents.js --check to verify
```
## Recording Changes
## Adding New Agent
### From Gitea Comments
Agent comments should follow this format:
```markdown
## ✅ agent-name completed
**Score**: X/10
**Duration**: X.Xh
**Files**: file1.ts, file2.ts
### Notes
- Description of work done
- Key decisions made
- Issues encountered
```
Extraction:
- `agent-name` → agent name
- `Score` → performance score (1-10)
- `Duration` → execution time
- `Files` → files modified
### From Git Commits
Commit message patterns:
- `feat: add flutter-developer agent` → agent_created
- `fix: update security-auditor model to nemotron-3-super` → model_change
- `docs: update lead-developer prompt` → prompt_change
## Gitea Webhook Setup
1. **Create webhook in Gitea**
- Target URL: `http://localhost:3000/api/evolution/webhook`
- Events: `issue_comment`, `issues`
2. **Webhook payload handling**
```typescript
// In agent-evolution/scripts/gitea-webhook.ts
app.post('/api/evolution/webhook', async (req, res) => {
const { action, issue, comment } = req.body;
if (action === 'created' && comment?.body.includes('## ✅')) {
await recordAgentPerformance(issue, comment);
}
res.json({ success: true });
});
1. Create `.kilo/agents/agent-name.md` with frontmatter:
```yaml
---
description: Agent description
mode: subagent|primary|all
model: provider/model-id
color: #HEX
permission:
read: allow
edit: allow
...
---
```
## Performance Metrics
### Tracked Metrics
For each agent execution:
| Metric | Source | Format |
|--------|--------|--------|
| Score | Gitea comment | X/10 |
| Duration | Agent timing | milliseconds |
| Success | Exit status | boolean |
| Files | Gitea comment | count |
| Issue | Context | number |
### Aggregated Metrics
| Metric | Calculation | Use |
|--------|-------------|-----|
| Average Score | `sum(scores) / count` | Agent effectiveness |
| Success Rate | `successes / total * 100` | Reliability |
| Average Duration | `sum(durations) / count` | Speed |
| Files per Task | `sum(files) / count` | Scope |
## Recommendations Generation
### Priority Levels
| Priority | Criteria | Action |
|----------|----------|--------|
| Critical | Fit score < 70 | Immediate update |
| High | Model unavailable | Switch to fallback |
| Medium | Better model available | Consider upgrade |
| Low | Optimization possible | Optional improvement |
### Example Recommendation
```json
{
"agent": "requirement-refiner",
"recommendations": [{
"target": "ollama-cloud/nemotron-3-super",
"reason": "+22% quality, 1M context for specifications",
"priority": "critical"
}]
}
```
## Evolution Rules
### When Model Change is Recorded
1. **Detect change**
- Compare current.model with previous value
- Extract reason from commit message
2. **Record in history**
2. Add to `kilo.json` under `agents`:
```json
{
"date": "2026-04-05T05:21:00Z",
"commit": "caf77f53c8",
"type": "model_change",
"from": "ollama-cloud/gpt-oss:120b",
"to": "ollama-cloud/nemotron-3-super",
"reason": "Better reasoning for security analysis"
"agent-name": {
"file": ".kilo/agents/agent-name.md",
"description": "Full description",
"model": "provider/model-id",
"mode": "subagent",
"category": "core|quality|meta|cognitive|testing"
}
```
3. **Update current**
- Set current.model to new value
- Update provider if changed
- Recalculate fit score
3. If subagent, add to `orchestrator.md`:
- Add to permission list
- Add to Task Tool Invocation table
### When Performance Drops
4. Run sync script:
```bash
node scripts/sync-agents.js --fix
```
1. **Detect pattern**
- Last 5 scores average < 7
- Success rate < 80%
## Model Changes
2. **Generate recommendation**
- Suggest model upgrade
- Trigger prompt-optimizer
When changing a model:
3. **Notify via Gitea comment**
- Post to related issue
- Include improvement suggestions
1. Update agent file frontmatter
2. Update `kilo.json`
3. Update `KILO_SPEC.md`
4. Document reason in commit message
## Integration in Pipeline
Example:
```
fix: update LeadDeveloper model from qwen3-coder:free to qwen3-coder:480b
Add to post-pipeline:
```yaml
# .kilo/commands/pipeline.md
post_steps:
- name: sync_evolution
run: bun run sync:evolution
- name: check_recommendations
run: bun run agent-evolution/scripts/check-recommendations.ts
Reason: Better code generation quality, supports larger context
```
## Dashboard Access
## Verification
Run sync verification before commits:
```bash
# Start local server
bun run evolution:dashboard
# Check only (CI mode)
node scripts/sync-agents.js --check
# Open in browser
bun run evolution:open
# or visit http://localhost:3001
# Fix discrepancies
node scripts/sync-agents.js --fix
```
## API Endpoints (Future)
## CI Integration
```typescript
// GET /api/evolution/agents
// Returns all agents with current state
Add to `.github/workflows/ci.yml`:
// GET /api/evolution/agents/:name/history
// Returns agent history
// GET /api/evolution/recommendations
// Returns pending recommendations
// POST /api/evolution/agents/:name/apply
// Apply recommendation
// POST /api/evolution/sync
// Trigger manual sync
```yaml
- name: Verify Agent Sync
run: node scripts/sync-agents.js --check
```
## Best Practices
## Prohibited Actions
1. **Sync after every pipeline run**
- Captures model changes
- Records performance
2. **Review dashboard weekly**
- Check pending recommendations
- Apply critical updates
3. **Track before/after metrics**
- When applying changes
- Compare performance
4. **Keep history clean**
- Deduplicate entries
- Merge related changes
5. **Use consistent naming**
- Agent names match file names
- Model IDs match capability-index.yaml
- DO NOT update KILO_SPEC.md without updating kilo.json
- DO NOT update agent model without updating all sync targets
- DO NOT add new agent without updating orchestrator permissions
- DO NOT skip running sync script after changes