GoClaw/todo.md

# GoClaw Control Center TODO

- [x] Basic Dashboard layout (Mission Control theme)
- [x] Agents page with mock data
- [x] Nodes page with mock data
- [x] Chat page with mock conversation
- [x] Settings page with provider cards
- [x] Docker Stack integration
- [x] Fix Home.tsx conflict after upgrade
- [x] Fix DashboardLayout.tsx conflict after upgrade
- [x] Create server-side Ollama API proxy routes (tRPC)
- [x] Integrate real Ollama /v1/models endpoint in Settings
- [x] Integrate real Ollama /v1/chat/completions in Chat page
- [x] Add OLLAMA_API_KEY and OLLAMA_BASE_URL secrets
- [x] Write vitest tests for Ollama API proxy
- [x] Update Dashboard with real model data
- [ ] Add streaming support for chat responses
- [ ] Connect real Docker Swarm API for node monitoring
- [ ] Add authentication/login protection

## Phase 1: Agent Management UI
- [x] Connect Agents page to trpc.agents.list (load real agents from DB)
- [x] Create AgentDetailModal component for viewing agent config
- [x] Create AgentCreateModal component with form validation
- [x] Implement agent update mutation (model, temperature, maxTokens, systemPrompt)
- [x] Implement agent delete mutation with confirmation
- [x] Add start/pause/restart actions for agents
- [x] Add agent metrics chart (requests, tokens, processing time)
- [x] Add agent history view (recent requests/responses)
- [x] Write vitest tests for agent management components

## Phase 2: Tool Binding System
- [x] Design Tool Binding API schema
- [x] Create tool registry in database
- [x] Implement tool execution sandbox
- [x] Add tool access control per agent
- [x] Create UI for tool management

## Phase 3: Tool Integration
- [x] Implement Browser tool (HTTP fetch-based)
- [x] Implement Shell tool (bash execution with safety checks)
- [x] Implement File tool (read/write with path restrictions)
- [x] Implement Docker tool (container management)
- [x] Implement HTTP tool (GET/POST with domain whitelist)

## Phase 4: Metrics & History
- [x] AgentMetrics page with request timeline chart
- [x] Conversation history log per agent
- [x] Raw metrics table with token/time data
- [x] Stats cards (total requests, success rate, avg response time, tokens)
- [x] Time range selector (6h/24h/48h/7d)
- [x] Metrics button on agent cards
- [x] Navigation: /agents/:id/metrics route
- [x] Tools page added to sidebar navigation

## Phase 5: Specialized Agents

### Browser Agent
- [ ] Install puppeteer-core + chromium dependencies
- [ ] Create server/browser-agent.ts — Puppeteer session manager
- [ ] tRPC routes: browser.start, browser.navigate, browser.screenshot, browser.click, browser.type, browser.extract, browser.close
- [ ] BrowserAgent.tsx page — live browser control UI with screenshot preview
- [ ] Session management: multiple concurrent browser sessions per agent
- [ ] Add browser_agent to agents DB as pre-seeded entry

### Tool Builder Agent
- [ ] Create server/tool-builder.ts — LLM-powered tool generator
- [ ] tRPC routes: toolBuilder.generate, toolBuilder.validate, toolBuilder.install
- [ ] Dynamic tool registration: add generated tools to TOOL_REGISTRY at runtime
- [ ] Persist custom tools to DB (tool_definitions table)
- [ ] ToolBuilder.tsx page — describe tool → preview code → install
- [ ] Add tool_builder_agent to agents DB as pre-seeded entry

### Agent Compiler
- [ ] Create server/agent-compiler.ts — LLM-powered agent factory
- [ ] tRPC routes: agentCompiler.compile, agentCompiler.preview, agentCompiler.deploy
- [ ] AgentCompiler.tsx page — ТЗ input → agent config preview → deploy
- [ ] Auto-populate: model, role, systemPrompt, allowedTools from ТЗ
- [ ] Add agent_compiler to agents DB as pre-seeded entry

### Integration
- [ ] Add all 3 pages to sidebar navigation
- [ ] Write vitest tests for all new server modules
- [ ] Push to Gitea (NW)

## Phase 6: Agents as Real Chat Entities
- [ ] Remove unused pages: BrowserAgent.tsx, ToolBuilder.tsx, AgentCompiler.tsx
- [ ] Seed 3 agents into DB: Browser Agent, Tool Builder Agent, Agent Compiler
- [ ] Add tRPC chat endpoint: agents.chat (LLM + tool execution per agent)
- [ ] Update Chat UI to support agent selection dropdown
- [ ] Create /skills page — skills registry with install/uninstall
- [ ] Update /agents to show seeded agents with Chat button
- [ ] Update /tools to show tools per agent with filter by agent
- [ ] Add /skills to sidebar navigation
- [ ] Write tests for chat and skills endpoints

## Phase 6: Orchestrator Agent (Main Chat)
- [x] Fix TS errors: browserSessions/toolDefinitions schema exports, z.record
- [x] Seed 3 specialized agents into DB (Browser, Tool Builder, Agent Compiler)
- [x] Create server/orchestrator.ts — main orchestrator with tool-use loop
- [x] Orchestrator tools: shell_exec, file_read, file_write, http_request, delegate_to_agent, list_agents, list_skills, install_skill
- [x] Add trpc.orchestrator.chat mutation (multi-step tool-use loop with LLM)
- [x] Update /chat UI: show tool call steps, agent delegation, streaming response
- [x] Create /skills page with skill registry (install/remove/describe)
- [x] Add /skills to sidebar navigation
- [x] Update /agents to show seeded agents with Chat button
- [ ] Write tests for orchestrator

## Phase 7: Orchestrator as Configurable System Agent
- [x] Add isSystem + isOrchestrator fields to agents table (DB migration)
- [x] Seed Orchestrator as system agent in DB (role=orchestrator, isSystem=true)
- [x] Update orchestrator.ts to load model/systemPrompt/allowedTools from DB
- [x] Update /chat to read orchestrator config from DB, show active model in header
- [x] Update /agents to show Orchestrator with SYSTEM badge, Configure button, no delete
- [x] AgentDetailModal: orchestrator gets extra tab with system tools (shell, docker, agents mgmt)
- [x] Add system tools to orchestrator: docker_ps, docker_restart, manage_agents, read_logs
- [x] /chat header: show current model name + link to Configure Orchestrator

## Phase 8: Fix Orchestrator Chat
- [x] Fix: orchestrator uses model from DB config (minimax-m2.7, not hardcoded fallback)
- [x] Fix: real tool-use loop — execute shell_exec, file_read, file_list tools
- [x] Fix: show tool call steps in Chat UI (tool name, args, result, duration)
- [x] Fix: Chat.tsx shows which model is being used from orchestrator config
- [x] Fix: Streamdown markdown rendering for assistant responses
- [ ] Add: streaming/SSE for real-time response display

## Phase 9: Go Gateway Migration (Variant C)
- [x] Create gateway/ directory with Go module (git.softuniq.eu/UniqAI/GoClaw/gateway)
- [x] Implement config/config.go — env-based configuration
- [x] Implement internal/llm/client.go — Ollama API client (chat, models, health)
- [x] Implement internal/db/db.go — MySQL connection, agent/config queries
- [x] Implement internal/tools/executor.go — Tool Executor (shell_exec, file_read, file_write, file_list, http_request, docker_exec, list_agents)
- [x] Implement internal/orchestrator/orchestrator.go — LLM tool-use loop, config from DB
- [x] Implement internal/api/handlers.go — REST API handlers
- [x] Implement cmd/gateway/main.go — HTTP server with chi router, graceful shutdown
- [x] Go Gateway compiles successfully (10.8MB binary)
- [x] Create server/gateway-proxy.ts — Node.js proxy client to Go Gateway
- [x] Create docker/docker-compose.yml — local dev (control-center + gateway + ollama + db)
- [x] Create docker/docker-stack.yml — Docker Swarm production (2 replicas, rolling updates)
- [x] Create docker/Dockerfile.gateway — multi-stage Go build
- [x] Create docker/Dockerfile.control-center — multi-stage Node.js build
- [ ] Update server/routers.ts: replace orchestrator.ts calls with gateway-proxy.ts calls
- [ ] Write Go unit tests (gateway/internal/tools/executor_test.go)
- [ ] Write Go integration test for orchestrator chat loop
- [ ] Push to Gitea (NW)

## Phase 10: LLM Provider Configuration
- [x] config.go: default LLM_BASE_URL = https://ollama.com/v1 (Ollama Cloud)
- [x] config.go: support LLM_BASE_URL + LLM_API_KEY env vars (legacy OLLAMA_* aliases kept)
- [x] config.go: normaliseLLMURL() — auto-append /v1 for bare Ollama hosts
- [x] docker-compose.yml: ollama service commented out (GPU only), LLM_BASE_URL/LLM_API_KEY added
- [x] docker-stack.yml: ollama service commented out (GPU only), llm-api-key secret added
- [x] docker/.env.example: 4 LLM provider options documented (Ollama Cloud, OpenAI, Groq, Local GPU)

## Phase 11: Frontend → Go Gateway Integration
- [x] gateway-proxy.ts: fix getGatewayTools() — map OpenAI format {type,function:{name,...}} to GatewayToolDef
- [x] gateway-proxy.ts: add executeGatewayTool(), getGatewayAgent(), isGatewayAvailable() methods
- [x] routers.ts: orchestrator.getConfig — Go Gateway first, Node.js fallback
- [x] routers.ts: orchestrator.chat — Go Gateway first, Node.js fallback
- [x] routers.ts: orchestrator.tools — Go Gateway first, Node.js fallback
- [x] routers.ts: orchestrator.gatewayHealth — new endpoint for UI status
- [x] routers.ts: ollama.health — Go Gateway first, direct Ollama fallback
- [x] routers.ts: ollama.models — Go Gateway first, direct Ollama fallback
- [x] gateway/db.go: TLS auto-detection for TiDB Cloud (tidbcloud/aws/gcp/azure hosts)
- [x] server/gateway-proxy.test.ts: 13 vitest tests (health, config, tools, mapping)
- [x] End-to-end test: orchestrator.chat via tRPC → Go Gateway → Ollama (source: "gateway")
- [x] End-to-end test: tool calling — file_list tool executed by Go Gateway

## Phase 12: Real-time Nodes Page
- [ ] Add Docker API client in Go Gateway: /api/nodes endpoint with real node data
- [ ] Add /api/nodes/stats endpoint for CPU/memory per node
- [ ] Add tRPC nodes.list and nodes.stats procedures via gateway-proxy
- [ ] Update Nodes.tsx: real data from tRPC + auto-refresh every 5 seconds
- [ ] Show: node ID, hostname, status, role (manager/worker), availability, CPU, RAM, Docker version, IP
- [ ] Show live indicator (green pulse) when data is fresh
- [x] Deploy to server 2.59.219.61
- [x] Docker API client: /api/nodes, /api/nodes/stats
- [x] tRPC nodes.list, nodes.stats procedures
- [x] Nodes.tsx rewritten with real data + auto-refresh 10s/15s
- [x] 14 vitest tests for nodes procedures

## Phase 13: Seed Data for Agents & Orchestrator
- [x] Create server/seed.ts with default agents (orchestrator, coder, browser, researcher)
- [x] Create default orchestrator config seed
- [x] Integrate seed into server startup (idempotent — runs only when tables are empty)
- [x] Write vitest tests for seed logic (18 tests, all pass)
- [x] Commit to Gitea and deploy to production server
- [x] Verify seed data on production DB — 6 agents seeded successfully

## Phase 14: Auto-migrate on Container Startup
- [ ] Create server/migrate.ts — programmatic Drizzle migration runner
- [ ] Create docker/entrypoint.sh — wait-for-db + migrate + start server
- [ ] Update Dockerfile.control-center — copy entrypoint, set as CMD
- [ ] Write vitest tests for migrate logic
- [ ] Commit to Gitea and deploy to production server
- [ ] Verify auto-migrate on production (check logs)

## Phase 14 (Bug Fixes): Real Header Metrics + Seed Fix
- [x] Fix seed: agents not appearing on production after restart (check isSystem column query)
- [x] Fix header metrics: UPTIME/NODES/AGENTS/CPU/MEM show hardcoded data instead of real values
- [x] Connect header stats to real tRPC endpoints (agents count from DB, nodes/CPU/MEM from Docker API)
- [x] Write vitest tests for header stats procedure (82 tests total, all pass)
- [x] Commit to Gitea and deploy to production (Phase 16) — verified: auto-migrate ran, seed skipped (6 agents exist), metrics-collector started, nodes.metricsHistory endpoint ready — verified: nodes=6/6, agents=6, CPU=0.2%, MEM=645MB, gatewayOnline=true

## Phase 15 (Bug Fix): Agents Page Shows Empty List
- [x] Diagnose: find why /agents page shows no agents (userId=0 in seed vs SYSTEM_USER_ID=1 in router)
- [x] Fix agents tRPC query: getAllAgents() instead of getUserAgents(SYSTEM_USER_ID)
- [x] Update vitest tests (86 tests, all pass)
- [x] Deploy to production (Phase 15) — verified: 6 agents visible (GoClaw Orchestrator, Browser Agent, Tool Builder, Agent Compiler, Coder Agent, Researcher)

## Phase 16: Auto-migrate + Historical Metrics + Alerts
- [ ] Create docker/entrypoint.sh with drizzle-kit migrate before server start
- [ ] Update Dockerfile.control-center to use entrypoint.sh
- [ ] Add nodeMetrics table to drizzle/schema.ts and run pnpm db:push
- [ ] Add db helpers: saveNodeMetric, getNodeMetricsHistory in server/db.ts
- [ ] Add tRPC endpoint: nodes.metricsHistory (last 1h per container)
- [ ] Add background job: collect CPU/MEM every 30s, alert on CPU>80% or unhealthy
- [ ] Update Nodes.tsx: sparkline charts per container card (recharts)
- [ ] Write vitest tests for new components
- [ ] Commit to Gitea and deploy to production

## Phase 17: Chat Resilience & Retry Logic
- [x] Diagnose: find why chat interrupts (timeout, LLM error, Gateway unavailable)
- [x] Create server/chat-resilience.ts: retryWithBackoff, exponential backoff, error classification
- [x] Add retry logic to orchestrator.chat with exponential backoff (3 attempts, 1s/2s/4s)
- [x] Update Chat.tsx: retry state, auto-retry on network errors, retry indicator
- [x] Write vitest tests for retry logic (17 tests, all pass — 103 total tests pass)
- [ ] Commit to Gitea and deploy to production (Phase 17)


## Phase 19: Complete Task Management System & Final Integration
- [x] Phase 19.1: Add tasks table to drizzle/schema.ts with full schema
- [x] Phase 19.2: Create query helpers in server/db.ts (createTask, getAgentTasks, etc)
- [x] Phase 19.3: Create tRPC endpoints in server/routers.ts (tasks.create, tasks.list, etc)
- [x] Phase 19.4: Create TasksPanel React component for right sidebar
- [x] Phase 19.5: Add auto-task creation functions in orchestrator.ts
- [x] Phase 19.6: Integrate TasksPanel into Chat UI with conversationId tracking
- [x] Phase 19.7: Write vitest tests for tasks (107 tests pass, 1 fails due to missing DB table)
- [x] Phase 19.8: Integrate auto-task creation into orchestratorChat loop
- [x] Phase 19.9: Create Web Research Workflow (server/web-research.ts)
- [x] Phase 19.10: Add research tRPC endpoints (search, compileReport, createTasks)
- [x] Phase 19.11: Create WebResearchPanel React component
- [x] Phase 19.12: Write vitest tests for Web Research (120 tests pass, 1 fails due to missing DB table)
- [x] Phase 19.13: Add research tool to ORCHESTRATOR_TOOLS
- [x] Phase 19.14: Add research case to executeTool function
- [x] Phase 19.15: Create production migration script (docker/migrate-production.sh)
- [x] Phase 19.16: Create PRODUCTION_DEPLOYMENT.md documentation
- [x] Phase 19.17: Run full test suite (120/121 tests pass)
- [ ] Phase 19.18: Commit to Gitea with NW authorship
- [ ] Phase 19.19: Deploy to production and verify