GoClaw/todo.md

GoClaw Control Center TODO

• Basic Dashboard layout (Mission Control theme)
• Agents page with mock data
• Nodes page with mock data
• Chat page with mock conversation
• Settings page with provider cards
• Docker Stack integration
• Fix Home.tsx conflict after upgrade
• Fix DashboardLayout.tsx conflict after upgrade
• Create server-side Ollama API proxy routes (tRPC)
• Integrate real Ollama /v1/models endpoint in Settings
• Integrate real Ollama /v1/chat/completions in Chat page
• Add OLLAMA_API_KEY and OLLAMA_BASE_URL secrets
• Write vitest tests for Ollama API proxy
• Update Dashboard with real model data
• Add streaming support for chat responses
• Connect real Docker Swarm API for node monitoring
• Add authentication/login protection

Phase 1: Agent Management UI

• Connect Agents page to trpc.agents.list (load real agents from DB)
• Create AgentDetailModal component for viewing agent config
• Create AgentCreateModal component with form validation
• Implement agent update mutation (model, temperature, maxTokens, systemPrompt)
• Implement agent delete mutation with confirmation
• Add start/pause/restart actions for agents
• Add agent metrics chart (requests, tokens, processing time)
• Add agent history view (recent requests/responses)
• Write vitest tests for agent management components

Phase 2: Tool Binding System

• Design Tool Binding API schema
• Create tool registry in database
• Implement tool execution sandbox
• Add tool access control per agent
• Create UI for tool management

Phase 3: Tool Integration

• Implement Browser tool (HTTP fetch-based)
• Implement Shell tool (bash execution with safety checks)
• Implement File tool (read/write with path restrictions)
• Implement Docker tool (container management)
• Implement HTTP tool (GET/POST with domain whitelist)

Phase 4: Metrics & History

• AgentMetrics page with request timeline chart
• Conversation history log per agent
• Raw metrics table with token/time data
• Stats cards (total requests, success rate, avg response time, tokens)
• Time range selector (6h/24h/48h/7d)
• Metrics button on agent cards
• Navigation: /agents/:id/metrics route
• Tools page added to sidebar navigation

Phase 5: Specialized Agents

Browser Agent

• Install puppeteer-core + chromium dependencies
• Create server/browser-agent.ts — Puppeteer session manager
• tRPC routes: browser.start, browser.navigate, browser.screenshot, browser.click, browser.type, browser.extract, browser.close
• BrowserAgent.tsx page — live browser control UI with screenshot preview
• Session management: multiple concurrent browser sessions per agent
• Add browser_agent to agents DB as pre-seeded entry

Tool Builder Agent

• Create server/tool-builder.ts — LLM-powered tool generator
• tRPC routes: toolBuilder.generate, toolBuilder.validate, toolBuilder.install
• Dynamic tool registration: add generated tools to TOOL_REGISTRY at runtime
• Persist custom tools to DB (tool_definitions table)
• ToolBuilder.tsx page — describe tool → preview code → install
• Add tool_builder_agent to agents DB as pre-seeded entry

Agent Compiler

• Create server/agent-compiler.ts — LLM-powered agent factory
• tRPC routes: agentCompiler.compile, agentCompiler.preview, agentCompiler.deploy
• AgentCompiler.tsx page — ТЗ input → agent config preview → deploy
• Auto-populate: model, role, systemPrompt, allowedTools from ТЗ
• Add agent_compiler to agents DB as pre-seeded entry

Integration

• Add all 3 pages to sidebar navigation
• Write vitest tests for all new server modules
• Push to Gitea (NW)

Phase 6: Agents as Real Chat Entities

• Remove unused pages: BrowserAgent.tsx, ToolBuilder.tsx, AgentCompiler.tsx
• Seed 3 agents into DB: Browser Agent, Tool Builder Agent, Agent Compiler
• Add tRPC chat endpoint: agents.chat (LLM + tool execution per agent)
• Update Chat UI to support agent selection dropdown
• Create /skills page — skills registry with install/uninstall
• Update /agents to show seeded agents with Chat button
• Update /tools to show tools per agent with filter by agent
• Add /skills to sidebar navigation
• Write tests for chat and skills endpoints

Phase 6: Orchestrator Agent (Main Chat)

• Fix TS errors: browserSessions/toolDefinitions schema exports, z.record
• Seed 3 specialized agents into DB (Browser, Tool Builder, Agent Compiler)
• Create server/orchestrator.ts — main orchestrator with tool-use loop
• Orchestrator tools: shell_exec, file_read, file_write, http_request, delegate_to_agent, list_agents, list_skills, install_skill
• Add trpc.orchestrator.chat mutation (multi-step tool-use loop with LLM)
• Update /chat UI: show tool call steps, agent delegation, streaming response
• Create /skills page with skill registry (install/remove/describe)
• Add /skills to sidebar navigation
• Update /agents to show seeded agents with Chat button
• Write tests for orchestrator

Phase 7: Orchestrator as Configurable System Agent

• Add isSystem + isOrchestrator fields to agents table (DB migration)
• Seed Orchestrator as system agent in DB (role=orchestrator, isSystem=true)
• Update orchestrator.ts to load model/systemPrompt/allowedTools from DB
• Update /chat to read orchestrator config from DB, show active model in header
• Update /agents to show Orchestrator with SYSTEM badge, Configure button, no delete
• AgentDetailModal: orchestrator gets extra tab with system tools (shell, docker, agents mgmt)
• Add system tools to orchestrator: docker_ps, docker_restart, manage_agents, read_logs
• /chat header: show current model name + link to Configure Orchestrator

Phase 8: Fix Orchestrator Chat

• Fix: orchestrator uses model from DB config (minimax-m2.7, not hardcoded fallback)
• Fix: real tool-use loop — execute shell_exec, file_read, file_list tools
• Fix: show tool call steps in Chat UI (tool name, args, result, duration)
• Fix: Chat.tsx shows which model is being used from orchestrator config
• Fix: Streamdown markdown rendering for assistant responses
• Add: streaming/SSE for real-time response display

Phase 9: Go Gateway Migration (Variant C)

• Create gateway/ directory with Go module (git.softuniq.eu/UniqAI/GoClaw/gateway)
• Implement config/config.go — env-based configuration
• Implement internal/llm/client.go — Ollama API client (chat, models, health)
• Implement internal/db/db.go — MySQL connection, agent/config queries
• Implement internal/tools/executor.go — Tool Executor (shell_exec, file_read, file_write, file_list, http_request, docker_exec, list_agents)
• Implement internal/orchestrator/orchestrator.go — LLM tool-use loop, config from DB
• Implement internal/api/handlers.go — REST API handlers
• Implement cmd/gateway/main.go — HTTP server with chi router, graceful shutdown
• Go Gateway compiles successfully (10.8MB binary)
• Create server/gateway-proxy.ts — Node.js proxy client to Go Gateway
• Create docker/docker-compose.yml — local dev (control-center + gateway + ollama + db)
• Create docker/docker-stack.yml — Docker Swarm production (2 replicas, rolling updates)
• Create docker/Dockerfile.gateway — multi-stage Go build
• Create docker/Dockerfile.control-center — multi-stage Node.js build
• Update server/routers.ts: replace orchestrator.ts calls with gateway-proxy.ts calls
• Write Go unit tests (gateway/internal/tools/executor_test.go)
• Write Go integration test for orchestrator chat loop
• Push to Gitea (NW)

Phase 10: LLM Provider Configuration

• config.go: default LLM_BASE_URL = https://ollama.com/v1 (Ollama Cloud)
• config.go: support LLM_BASE_URL + LLM_API_KEY env vars (legacy OLLAMA_* aliases kept)
• config.go: normaliseLLMURL() — auto-append /v1 for bare Ollama hosts
• docker-compose.yml: ollama service commented out (GPU only), LLM_BASE_URL/LLM_API_KEY added
• docker-stack.yml: ollama service commented out (GPU only), llm-api-key secret added
• docker/.env.example: 4 LLM provider options documented (Ollama Cloud, OpenAI, Groq, Local GPU)

Phase 11: Frontend → Go Gateway Integration

• gateway-proxy.ts: fix getGatewayTools() — map OpenAI format {type,function:{name,...}} to GatewayToolDef
• gateway-proxy.ts: add executeGatewayTool(), getGatewayAgent(), isGatewayAvailable() methods
• routers.ts: orchestrator.getConfig — Go Gateway first, Node.js fallback
• routers.ts: orchestrator.chat — Go Gateway first, Node.js fallback
• routers.ts: orchestrator.tools — Go Gateway first, Node.js fallback
• routers.ts: orchestrator.gatewayHealth — new endpoint for UI status
• routers.ts: ollama.health — Go Gateway first, direct Ollama fallback
• routers.ts: ollama.models — Go Gateway first, direct Ollama fallback
• gateway/db.go: TLS auto-detection for TiDB Cloud (tidbcloud/aws/gcp/azure hosts)
• server/gateway-proxy.test.ts: 13 vitest tests (health, config, tools, mapping)
• End-to-end test: orchestrator.chat via tRPC → Go Gateway → Ollama (source: "gateway")
• End-to-end test: tool calling — file_list tool executed by Go Gateway

Phase 12: Real-time Nodes Page

• Add Docker API client in Go Gateway: /api/nodes endpoint with real node data
• Add /api/nodes/stats endpoint for CPU/memory per node
• Add tRPC nodes.list and nodes.stats procedures via gateway-proxy
• Update Nodes.tsx: real data from tRPC + auto-refresh every 5 seconds
• Show: node ID, hostname, status, role (manager/worker), availability, CPU, RAM, Docker version, IP
• Show live indicator (green pulse) when data is fresh
• Deploy to server 2.59.219.61
• Docker API client: /api/nodes, /api/nodes/stats
• tRPC nodes.list, nodes.stats procedures
• Nodes.tsx rewritten with real data + auto-refresh 10s/15s
• 14 vitest tests for nodes procedures

Phase 13: Seed Data for Agents & Orchestrator

• Create server/seed.ts with default agents (orchestrator, coder, browser, researcher)
• Create default orchestrator config seed
• Integrate seed into server startup (idempotent — runs only when tables are empty)
• Write vitest tests for seed logic (18 tests, all pass)
• Commit to Gitea and deploy to production server
• Verify seed data on production DB — 6 agents seeded successfully

Phase 14: Auto-migrate on Container Startup

• Create server/migrate.ts — programmatic Drizzle migration runner
• Create docker/entrypoint.sh — wait-for-db + migrate + start server
• Update Dockerfile.control-center — copy entrypoint, set as CMD
• Write vitest tests for migrate logic
• Commit to Gitea and deploy to production server
• Verify auto-migrate on production (check logs)

Phase 14 (Bug Fixes): Real Header Metrics + Seed Fix

• Fix seed: agents not appearing on production after restart (check isSystem column query)
• Fix header metrics: UPTIME/NODES/AGENTS/CPU/MEM show hardcoded data instead of real values
• Connect header stats to real tRPC endpoints (agents count from DB, nodes/CPU/MEM from Docker API)
• Write vitest tests for header stats procedure (82 tests total, all pass)
• Commit to Gitea and deploy to production (Phase 16) — verified: auto-migrate ran, seed skipped (6 agents exist), metrics-collector started, nodes.metricsHistory endpoint ready — verified: nodes=6/6, agents=6, CPU=0.2%, MEM=645MB, gatewayOnline=true

Phase 15 (Bug Fix): Agents Page Shows Empty List

• Diagnose: find why /agents page shows no agents (userId=0 in seed vs SYSTEM_USER_ID=1 in router)
• Fix agents tRPC query: getAllAgents() instead of getUserAgents(SYSTEM_USER_ID)
• Update vitest tests (86 tests, all pass)
• Deploy to production (Phase 15) — verified: 6 agents visible (GoClaw Orchestrator, Browser Agent, Tool Builder, Agent Compiler, Coder Agent, Researcher)

Phase 16: Auto-migrate + Historical Metrics + Alerts

• Create docker/entrypoint.sh with drizzle-kit migrate before server start
• Update Dockerfile.control-center to use entrypoint.sh
• Add nodeMetrics table to drizzle/schema.ts and run pnpm db:push
• Add db helpers: saveNodeMetric, getNodeMetricsHistory in server/db.ts
• Add tRPC endpoint: nodes.metricsHistory (last 1h per container)
• Add background job: collect CPU/MEM every 30s, alert on CPU>80% or unhealthy
• Update Nodes.tsx: sparkline charts per container card (recharts)
• Write vitest tests for new components
• Commit to Gitea and deploy to production

Phase 17: Chat Resilience & Retry Logic

• Diagnose: find why chat interrupts (timeout, LLM error, Gateway unavailable)
• Create server/chat-resilience.ts: retryWithBackoff, exponential backoff, error classification
• Add retry logic to orchestrator.chat with exponential backoff (3 attempts, 1s/2s/4s)
• Update Chat.tsx: retry state, auto-retry on network errors, retry indicator
• Write vitest tests for retry logic (17 tests, all pass — 103 total tests pass)
• Commit to Gitea and deploy to production (Phase 17)

Phase 19: Complete Task Management System & Final Integration

• Phase 19.1: Add tasks table to drizzle/schema.ts with full schema
• Phase 19.2: Create query helpers in server/db.ts (createTask, getAgentTasks, etc)
• Phase 19.3: Create tRPC endpoints in server/routers.ts (tasks.create, tasks.list, etc)
• Phase 19.4: Create TasksPanel React component for right sidebar
• Phase 19.5: Add auto-task creation functions in orchestrator.ts
• Phase 19.6: Integrate TasksPanel into Chat UI with conversationId tracking
• Phase 19.7: Write vitest tests for tasks (107 tests pass, 1 fails due to missing DB table)
• Phase 19.8: Integrate auto-task creation into orchestratorChat loop
• Phase 19.9: Create Web Research Workflow (server/web-research.ts)
• Phase 19.10: Add research tRPC endpoints (search, compileReport, createTasks)
• Phase 19.11: Create WebResearchPanel React component
• Phase 19.12: Write vitest tests for Web Research (120 tests pass, 1 fails due to missing DB table)
• Phase 19.13: Add research tool to ORCHESTRATOR_TOOLS
• Phase 19.14: Add research case to executeTool function
• Phase 19.15: Create production migration script (docker/migrate-production.sh)
• Phase 19.16: Create PRODUCTION_DEPLOYMENT.md documentation
• Phase 19.17: Run full test suite (120/121 tests pass)
• Phase 19.18: Commit to Gitea with NW authorship
• Phase 19.19: Deploy to production and verify