Checkpoint: Phase 9: Go Gateway — полный перенос оркестратора и tool executor на Go. Добавлены gateway/ (Go), docker/ (docker-compose + stack + Dockerfiles), server/gateway-proxy.ts
This commit is contained in:
60
docker/Dockerfile.control-center
Normal file
60
docker/Dockerfile.control-center
Normal file
@@ -0,0 +1,60 @@
|
||||
# ── Build Stage ──────────────────────────────────────────────────────────────
|
||||
FROM node:22-alpine AS builder
|
||||
|
||||
# Install pnpm
|
||||
RUN npm install -g pnpm@latest
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy package files for layer caching
|
||||
COPY package.json pnpm-lock.yaml ./
|
||||
RUN pnpm install --frozen-lockfile
|
||||
|
||||
# Copy source code (exclude gateway/ and docker/)
|
||||
COPY client/ ./client/
|
||||
COPY server/ ./server/
|
||||
COPY shared/ ./shared/
|
||||
COPY drizzle/ ./drizzle/
|
||||
COPY drizzle.config.ts tsconfig.json vite.config.ts ./
|
||||
|
||||
# Build frontend and backend
|
||||
RUN pnpm build
|
||||
|
||||
# ── Runtime Stage ─────────────────────────────────────────────────────────────
|
||||
FROM node:22-alpine
|
||||
|
||||
# Install runtime dependencies
|
||||
RUN apk add --no-cache \
|
||||
wget \
|
||||
curl \
|
||||
&& rm -rf /var/cache/apk/*
|
||||
|
||||
# Create non-root user
|
||||
RUN addgroup -g 1001 goclaw && \
|
||||
adduser -u 1001 -G goclaw -s /bin/sh -D goclaw
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install pnpm
|
||||
RUN npm install -g pnpm@latest
|
||||
|
||||
# Copy package files and install production deps only
|
||||
COPY package.json pnpm-lock.yaml ./
|
||||
RUN pnpm install --frozen-lockfile --prod
|
||||
|
||||
# Copy built artifacts
|
||||
COPY --from=builder /app/dist ./dist
|
||||
COPY --from=builder /app/drizzle ./drizzle
|
||||
|
||||
# Set ownership
|
||||
RUN chown -R goclaw:goclaw /app
|
||||
|
||||
USER goclaw
|
||||
|
||||
EXPOSE 3000
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=15s --timeout=5s --start-period=20s --retries=3 \
|
||||
CMD wget -qO- http://localhost:3000/api/health || exit 1
|
||||
|
||||
CMD ["node", "dist/index.js"]
|
||||
60
docker/Dockerfile.gateway
Normal file
60
docker/Dockerfile.gateway
Normal file
@@ -0,0 +1,60 @@
|
||||
# ── Build Stage ──────────────────────────────────────────────────────────────
|
||||
FROM golang:1.23-alpine AS builder
|
||||
|
||||
# Install build dependencies
|
||||
RUN apk add --no-cache git ca-certificates tzdata
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
# Copy go.mod and go.sum first for layer caching
|
||||
COPY gateway/go.mod gateway/go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
# Copy source code
|
||||
COPY gateway/ .
|
||||
|
||||
# Build the binary
|
||||
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
|
||||
go build -ldflags="-w -s -X main.version=$(git describe --tags --always 2>/dev/null || echo dev)" \
|
||||
-o gateway ./cmd/gateway/
|
||||
|
||||
# ── Runtime Stage ─────────────────────────────────────────────────────────────
|
||||
FROM alpine:3.20
|
||||
|
||||
# Install runtime dependencies
|
||||
RUN apk add --no-cache \
|
||||
ca-certificates \
|
||||
tzdata \
|
||||
wget \
|
||||
bash \
|
||||
curl \
|
||||
# For shell_exec tool
|
||||
jq \
|
||||
&& rm -rf /var/cache/apk/*
|
||||
|
||||
# Create non-root user
|
||||
RUN addgroup -g 1001 goclaw && \
|
||||
adduser -u 1001 -G goclaw -s /bin/sh -D goclaw
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy binary from builder
|
||||
COPY --from=builder /build/gateway /usr/local/bin/gateway
|
||||
|
||||
# Copy timezone data
|
||||
COPY --from=builder /usr/share/zoneinfo /usr/share/zoneinfo
|
||||
|
||||
# Ensure binary is executable
|
||||
RUN chmod +x /usr/local/bin/gateway
|
||||
|
||||
# Use non-root user
|
||||
USER goclaw
|
||||
|
||||
# Expose port
|
||||
EXPOSE 18789
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=3 \
|
||||
CMD wget -qO- http://localhost:18789/health || exit 1
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/gateway"]
|
||||
44
docker/README-env.md
Normal file
44
docker/README-env.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# GoClaw — Environment Variables Reference
|
||||
|
||||
Copy and rename this as `.env` in the `docker/` directory. **Never commit `.env` to git.**
|
||||
|
||||
## Database
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `MYSQL_ROOT_PASSWORD` | — | MySQL root password (required) |
|
||||
| `MYSQL_DATABASE` | `goclaw` | Database name |
|
||||
| `MYSQL_USER` | `goclaw` | Application DB user |
|
||||
| `MYSQL_PASSWORD` | — | Application DB password (required) |
|
||||
|
||||
## Security
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `JWT_SECRET` | Session cookie signing secret (min 32 chars) |
|
||||
|
||||
## Manus OAuth
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `VITE_APP_ID` | Manus OAuth application ID |
|
||||
| `OAUTH_SERVER_URL` | Manus OAuth backend URL |
|
||||
| `VITE_OAUTH_PORTAL_URL` | Manus login portal URL |
|
||||
|
||||
## Manus Built-in APIs
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `BUILT_IN_FORGE_API_URL` | Manus built-in API base URL |
|
||||
| `BUILT_IN_FORGE_API_KEY` | Server-side API key |
|
||||
| `VITE_FRONTEND_FORGE_API_KEY` | Frontend API key |
|
||||
| `VITE_FRONTEND_FORGE_API_URL` | Frontend API URL |
|
||||
|
||||
## Go Gateway
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `GATEWAY_URL` | `http://gateway:18789` | Internal URL for Control Center → Gateway |
|
||||
| `OLLAMA_BASE_URL` | `http://ollama:11434` | Ollama LLM server URL |
|
||||
| `OLLAMA_API_KEY` | — | API key if Ollama requires auth |
|
||||
| `REQUEST_TIMEOUT_SECS` | `120` | Max seconds per LLM request |
|
||||
138
docker/docker-compose.yml
Normal file
138
docker/docker-compose.yml
Normal file
@@ -0,0 +1,138 @@
|
||||
##############################################################################
|
||||
# GoClaw Control Center — Docker Compose (Local Development)
|
||||
#
|
||||
# Services:
|
||||
# control-center — React + Node.js tRPC frontend/backend (:3000)
|
||||
# gateway — Go Orchestrator + Tool Executor (:18789)
|
||||
# ollama — Local LLM server (:11434)
|
||||
# db — MySQL 8 (:3306)
|
||||
#
|
||||
# Usage:
|
||||
# docker compose -f docker/docker-compose.yml up -d
|
||||
# docker compose -f docker/docker-compose.yml logs -f gateway
|
||||
##############################################################################
|
||||
|
||||
version: "3.9"
|
||||
|
||||
networks:
|
||||
goclaw-net:
|
||||
driver: bridge
|
||||
|
||||
volumes:
|
||||
ollama-data:
|
||||
mysql-data:
|
||||
|
||||
services:
|
||||
|
||||
# ── MySQL 8 ──────────────────────────────────────────────────────────────
|
||||
db:
|
||||
image: mysql:8.0
|
||||
container_name: goclaw-db
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-goClawRoot123}
|
||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-goclaw}
|
||||
MYSQL_USER: ${MYSQL_USER:-goclaw}
|
||||
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-goClawPass123}
|
||||
ports:
|
||||
- "3306:3306"
|
||||
volumes:
|
||||
- mysql-data:/var/lib/mysql
|
||||
networks:
|
||||
- goclaw-net
|
||||
healthcheck:
|
||||
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-p${MYSQL_ROOT_PASSWORD:-goClawRoot123}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
# ── Ollama LLM Server ─────────────────────────────────────────────────────
|
||||
ollama:
|
||||
image: ollama/ollama:latest
|
||||
container_name: goclaw-ollama
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "11434:11434"
|
||||
volumes:
|
||||
- ollama-data:/root/.ollama
|
||||
networks:
|
||||
- goclaw-net
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: all
|
||||
capabilities: [gpu]
|
||||
# Uncomment if no GPU:
|
||||
# environment:
|
||||
# - OLLAMA_NUM_PARALLEL=2
|
||||
|
||||
# ── Go Gateway (Orchestrator + Tool Executor) ─────────────────────────────
|
||||
gateway:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: docker/Dockerfile.gateway
|
||||
container_name: goclaw-gateway
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "18789:18789"
|
||||
environment:
|
||||
PORT: "18789"
|
||||
OLLAMA_BASE_URL: "http://ollama:11434"
|
||||
DATABASE_URL: "${MYSQL_USER:-goclaw}:${MYSQL_PASSWORD:-goClawPass123}@tcp(db:3306)/${MYSQL_DATABASE:-goclaw}?parseTime=true"
|
||||
PROJECT_ROOT: "/app"
|
||||
REQUEST_TIMEOUT_SECS: "120"
|
||||
LOG_LEVEL: "info"
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
ollama:
|
||||
condition: service_started
|
||||
networks:
|
||||
- goclaw-net
|
||||
volumes:
|
||||
# Mount project root for file tools (read-only in prod, rw in dev)
|
||||
- ..:/app:ro
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:18789/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
|
||||
# ── Control Center (React + Node.js) ─────────────────────────────────────
|
||||
control-center:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: docker/Dockerfile.control-center
|
||||
container_name: goclaw-control-center
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "3000:3000"
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
DATABASE_URL: "mysql://${MYSQL_USER:-goclaw}:${MYSQL_PASSWORD:-goClawPass123}@db:3306/${MYSQL_DATABASE:-goclaw}"
|
||||
GATEWAY_URL: "http://gateway:18789"
|
||||
JWT_SECRET: "${JWT_SECRET:-change-me-in-production}"
|
||||
VITE_APP_ID: "${VITE_APP_ID:-}"
|
||||
OAUTH_SERVER_URL: "${OAUTH_SERVER_URL:-}"
|
||||
VITE_OAUTH_PORTAL_URL: "${VITE_OAUTH_PORTAL_URL:-}"
|
||||
BUILT_IN_FORGE_API_URL: "${BUILT_IN_FORGE_API_URL:-}"
|
||||
BUILT_IN_FORGE_API_KEY: "${BUILT_IN_FORGE_API_KEY:-}"
|
||||
VITE_FRONTEND_FORGE_API_KEY: "${VITE_FRONTEND_FORGE_API_KEY:-}"
|
||||
VITE_FRONTEND_FORGE_API_URL: "${VITE_FRONTEND_FORGE_API_URL:-}"
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
gateway:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- goclaw-net
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
217
docker/docker-stack.yml
Normal file
217
docker/docker-stack.yml
Normal file
@@ -0,0 +1,217 @@
|
||||
##############################################################################
|
||||
# GoClaw Control Center — Docker Stack (Docker Swarm Production)
|
||||
#
|
||||
# Deploy:
|
||||
# docker stack deploy -c docker/docker-stack.yml goclaw
|
||||
#
|
||||
# Remove:
|
||||
# docker stack rm goclaw
|
||||
#
|
||||
# Scale gateway:
|
||||
# docker service scale goclaw_gateway=3
|
||||
##############################################################################
|
||||
|
||||
version: "3.9"
|
||||
|
||||
networks:
|
||||
goclaw-net:
|
||||
driver: overlay
|
||||
attachable: true
|
||||
|
||||
volumes:
|
||||
ollama-data:
|
||||
driver: local
|
||||
mysql-data:
|
||||
driver: local
|
||||
|
||||
configs:
|
||||
gateway-env:
|
||||
external: true
|
||||
|
||||
secrets:
|
||||
mysql-root-password:
|
||||
external: true
|
||||
mysql-password:
|
||||
external: true
|
||||
jwt-secret:
|
||||
external: true
|
||||
|
||||
services:
|
||||
|
||||
# ── MySQL 8 ──────────────────────────────────────────────────────────────
|
||||
db:
|
||||
image: mysql:8.0
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/mysql-root-password
|
||||
MYSQL_DATABASE: goclaw
|
||||
MYSQL_USER: goclaw
|
||||
MYSQL_PASSWORD_FILE: /run/secrets/mysql-password
|
||||
secrets:
|
||||
- mysql-root-password
|
||||
- mysql-password
|
||||
volumes:
|
||||
- mysql-data:/var/lib/mysql
|
||||
networks:
|
||||
- goclaw-net
|
||||
deploy:
|
||||
replicas: 1
|
||||
placement:
|
||||
constraints:
|
||||
- node.role == manager
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 10s
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
reservations:
|
||||
memory: 512M
|
||||
healthcheck:
|
||||
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
# ── Ollama LLM Server ─────────────────────────────────────────────────────
|
||||
ollama:
|
||||
image: ollama/ollama:latest
|
||||
volumes:
|
||||
- ollama-data:/root/.ollama
|
||||
networks:
|
||||
- goclaw-net
|
||||
deploy:
|
||||
replicas: 1
|
||||
placement:
|
||||
constraints:
|
||||
# Pin to GPU node if available
|
||||
- node.labels.gpu == true
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 15s
|
||||
resources:
|
||||
limits:
|
||||
memory: 16G
|
||||
reservations:
|
||||
memory: 4G
|
||||
# GPU support via nvidia-container-runtime
|
||||
# Uncomment on GPU-enabled nodes:
|
||||
# runtime: nvidia
|
||||
# environment:
|
||||
# - NVIDIA_VISIBLE_DEVICES=all
|
||||
|
||||
# ── Go Gateway (Orchestrator + Tool Executor) ─────────────────────────────
|
||||
gateway:
|
||||
image: git.softuniq.eu/uniqai/goclaw/gateway:latest
|
||||
environment:
|
||||
PORT: "18789"
|
||||
OLLAMA_BASE_URL: "http://ollama:11434"
|
||||
DATABASE_URL: "goclaw:{{MYSQL_PASSWORD}}@tcp(db:3306)/goclaw?parseTime=true"
|
||||
PROJECT_ROOT: "/app"
|
||||
REQUEST_TIMEOUT_SECS: "120"
|
||||
LOG_LEVEL: "info"
|
||||
secrets:
|
||||
- mysql-password
|
||||
networks:
|
||||
- goclaw-net
|
||||
ports:
|
||||
- target: 18789
|
||||
published: 18789
|
||||
protocol: tcp
|
||||
mode: ingress
|
||||
deploy:
|
||||
replicas: 2
|
||||
update_config:
|
||||
parallelism: 1
|
||||
delay: 10s
|
||||
order: start-first
|
||||
failure_action: rollback
|
||||
rollback_config:
|
||||
parallelism: 1
|
||||
delay: 5s
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
max_attempts: 3
|
||||
resources:
|
||||
limits:
|
||||
memory: 512M
|
||||
cpus: "1.0"
|
||||
reservations:
|
||||
memory: 128M
|
||||
cpus: "0.25"
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:18789/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
|
||||
# ── Control Center (React + Node.js) ─────────────────────────────────────
|
||||
control-center:
|
||||
image: git.softuniq.eu/uniqai/goclaw/control-center:latest
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
DATABASE_URL: "mysql://goclaw:{{MYSQL_PASSWORD}}@db:3306/goclaw"
|
||||
GATEWAY_URL: "http://gateway:18789"
|
||||
secrets:
|
||||
- mysql-password
|
||||
- jwt-secret
|
||||
networks:
|
||||
- goclaw-net
|
||||
ports:
|
||||
- target: 3000
|
||||
published: 3000
|
||||
protocol: tcp
|
||||
mode: ingress
|
||||
deploy:
|
||||
replicas: 2
|
||||
update_config:
|
||||
parallelism: 1
|
||||
delay: 10s
|
||||
order: start-first
|
||||
failure_action: rollback
|
||||
rollback_config:
|
||||
parallelism: 1
|
||||
delay: 5s
|
||||
restart_policy:
|
||||
condition: on-failure
|
||||
delay: 5s
|
||||
max_attempts: 3
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
cpus: "1.0"
|
||||
reservations:
|
||||
memory: 256M
|
||||
cpus: "0.25"
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
|
||||
# ── Traefik Reverse Proxy (optional) ─────────────────────────────────────
|
||||
# traefik:
|
||||
# image: traefik:v3.0
|
||||
# command:
|
||||
# - "--providers.docker.swarmMode=true"
|
||||
# - "--providers.docker.exposedbydefault=false"
|
||||
# - "--entrypoints.web.address=:80"
|
||||
# - "--entrypoints.websecure.address=:443"
|
||||
# - "--certificatesresolvers.letsencrypt.acme.email=admin@softuniq.eu"
|
||||
# - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
|
||||
# - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
|
||||
# ports:
|
||||
# - "80:80"
|
||||
# - "443:443"
|
||||
# volumes:
|
||||
# - /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
# - traefik-certs:/letsencrypt
|
||||
# networks:
|
||||
# - goclaw-net
|
||||
# deploy:
|
||||
# placement:
|
||||
# constraints:
|
||||
# - node.role == manager
|
||||
121
gateway/cmd/gateway/main.go
Normal file
121
gateway/cmd/gateway/main.go
Normal file
@@ -0,0 +1,121 @@
|
||||
// GoClaw Gateway — Go-based orchestrator and tool executor.
|
||||
// Exposes a REST API on :18789 that the Node.js Control Center proxies to.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"github.com/go-chi/cors"
|
||||
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/config"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/api"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/db"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/llm"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/orchestrator"
|
||||
)
|
||||
|
||||
func main() {
|
||||
log.SetFlags(log.LstdFlags | log.Lshortfile)
|
||||
log.Println("[Gateway] Starting GoClaw Gateway v1.0.0")
|
||||
|
||||
// ── Load config ──────────────────────────────────────────────────────────
|
||||
cfg := config.Load()
|
||||
log.Printf("[Gateway] Port=%s OllamaURL=%s", cfg.Port, cfg.OllamaBaseURL)
|
||||
|
||||
// ── Connect to DB (optional — gateway works without it) ──────────────────
|
||||
var database *db.DB
|
||||
if cfg.DatabaseURL != "" {
|
||||
var err error
|
||||
database, err = db.Connect(cfg.DatabaseURL)
|
||||
if err != nil {
|
||||
log.Printf("[Gateway] WARNING: DB connection failed: %v — running without DB", err)
|
||||
} else {
|
||||
defer database.Close()
|
||||
}
|
||||
}
|
||||
|
||||
// ── LLM Client ───────────────────────────────────────────────────────────
|
||||
llmClient := llm.NewClient(cfg.OllamaBaseURL, cfg.OllamaAPIKey)
|
||||
|
||||
// ── Orchestrator ─────────────────────────────────────────────────────────
|
||||
orch := orchestrator.New(llmClient, database, cfg.ProjectRoot)
|
||||
|
||||
// ── HTTP Handlers ────────────────────────────────────────────────────────
|
||||
h := api.NewHandler(cfg, llmClient, orch, database)
|
||||
|
||||
// ── Router ───────────────────────────────────────────────────────────────
|
||||
r := chi.NewRouter()
|
||||
|
||||
// Middleware
|
||||
r.Use(middleware.RequestID)
|
||||
r.Use(middleware.RealIP)
|
||||
r.Use(middleware.Logger)
|
||||
r.Use(middleware.Recoverer)
|
||||
r.Use(middleware.Timeout(time.Duration(cfg.RequestTimeoutSecs+30) * time.Second))
|
||||
|
||||
// CORS — allow Control Center (Node.js) to call us
|
||||
r.Use(cors.Handler(cors.Options{
|
||||
AllowedOrigins: []string{"*"},
|
||||
AllowedMethods: []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"},
|
||||
AllowedHeaders: []string{"Accept", "Authorization", "Content-Type", "X-Request-ID"},
|
||||
AllowCredentials: false,
|
||||
MaxAge: 300,
|
||||
}))
|
||||
|
||||
// Routes
|
||||
r.Get("/health", h.Health)
|
||||
|
||||
r.Route("/api", func(r chi.Router) {
|
||||
// Orchestrator
|
||||
r.Post("/orchestrator/chat", h.OrchestratorChat)
|
||||
r.Get("/orchestrator/config", h.OrchestratorConfig)
|
||||
|
||||
// Agents
|
||||
r.Get("/agents", h.ListAgents)
|
||||
r.Get("/agents/{id}", h.GetAgent)
|
||||
|
||||
// Models
|
||||
r.Get("/models", h.ListModels)
|
||||
|
||||
// Tools
|
||||
r.Get("/tools", h.ListTools)
|
||||
})
|
||||
|
||||
// ── Start Server ─────────────────────────────────────────────────────────
|
||||
srv := &http.Server{
|
||||
Addr: ":" + cfg.Port,
|
||||
Handler: r,
|
||||
ReadTimeout: 30 * time.Second,
|
||||
WriteTimeout: time.Duration(cfg.RequestTimeoutSecs+60) * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
}
|
||||
|
||||
// Graceful shutdown
|
||||
quit := make(chan os.Signal, 1)
|
||||
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
|
||||
|
||||
go func() {
|
||||
log.Printf("[Gateway] Listening on http://0.0.0.0:%s", cfg.Port)
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
log.Fatalf("[Gateway] Server error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
<-quit
|
||||
log.Println("[Gateway] Shutting down gracefully...")
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
defer cancel()
|
||||
if err := srv.Shutdown(ctx); err != nil {
|
||||
log.Printf("[Gateway] Shutdown error: %v", err)
|
||||
}
|
||||
log.Println("[Gateway] Stopped.")
|
||||
}
|
||||
65
gateway/config/config.go
Normal file
65
gateway/config/config.go
Normal file
@@ -0,0 +1,65 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"log"
|
||||
"os"
|
||||
"strconv"
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
// Server
|
||||
Port string
|
||||
|
||||
// Ollama / LLM
|
||||
OllamaBaseURL string
|
||||
OllamaAPIKey string
|
||||
|
||||
// Database (MySQL/TiDB)
|
||||
DatabaseURL string
|
||||
|
||||
// Project root (for file tools)
|
||||
ProjectRoot string
|
||||
|
||||
// Gateway defaults
|
||||
DefaultModel string
|
||||
MaxToolIterations int
|
||||
RequestTimeoutSecs int
|
||||
}
|
||||
|
||||
func Load() *Config {
|
||||
// Try to load .env from parent directory (project root)
|
||||
_ = godotenv.Load("../.env")
|
||||
_ = godotenv.Load(".env")
|
||||
|
||||
maxIter, _ := strconv.Atoi(getEnv("GATEWAY_MAX_TOOL_ITERATIONS", "10"))
|
||||
timeout, _ := strconv.Atoi(getEnv("GATEWAY_REQUEST_TIMEOUT_SECS", "120"))
|
||||
|
||||
cfg := &Config{
|
||||
Port: getEnv("GATEWAY_PORT", "18789"),
|
||||
OllamaBaseURL: getEnv("OLLAMA_BASE_URL", "https://ollama.com/v1"),
|
||||
OllamaAPIKey: getEnv("OLLAMA_API_KEY", ""),
|
||||
DatabaseURL: getEnv("DATABASE_URL", ""),
|
||||
ProjectRoot: getEnv("PROJECT_ROOT", "/home/ubuntu/goclaw-control-center"),
|
||||
DefaultModel: getEnv("DEFAULT_MODEL", "qwen2.5:7b"),
|
||||
MaxToolIterations: maxIter,
|
||||
RequestTimeoutSecs: timeout,
|
||||
}
|
||||
|
||||
if cfg.OllamaAPIKey == "" {
|
||||
log.Println("[Config] WARNING: OLLAMA_API_KEY is not set")
|
||||
}
|
||||
if cfg.DatabaseURL == "" {
|
||||
log.Println("[Config] WARNING: DATABASE_URL is not set — agent config will use defaults")
|
||||
}
|
||||
|
||||
return cfg
|
||||
}
|
||||
|
||||
func getEnv(key, fallback string) string {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
return v
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
12
gateway/go.mod
Normal file
12
gateway/go.mod
Normal file
@@ -0,0 +1,12 @@
|
||||
module git.softuniq.eu/UniqAI/GoClaw/gateway
|
||||
|
||||
go 1.23.4
|
||||
|
||||
require (
|
||||
filippo.io/edwards25519 v1.1.0 // indirect
|
||||
github.com/go-chi/chi/v5 v5.2.1 // indirect
|
||||
github.com/go-chi/cors v1.2.1 // indirect
|
||||
github.com/go-sql-driver/mysql v1.8.1 // indirect
|
||||
github.com/jmoiron/sqlx v1.4.0 // indirect
|
||||
github.com/joho/godotenv v1.5.1 // indirect
|
||||
)
|
||||
14
gateway/go.sum
Normal file
14
gateway/go.sum
Normal file
@@ -0,0 +1,14 @@
|
||||
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
|
||||
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
|
||||
github.com/go-chi/chi/v5 v5.2.1 h1:KOIHODQj58PmL80G2Eak4WdvUzjSJSm0vG72crDCqb8=
|
||||
github.com/go-chi/chi/v5 v5.2.1/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
|
||||
github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4=
|
||||
github.com/go-chi/cors v1.2.1/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
|
||||
github.com/go-sql-driver/mysql v1.8.1 h1:LedoTUt/eveggdHS9qUFC1EFSa8bU2+1pZjSRpvNJ1Y=
|
||||
github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
|
||||
github.com/jmoiron/sqlx v1.4.0 h1:1PLqN7S1UYp5t4SrVVnt4nUVNemrDAtxlulVe+Qgm3o=
|
||||
github.com/jmoiron/sqlx v1.4.0/go.mod h1:ZrZ7UsYB/weZdl2Bxg6jCRO9c3YHl8r3ahlKmRT4JLY=
|
||||
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
|
||||
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||
179
gateway/internal/api/handlers.go
Normal file
179
gateway/internal/api/handlers.go
Normal file
@@ -0,0 +1,179 @@
|
||||
// Package api implements the HTTP REST API for the GoClaw Gateway.
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/config"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/db"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/llm"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/orchestrator"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/tools"
|
||||
)
|
||||
|
||||
// Handler holds all dependencies for HTTP handlers.
|
||||
type Handler struct {
|
||||
cfg *config.Config
|
||||
llm *llm.Client
|
||||
orch *orchestrator.Orchestrator
|
||||
db *db.DB
|
||||
}
|
||||
|
||||
func NewHandler(cfg *config.Config, llmClient *llm.Client, orch *orchestrator.Orchestrator, database *db.DB) *Handler {
|
||||
return &Handler{
|
||||
cfg: cfg,
|
||||
llm: llmClient,
|
||||
orch: orch,
|
||||
db: database,
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Health ───────────────────────────────────────────────────────────────────
|
||||
|
||||
// GET /health
|
||||
func (h *Handler) Health(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
ollamaOK, latency, _ := h.llm.Health(ctx)
|
||||
|
||||
respond(w, http.StatusOK, map[string]any{
|
||||
"status": "ok",
|
||||
"service": "goclaw-gateway",
|
||||
"version": "1.0.0",
|
||||
"timestamp": time.Now().UTC().Format(time.RFC3339),
|
||||
"ollama": map[string]any{
|
||||
"connected": ollamaOK,
|
||||
"latencyMs": latency,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
// ─── Orchestrator ─────────────────────────────────────────────────────────────
|
||||
|
||||
// POST /api/orchestrator/chat
|
||||
// Body: { "messages": [{"role":"user","content":"..."}], "model": "optional-override" }
|
||||
func (h *Handler) OrchestratorChat(w http.ResponseWriter, r *http.Request) {
|
||||
var req struct {
|
||||
Messages []orchestrator.Message `json:"messages"`
|
||||
Model string `json:"model,omitempty"`
|
||||
MaxIter int `json:"maxIter,omitempty"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
respondError(w, http.StatusBadRequest, "invalid request body: "+err.Error())
|
||||
return
|
||||
}
|
||||
if len(req.Messages) == 0 {
|
||||
respondError(w, http.StatusBadRequest, "messages array is required")
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("[API] POST /api/orchestrator/chat — messages=%d model=%q", len(req.Messages), req.Model)
|
||||
|
||||
ctx, cancel := context.WithTimeout(r.Context(), time.Duration(h.cfg.RequestTimeoutSecs)*time.Second)
|
||||
defer cancel()
|
||||
|
||||
result := h.orch.Chat(ctx, req.Messages, req.Model, req.MaxIter)
|
||||
respond(w, http.StatusOK, result)
|
||||
}
|
||||
|
||||
// GET /api/orchestrator/config
|
||||
func (h *Handler) OrchestratorConfig(w http.ResponseWriter, r *http.Request) {
|
||||
cfg := h.orch.GetConfig()
|
||||
respond(w, http.StatusOK, map[string]any{
|
||||
"id": cfg.ID,
|
||||
"name": cfg.Name,
|
||||
"model": cfg.Model,
|
||||
"temperature": cfg.Temperature,
|
||||
"maxTokens": cfg.MaxTokens,
|
||||
"allowedTools": cfg.AllowedTools,
|
||||
// Don't expose full system prompt for security
|
||||
"systemPromptPreview": truncate(cfg.SystemPrompt, 200),
|
||||
})
|
||||
}
|
||||
|
||||
// ─── Agents ───────────────────────────────────────────────────────────────────
|
||||
|
||||
// GET /api/agents
|
||||
func (h *Handler) ListAgents(w http.ResponseWriter, r *http.Request) {
|
||||
if h.db == nil {
|
||||
respond(w, http.StatusOK, map[string]any{"agents": []any{}, "note": "DB not connected"})
|
||||
return
|
||||
}
|
||||
agents, err := h.db.ListAgents()
|
||||
if err != nil {
|
||||
respondError(w, http.StatusInternalServerError, "failed to list agents: "+err.Error())
|
||||
return
|
||||
}
|
||||
respond(w, http.StatusOK, map[string]any{"agents": agents, "count": len(agents)})
|
||||
}
|
||||
|
||||
// GET /api/agents/{id}
|
||||
func (h *Handler) GetAgent(w http.ResponseWriter, r *http.Request) {
|
||||
idStr := r.PathValue("id")
|
||||
id, err := strconv.Atoi(idStr)
|
||||
if err != nil {
|
||||
respondError(w, http.StatusBadRequest, "invalid agent id")
|
||||
return
|
||||
}
|
||||
if h.db == nil {
|
||||
respondError(w, http.StatusServiceUnavailable, "DB not connected")
|
||||
return
|
||||
}
|
||||
agent, err := h.db.GetAgentByID(id)
|
||||
if err != nil {
|
||||
respondError(w, http.StatusNotFound, "agent not found")
|
||||
return
|
||||
}
|
||||
respond(w, http.StatusOK, agent)
|
||||
}
|
||||
|
||||
// ─── Models ───────────────────────────────────────────────────────────────────
|
||||
|
||||
// GET /api/models
|
||||
func (h *Handler) ListModels(w http.ResponseWriter, r *http.Request) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
|
||||
defer cancel()
|
||||
|
||||
models, err := h.llm.ListModels(ctx)
|
||||
if err != nil {
|
||||
respondError(w, http.StatusBadGateway, "failed to fetch models: "+err.Error())
|
||||
return
|
||||
}
|
||||
respond(w, http.StatusOK, models)
|
||||
}
|
||||
|
||||
// ─── Tools ────────────────────────────────────────────────────────────────────
|
||||
|
||||
// GET /api/tools
|
||||
func (h *Handler) ListTools(w http.ResponseWriter, r *http.Request) {
|
||||
toolDefs := tools.OrchestratorTools()
|
||||
respond(w, http.StatusOK, map[string]any{
|
||||
"tools": toolDefs,
|
||||
"count": len(toolDefs),
|
||||
})
|
||||
}
|
||||
|
||||
// ─── Helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
func respond(w http.ResponseWriter, status int, data any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(data)
|
||||
}
|
||||
|
||||
func respondError(w http.ResponseWriter, status int, msg string) {
|
||||
respond(w, status, map[string]any{"error": msg})
|
||||
}
|
||||
|
||||
func truncate(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return s[:n] + "..."
|
||||
}
|
||||
184
gateway/internal/db/db.go
Normal file
184
gateway/internal/db/db.go
Normal file
@@ -0,0 +1,184 @@
|
||||
// Package db provides MySQL/TiDB connectivity and agent config queries.
|
||||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
|
||||
_ "github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
// AgentConfig holds the orchestrator/agent configuration loaded from DB.
|
||||
type AgentConfig struct {
|
||||
ID int
|
||||
Name string
|
||||
Model string
|
||||
SystemPrompt string
|
||||
AllowedTools []string
|
||||
Temperature float64
|
||||
MaxTokens int
|
||||
IsOrchestrator bool
|
||||
IsSystem bool
|
||||
IsActive bool
|
||||
}
|
||||
|
||||
// AgentRow is a minimal agent representation for listing.
|
||||
type AgentRow struct {
|
||||
ID int `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Role string `json:"role"`
|
||||
Model string `json:"model"`
|
||||
Description string `json:"description"`
|
||||
IsActive bool `json:"isActive"`
|
||||
IsSystem bool `json:"isSystem"`
|
||||
IsOrchestrator bool `json:"isOrchestrator"`
|
||||
}
|
||||
|
||||
type DB struct {
|
||||
conn *sql.DB
|
||||
}
|
||||
|
||||
func Connect(dsn string) (*DB, error) {
|
||||
if dsn == "" {
|
||||
return nil, fmt.Errorf("DATABASE_URL is empty")
|
||||
}
|
||||
// Convert mysql:// URL to DSN format if needed
|
||||
dsn = normalizeDSN(dsn)
|
||||
|
||||
conn, err := sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to open DB: %w", err)
|
||||
}
|
||||
if err := conn.Ping(); err != nil {
|
||||
return nil, fmt.Errorf("failed to ping DB: %w", err)
|
||||
}
|
||||
log.Println("[DB] Connected to MySQL")
|
||||
return &DB{conn: conn}, nil
|
||||
}
|
||||
|
||||
func (d *DB) Close() {
|
||||
if d.conn != nil {
|
||||
_ = d.conn.Close()
|
||||
}
|
||||
}
|
||||
|
||||
// GetOrchestratorConfig loads the agent with isOrchestrator=1 from DB.
|
||||
func (d *DB) GetOrchestratorConfig() (*AgentConfig, error) {
|
||||
row := d.conn.QueryRow(`
|
||||
SELECT id, name, model, systemPrompt, allowedTools, temperature, maxTokens, isOrchestrator, isSystem, isActive
|
||||
FROM agents
|
||||
WHERE isOrchestrator = 1
|
||||
LIMIT 1
|
||||
`)
|
||||
return scanAgentConfig(row)
|
||||
}
|
||||
|
||||
// GetAgentByID loads a specific agent by ID.
|
||||
func (d *DB) GetAgentByID(id int) (*AgentConfig, error) {
|
||||
row := d.conn.QueryRow(`
|
||||
SELECT id, name, model, systemPrompt, allowedTools, temperature, maxTokens, isOrchestrator, isSystem, isActive
|
||||
FROM agents
|
||||
WHERE id = ?
|
||||
LIMIT 1
|
||||
`, id)
|
||||
return scanAgentConfig(row)
|
||||
}
|
||||
|
||||
// ListAgents returns all active agents.
|
||||
func (d *DB) ListAgents() ([]AgentRow, error) {
|
||||
rows, err := d.conn.Query(`
|
||||
SELECT id, name, role, model, COALESCE(description,''), isActive, isSystem, isOrchestrator
|
||||
FROM agents
|
||||
ORDER BY isOrchestrator DESC, isSystem DESC, id ASC
|
||||
`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var agents []AgentRow
|
||||
for rows.Next() {
|
||||
var a AgentRow
|
||||
var isActive, isSystem, isOrch int
|
||||
if err := rows.Scan(&a.ID, &a.Name, &a.Role, &a.Model, &a.Description, &isActive, &isSystem, &isOrch); err != nil {
|
||||
continue
|
||||
}
|
||||
a.IsActive = isActive == 1
|
||||
a.IsSystem = isSystem == 1
|
||||
a.IsOrchestrator = isOrch == 1
|
||||
agents = append(agents, a)
|
||||
}
|
||||
return agents, nil
|
||||
}
|
||||
|
||||
// ─── Helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
func scanAgentConfig(row *sql.Row) (*AgentConfig, error) {
|
||||
var cfg AgentConfig
|
||||
var systemPrompt sql.NullString
|
||||
var allowedToolsJSON sql.NullString
|
||||
var temperature sql.NullFloat64
|
||||
var maxTokens sql.NullInt64
|
||||
var isOrch, isSystem, isActive int
|
||||
|
||||
err := row.Scan(
|
||||
&cfg.ID, &cfg.Name, &cfg.Model,
|
||||
&systemPrompt, &allowedToolsJSON,
|
||||
&temperature, &maxTokens,
|
||||
&isOrch, &isSystem, &isActive,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
cfg.SystemPrompt = systemPrompt.String
|
||||
cfg.Temperature = temperature.Float64
|
||||
if cfg.Temperature == 0 {
|
||||
cfg.Temperature = 0.5
|
||||
}
|
||||
cfg.MaxTokens = int(maxTokens.Int64)
|
||||
if cfg.MaxTokens == 0 {
|
||||
cfg.MaxTokens = 8192
|
||||
}
|
||||
cfg.IsOrchestrator = isOrch == 1
|
||||
cfg.IsSystem = isSystem == 1
|
||||
cfg.IsActive = isActive == 1
|
||||
|
||||
if allowedToolsJSON.Valid && allowedToolsJSON.String != "" && allowedToolsJSON.String != "null" {
|
||||
_ = json.Unmarshal([]byte(allowedToolsJSON.String), &cfg.AllowedTools)
|
||||
}
|
||||
|
||||
return &cfg, nil
|
||||
}
|
||||
|
||||
// normalizeDSN converts mysql://user:pass@host:port/db to user:pass@tcp(host:port)/db
|
||||
func normalizeDSN(dsn string) string {
|
||||
if !strings.HasPrefix(dsn, "mysql://") {
|
||||
return dsn
|
||||
}
|
||||
// Strip scheme
|
||||
dsn = strings.TrimPrefix(dsn, "mysql://")
|
||||
// user:pass@host:port/db → user:pass@tcp(host:port)/db
|
||||
atIdx := strings.LastIndex(dsn, "@")
|
||||
if atIdx < 0 {
|
||||
return dsn
|
||||
}
|
||||
userInfo := dsn[:atIdx]
|
||||
hostDB := dsn[atIdx+1:]
|
||||
|
||||
// Split host:port/db
|
||||
slashIdx := strings.Index(hostDB, "/")
|
||||
var hostPort, dbName string
|
||||
if slashIdx >= 0 {
|
||||
hostPort = hostDB[:slashIdx]
|
||||
dbName = hostDB[slashIdx:]
|
||||
} else {
|
||||
hostPort = hostDB
|
||||
dbName = ""
|
||||
}
|
||||
|
||||
return fmt.Sprintf("%s@tcp(%s)%s?parseTime=true&charset=utf8mb4", userInfo, hostPort, dbName)
|
||||
}
|
||||
196
gateway/internal/llm/client.go
Normal file
196
gateway/internal/llm/client.go
Normal file
@@ -0,0 +1,196 @@
|
||||
// Package llm provides an OpenAI-compatible client for the Ollama Cloud API.
|
||||
package llm
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ─── Types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
type Message struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
ToolCallID string `json:"tool_call_id,omitempty"`
|
||||
Name string `json:"name,omitempty"`
|
||||
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
|
||||
}
|
||||
|
||||
type ToolCall struct {
|
||||
ID string `json:"id"`
|
||||
Type string `json:"type"`
|
||||
Function ToolCallFunction `json:"function"`
|
||||
}
|
||||
|
||||
type ToolCallFunction struct {
|
||||
Name string `json:"name"`
|
||||
Arguments string `json:"arguments"`
|
||||
}
|
||||
|
||||
type Tool struct {
|
||||
Type string `json:"type"`
|
||||
Function ToolFunction `json:"function"`
|
||||
}
|
||||
|
||||
type ToolFunction struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
Parameters map[string]any `json:"parameters"`
|
||||
}
|
||||
|
||||
type ChatRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages []Message `json:"messages"`
|
||||
Stream bool `json:"stream"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
Tools []Tool `json:"tools,omitempty"`
|
||||
ToolChoice string `json:"tool_choice,omitempty"`
|
||||
}
|
||||
|
||||
type ChatChoice struct {
|
||||
Index int `json:"index"`
|
||||
Message Message `json:"message"`
|
||||
FinishReason string `json:"finish_reason"`
|
||||
}
|
||||
|
||||
type Usage struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
}
|
||||
|
||||
type ChatResponse struct {
|
||||
ID string `json:"id"`
|
||||
Object string `json:"object"`
|
||||
Created int64 `json:"created"`
|
||||
Model string `json:"model"`
|
||||
Choices []ChatChoice `json:"choices"`
|
||||
Usage *Usage `json:"usage,omitempty"`
|
||||
}
|
||||
|
||||
type Model struct {
|
||||
ID string `json:"id"`
|
||||
Object string `json:"object"`
|
||||
Created int64 `json:"created"`
|
||||
OwnedBy string `json:"owned_by"`
|
||||
}
|
||||
|
||||
type ModelsResponse struct {
|
||||
Object string `json:"object"`
|
||||
Data []Model `json:"data"`
|
||||
}
|
||||
|
||||
// ─── Client ───────────────────────────────────────────────────────────────────
|
||||
|
||||
type Client struct {
|
||||
baseURL string
|
||||
apiKey string
|
||||
httpClient *http.Client
|
||||
}
|
||||
|
||||
func NewClient(baseURL, apiKey string) *Client {
|
||||
return &Client{
|
||||
baseURL: strings.TrimRight(baseURL, "/"),
|
||||
apiKey: apiKey,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 180 * time.Second,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Client) headers() map[string]string {
|
||||
h := map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
if c.apiKey != "" {
|
||||
h["Authorization"] = "Bearer " + c.apiKey
|
||||
}
|
||||
return h
|
||||
}
|
||||
|
||||
// Health checks if the Ollama API is reachable.
|
||||
func (c *Client) Health(ctx context.Context) (bool, int64, error) {
|
||||
start := time.Now()
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"/models", nil)
|
||||
if err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
for k, v := range c.headers() {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
resp, err := c.httpClient.Do(req)
|
||||
latency := time.Since(start).Milliseconds()
|
||||
if err != nil {
|
||||
return false, latency, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
return resp.StatusCode == http.StatusOK, latency, nil
|
||||
}
|
||||
|
||||
// ListModels returns available models.
|
||||
func (c *Client) ListModels(ctx context.Context) (*ModelsResponse, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"/models", nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for k, v := range c.headers() {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
resp, err := c.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return nil, fmt.Errorf("ollama API error (%d): %s", resp.StatusCode, string(body))
|
||||
}
|
||||
var result ModelsResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
// Chat sends a chat completion request (non-streaming).
|
||||
func (c *Client) Chat(ctx context.Context, req ChatRequest) (*ChatResponse, error) {
|
||||
req.Stream = false
|
||||
|
||||
body, err := json.Marshal(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost,
|
||||
c.baseURL+"/chat/completions", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for k, v := range c.headers() {
|
||||
httpReq.Header.Set(k, v)
|
||||
}
|
||||
|
||||
resp, err := c.httpClient.Do(httpReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
respBody, _ := io.ReadAll(resp.Body)
|
||||
return nil, fmt.Errorf("ollama chat API error (%d): %s", resp.StatusCode, string(respBody))
|
||||
}
|
||||
|
||||
var result ChatResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &result, nil
|
||||
}
|
||||
305
gateway/internal/orchestrator/orchestrator.go
Normal file
305
gateway/internal/orchestrator/orchestrator.go
Normal file
@@ -0,0 +1,305 @@
|
||||
// Package orchestrator implements the GoClaw main AI orchestration loop.
|
||||
// It loads config from DB, calls the LLM with tool definitions,
|
||||
// executes tool calls, and returns the final response.
|
||||
package orchestrator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/db"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/llm"
|
||||
"git.softuniq.eu/UniqAI/GoClaw/gateway/internal/tools"
|
||||
)
|
||||
|
||||
// ─── Types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
type Message struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
}
|
||||
|
||||
type ToolCallStep struct {
|
||||
Tool string `json:"tool"`
|
||||
Args any `json:"args"`
|
||||
Result any `json:"result,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
Success bool `json:"success"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
}
|
||||
|
||||
type ChatResult struct {
|
||||
Success bool `json:"success"`
|
||||
Response string `json:"response"`
|
||||
ToolCalls []ToolCallStep `json:"toolCalls"`
|
||||
Model string `json:"model"`
|
||||
Usage *llm.Usage `json:"usage,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// OrchestratorConfig is the runtime config loaded from DB or defaults.
|
||||
type OrchestratorConfig struct {
|
||||
ID int
|
||||
Name string
|
||||
Model string
|
||||
SystemPrompt string
|
||||
AllowedTools []string
|
||||
Temperature float64
|
||||
MaxTokens int
|
||||
}
|
||||
|
||||
// ─── Default System Prompt ────────────────────────────────────────────────────
|
||||
|
||||
const defaultSystemPrompt = `You are GoClaw Orchestrator — the main AI agent managing the GoClaw distributed AI system.
|
||||
|
||||
You have full access to:
|
||||
1. **Specialized Agents**: Browser Agent (web browsing), Tool Builder (create tools), Agent Compiler (create agents)
|
||||
2. **System Tools**: shell_exec (run commands), file_read/write (manage files), http_request (web requests), docker_exec (Docker management)
|
||||
|
||||
Your responsibilities:
|
||||
- Answer user questions directly when possible
|
||||
- Delegate complex web tasks to Browser Agent
|
||||
- Execute shell commands to manage the system
|
||||
- Read and write files to modify the codebase
|
||||
- Monitor Docker containers and services
|
||||
|
||||
Decision making:
|
||||
- For simple questions: answer directly without tools
|
||||
- For system tasks: use shell_exec, file_read/write
|
||||
- For Docker: use docker_exec
|
||||
- Always use list_agents first if you're unsure which agent to delegate to
|
||||
|
||||
Response style:
|
||||
- Be concise and actionable
|
||||
- Show what tools you used and their results
|
||||
- Respond in the same language as the user
|
||||
|
||||
You are running on a Linux server with Docker and full internet access.`
|
||||
|
||||
// ─── Orchestrator ─────────────────────────────────────────────────────────────
|
||||
|
||||
type Orchestrator struct {
|
||||
llmClient *llm.Client
|
||||
executor *tools.Executor
|
||||
database *db.DB
|
||||
projectRoot string
|
||||
}
|
||||
|
||||
func New(llmClient *llm.Client, database *db.DB, projectRoot string) *Orchestrator {
|
||||
o := &Orchestrator{
|
||||
llmClient: llmClient,
|
||||
database: database,
|
||||
projectRoot: projectRoot,
|
||||
}
|
||||
// Inject agent list function to avoid circular dependency
|
||||
o.executor = tools.NewExecutor(projectRoot, o.listAgentsFn)
|
||||
return o
|
||||
}
|
||||
|
||||
// GetConfig loads orchestrator config from DB, falls back to defaults.
|
||||
func (o *Orchestrator) GetConfig() *OrchestratorConfig {
|
||||
if o.database != nil {
|
||||
cfg, err := o.database.GetOrchestratorConfig()
|
||||
if err == nil && cfg != nil {
|
||||
systemPrompt := cfg.SystemPrompt
|
||||
if systemPrompt == "" {
|
||||
systemPrompt = defaultSystemPrompt
|
||||
}
|
||||
return &OrchestratorConfig{
|
||||
ID: cfg.ID,
|
||||
Name: cfg.Name,
|
||||
Model: cfg.Model,
|
||||
SystemPrompt: systemPrompt,
|
||||
AllowedTools: cfg.AllowedTools,
|
||||
Temperature: cfg.Temperature,
|
||||
MaxTokens: cfg.MaxTokens,
|
||||
}
|
||||
}
|
||||
log.Printf("[Orchestrator] Failed to load config from DB: %v — using defaults", err)
|
||||
}
|
||||
return &OrchestratorConfig{
|
||||
Name: "GoClaw Orchestrator",
|
||||
Model: "qwen2.5:7b",
|
||||
SystemPrompt: defaultSystemPrompt,
|
||||
Temperature: 0.5,
|
||||
MaxTokens: 8192,
|
||||
}
|
||||
}
|
||||
|
||||
// Chat runs the full orchestration loop: LLM → tool calls → LLM → response.
|
||||
func (o *Orchestrator) Chat(ctx context.Context, messages []Message, overrideModel string, maxIter int) ChatResult {
|
||||
if maxIter <= 0 {
|
||||
maxIter = 10
|
||||
}
|
||||
|
||||
cfg := o.GetConfig()
|
||||
model := cfg.Model
|
||||
if overrideModel != "" {
|
||||
model = overrideModel
|
||||
}
|
||||
|
||||
log.Printf("[Orchestrator] Chat started: model=%s, messages=%d", model, len(messages))
|
||||
|
||||
// Build conversation
|
||||
conv := []llm.Message{
|
||||
{Role: "system", Content: cfg.SystemPrompt},
|
||||
}
|
||||
for _, m := range messages {
|
||||
conv = append(conv, llm.Message{Role: m.Role, Content: m.Content})
|
||||
}
|
||||
|
||||
// Build tools list
|
||||
toolDefs := tools.OrchestratorTools()
|
||||
llmTools := make([]llm.Tool, len(toolDefs))
|
||||
for i, t := range toolDefs {
|
||||
llmTools[i] = llm.Tool{
|
||||
Type: t.Type,
|
||||
Function: llm.ToolFunction{
|
||||
Name: t.Function.Name,
|
||||
Description: t.Function.Description,
|
||||
Parameters: t.Function.Parameters,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
temp := cfg.Temperature
|
||||
maxTok := cfg.MaxTokens
|
||||
|
||||
var toolCallSteps []ToolCallStep
|
||||
var finalResponse string
|
||||
var lastUsage *llm.Usage
|
||||
var lastModel string
|
||||
|
||||
for iter := 0; iter < maxIter; iter++ {
|
||||
req := llm.ChatRequest{
|
||||
Model: model,
|
||||
Messages: conv,
|
||||
Temperature: &temp,
|
||||
MaxTokens: &maxTok,
|
||||
Tools: llmTools,
|
||||
ToolChoice: "auto",
|
||||
}
|
||||
|
||||
resp, err := o.llmClient.Chat(ctx, req)
|
||||
if err != nil {
|
||||
// Fallback: try without tools
|
||||
log.Printf("[Orchestrator] LLM error with tools: %v — retrying without tools", err)
|
||||
req.Tools = nil
|
||||
req.ToolChoice = ""
|
||||
resp2, err2 := o.llmClient.Chat(ctx, req)
|
||||
if err2 != nil {
|
||||
return ChatResult{
|
||||
Success: false,
|
||||
Error: fmt.Sprintf("LLM error (model: %s): %v", model, err2),
|
||||
}
|
||||
}
|
||||
if len(resp2.Choices) > 0 {
|
||||
finalResponse = resp2.Choices[0].Message.Content
|
||||
lastUsage = resp2.Usage
|
||||
lastModel = resp2.Model
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
if len(resp.Choices) == 0 {
|
||||
break
|
||||
}
|
||||
|
||||
choice := resp.Choices[0]
|
||||
lastUsage = resp.Usage
|
||||
lastModel = resp.Model
|
||||
if lastModel == "" {
|
||||
lastModel = model
|
||||
}
|
||||
|
||||
// Check if LLM wants to call tools
|
||||
if choice.FinishReason == "tool_calls" && len(choice.Message.ToolCalls) > 0 {
|
||||
// Add assistant message with tool calls to conversation
|
||||
conv = append(conv, choice.Message)
|
||||
|
||||
// Execute each tool call
|
||||
for _, tc := range choice.Message.ToolCalls {
|
||||
toolName := tc.Function.Name
|
||||
argsJSON := tc.Function.Arguments
|
||||
|
||||
log.Printf("[Orchestrator] Executing tool: %s args=%s", toolName, argsJSON)
|
||||
start := time.Now()
|
||||
|
||||
result := o.executor.Execute(ctx, toolName, argsJSON)
|
||||
|
||||
step := ToolCallStep{
|
||||
Tool: toolName,
|
||||
Success: result.Success,
|
||||
DurationMs: time.Since(start).Milliseconds(),
|
||||
}
|
||||
|
||||
// Parse args for display
|
||||
var argsMap any
|
||||
_ = json.Unmarshal([]byte(argsJSON), &argsMap)
|
||||
step.Args = argsMap
|
||||
|
||||
var toolResultContent string
|
||||
if result.Success {
|
||||
step.Result = result.Result
|
||||
resultBytes, _ := json.Marshal(result.Result)
|
||||
toolResultContent = string(resultBytes)
|
||||
} else {
|
||||
step.Error = result.Error
|
||||
toolResultContent = fmt.Sprintf(`{"error": %q}`, result.Error)
|
||||
}
|
||||
|
||||
toolCallSteps = append(toolCallSteps, step)
|
||||
|
||||
// Add tool result to conversation
|
||||
conv = append(conv, llm.Message{
|
||||
Role: "tool",
|
||||
Content: toolResultContent,
|
||||
ToolCallID: tc.ID,
|
||||
Name: toolName,
|
||||
})
|
||||
}
|
||||
// Continue loop — LLM will process tool results
|
||||
continue
|
||||
}
|
||||
|
||||
// LLM finished — extract final response
|
||||
finalResponse = choice.Message.Content
|
||||
break
|
||||
}
|
||||
|
||||
return ChatResult{
|
||||
Success: true,
|
||||
Response: finalResponse,
|
||||
ToolCalls: toolCallSteps,
|
||||
Model: lastModel,
|
||||
Usage: lastUsage,
|
||||
}
|
||||
}
|
||||
|
||||
// listAgentsFn is injected into the tool executor to list agents from DB.
|
||||
func (o *Orchestrator) listAgentsFn() ([]map[string]any, error) {
|
||||
if o.database == nil {
|
||||
return []map[string]any{}, nil
|
||||
}
|
||||
rows, err := o.database.ListAgents()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := make([]map[string]any, len(rows))
|
||||
for i, r := range rows {
|
||||
result[i] = map[string]any{
|
||||
"id": r.ID,
|
||||
"name": r.Name,
|
||||
"role": r.Role,
|
||||
"model": r.Model,
|
||||
"description": r.Description,
|
||||
"isActive": r.IsActive,
|
||||
"isSystem": r.IsSystem,
|
||||
"isOrchestrator": r.IsOrchestrator,
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
475
gateway/internal/tools/executor.go
Normal file
475
gateway/internal/tools/executor.go
Normal file
@@ -0,0 +1,475 @@
|
||||
// Package tools implements the GoClaw Tool Executor.
|
||||
// Each tool corresponds to a function the LLM can call via OpenAI function calling.
|
||||
package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ─── Types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
type ToolResult struct {
|
||||
Success bool `json:"success"`
|
||||
Result any `json:"result,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
}
|
||||
|
||||
// ─── Tool Definitions (OpenAI function calling schema) ────────────────────────
|
||||
|
||||
type ToolDef struct {
|
||||
Type string `json:"type"`
|
||||
Function FuncDef `json:"function"`
|
||||
}
|
||||
|
||||
type FuncDef struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
Parameters map[string]any `json:"parameters"`
|
||||
}
|
||||
|
||||
// OrchestratorTools returns the full list of tools available to the orchestrator.
|
||||
func OrchestratorTools() []ToolDef {
|
||||
return []ToolDef{
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "shell_exec",
|
||||
Description: "Execute a bash command on the host system. Returns stdout and stderr.",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"command": map[string]any{"type": "string", "description": "Bash command to execute"},
|
||||
"timeout": map[string]any{"type": "number", "description": "Timeout in seconds (default: 30)"},
|
||||
},
|
||||
"required": []string{"command"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "file_read",
|
||||
Description: "Read a file from the filesystem. Returns file content as text.",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"path": map[string]any{"type": "string", "description": "Absolute or relative file path"},
|
||||
},
|
||||
"required": []string{"path"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "file_write",
|
||||
Description: "Write content to a file. Creates parent directories if needed.",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"path": map[string]any{"type": "string", "description": "File path to write"},
|
||||
"content": map[string]any{"type": "string", "description": "Content to write"},
|
||||
"append": map[string]any{"type": "boolean", "description": "Append instead of overwrite"},
|
||||
},
|
||||
"required": []string{"path", "content"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "file_list",
|
||||
Description: "List files and directories at a given path.",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"path": map[string]any{"type": "string", "description": "Directory path"},
|
||||
"recursive": map[string]any{"type": "boolean", "description": "List recursively"},
|
||||
},
|
||||
"required": []string{"path"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "http_request",
|
||||
Description: "Make an HTTP request (GET, POST, PUT, DELETE) to any URL.",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"url": map[string]any{"type": "string", "description": "Target URL"},
|
||||
"method": map[string]any{"type": "string", "description": "HTTP method (default: GET)"},
|
||||
"headers": map[string]any{"type": "object", "description": "Request headers"},
|
||||
"body": map[string]any{"type": "string", "description": "Request body for POST/PUT"},
|
||||
},
|
||||
"required": []string{"url"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "docker_exec",
|
||||
Description: "Execute a Docker CLI command (docker ps, docker logs, docker exec, etc.).",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"command": map[string]any{"type": "string", "description": "Docker command without 'docker' prefix (e.g. 'ps -a', 'logs mycontainer')"},
|
||||
},
|
||||
"required": []string{"command"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "list_agents",
|
||||
Description: "List all available specialized agents with their capabilities.",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Type: "function",
|
||||
Function: FuncDef{
|
||||
Name: "delegate_to_agent",
|
||||
Description: "Delegate a task to a specialized agent (Browser Agent, Tool Builder, Agent Compiler).",
|
||||
Parameters: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"agentId": map[string]any{"type": "number", "description": "Agent ID to delegate to"},
|
||||
"message": map[string]any{"type": "string", "description": "Task description for the agent"},
|
||||
},
|
||||
"required": []string{"agentId", "message"},
|
||||
"additionalProperties": false,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Executor ─────────────────────────────────────────────────────────────────
|
||||
|
||||
type Executor struct {
|
||||
projectRoot string
|
||||
httpClient *http.Client
|
||||
// agentListFn is injected to avoid circular dependency with orchestrator
|
||||
agentListFn func() ([]map[string]any, error)
|
||||
}
|
||||
|
||||
func NewExecutor(projectRoot string, agentListFn func() ([]map[string]any, error)) *Executor {
|
||||
return &Executor{
|
||||
projectRoot: projectRoot,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 60 * time.Second,
|
||||
},
|
||||
agentListFn: agentListFn,
|
||||
}
|
||||
}
|
||||
|
||||
// Execute dispatches a tool call by name.
|
||||
func (e *Executor) Execute(ctx context.Context, toolName string, argsJSON string) ToolResult {
|
||||
start := time.Now()
|
||||
|
||||
var args map[string]any
|
||||
if err := json.Unmarshal([]byte(argsJSON), &args); err != nil {
|
||||
return ToolResult{Success: false, Error: "invalid args JSON: " + err.Error(), DurationMs: ms(start)}
|
||||
}
|
||||
|
||||
var result any
|
||||
var execErr error
|
||||
|
||||
switch toolName {
|
||||
case "shell_exec":
|
||||
result, execErr = e.shellExec(ctx, args)
|
||||
case "file_read":
|
||||
result, execErr = e.fileRead(args)
|
||||
case "file_write":
|
||||
result, execErr = e.fileWrite(args)
|
||||
case "file_list":
|
||||
result, execErr = e.fileList(args)
|
||||
case "http_request":
|
||||
result, execErr = e.httpRequest(ctx, args)
|
||||
case "docker_exec":
|
||||
result, execErr = e.dockerExec(ctx, args)
|
||||
case "list_agents":
|
||||
result, execErr = e.listAgents()
|
||||
case "delegate_to_agent":
|
||||
result, execErr = e.delegateToAgent(args)
|
||||
default:
|
||||
return ToolResult{Success: false, Error: fmt.Sprintf("unknown tool: %s", toolName), DurationMs: ms(start)}
|
||||
}
|
||||
|
||||
if execErr != nil {
|
||||
return ToolResult{Success: false, Error: execErr.Error(), DurationMs: ms(start)}
|
||||
}
|
||||
return ToolResult{Success: true, Result: result, DurationMs: ms(start)}
|
||||
}
|
||||
|
||||
// ─── Tool Implementations ─────────────────────────────────────────────────────
|
||||
|
||||
func (e *Executor) shellExec(ctx context.Context, args map[string]any) (any, error) {
|
||||
command, _ := args["command"].(string)
|
||||
if command == "" {
|
||||
return nil, fmt.Errorf("command is required")
|
||||
}
|
||||
|
||||
// Safety: block destructive patterns
|
||||
blocked := []string{"rm -rf /", "mkfs", "dd if=/dev/zero", ":(){ :|:& };:"}
|
||||
for _, b := range blocked {
|
||||
if strings.Contains(command, b) {
|
||||
return nil, fmt.Errorf("command blocked for safety: contains '%s'", b)
|
||||
}
|
||||
}
|
||||
|
||||
timeoutSec := 30
|
||||
if t, ok := args["timeout"].(float64); ok && t > 0 {
|
||||
timeoutSec = int(t)
|
||||
}
|
||||
|
||||
ctx2, cancel := context.WithTimeout(ctx, time.Duration(timeoutSec)*time.Second)
|
||||
defer cancel()
|
||||
|
||||
cmd := exec.CommandContext(ctx2, "bash", "-c", command)
|
||||
cmd.Dir = e.projectRoot
|
||||
|
||||
out, err := cmd.CombinedOutput()
|
||||
stdout := string(out)
|
||||
if len(stdout) > 20000 {
|
||||
stdout = stdout[:20000] + "\n...[truncated]"
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
// Return partial output even on error
|
||||
return map[string]any{"stdout": stdout, "stderr": err.Error(), "exitCode": cmd.ProcessState.ExitCode()}, nil
|
||||
}
|
||||
return map[string]any{"stdout": stdout, "stderr": "", "exitCode": 0}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) fileRead(args map[string]any) (any, error) {
|
||||
path, _ := args["path"].(string)
|
||||
if path == "" {
|
||||
return nil, fmt.Errorf("path is required")
|
||||
}
|
||||
path = e.resolvePath(path)
|
||||
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
content := string(data)
|
||||
if len(content) > 50000 {
|
||||
content = content[:50000] + "\n...[truncated]"
|
||||
}
|
||||
return map[string]any{"content": content, "size": len(data), "path": path}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) fileWrite(args map[string]any) (any, error) {
|
||||
path, _ := args["path"].(string)
|
||||
content, _ := args["content"].(string)
|
||||
appendMode, _ := args["append"].(bool)
|
||||
|
||||
if path == "" {
|
||||
return nil, fmt.Errorf("path is required")
|
||||
}
|
||||
path = e.resolvePath(path)
|
||||
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
flag := os.O_WRONLY | os.O_CREATE | os.O_TRUNC
|
||||
if appendMode {
|
||||
flag = os.O_WRONLY | os.O_CREATE | os.O_APPEND
|
||||
}
|
||||
|
||||
f, err := os.OpenFile(path, flag, 0644)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
n, err := f.WriteString(content)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"written": n, "path": path, "append": appendMode}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) fileList(args map[string]any) (any, error) {
|
||||
path, _ := args["path"].(string)
|
||||
if path == "" {
|
||||
path = "."
|
||||
}
|
||||
path = e.resolvePath(path)
|
||||
recursive, _ := args["recursive"].(bool)
|
||||
|
||||
var entries []map[string]any
|
||||
|
||||
if recursive {
|
||||
err := filepath.Walk(path, func(p string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
rel, _ := filepath.Rel(path, p)
|
||||
entries = append(entries, map[string]any{
|
||||
"name": rel,
|
||||
"isDir": info.IsDir(),
|
||||
"size": info.Size(),
|
||||
})
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
dirEntries, err := os.ReadDir(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, de := range dirEntries {
|
||||
info, _ := de.Info()
|
||||
size := int64(0)
|
||||
if info != nil {
|
||||
size = info.Size()
|
||||
}
|
||||
entries = append(entries, map[string]any{
|
||||
"name": de.Name(),
|
||||
"isDir": de.IsDir(),
|
||||
"size": size,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return map[string]any{"path": path, "entries": entries, "count": len(entries)}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) httpRequest(ctx context.Context, args map[string]any) (any, error) {
|
||||
url, _ := args["url"].(string)
|
||||
if url == "" {
|
||||
return nil, fmt.Errorf("url is required")
|
||||
}
|
||||
method := "GET"
|
||||
if m, ok := args["method"].(string); ok && m != "" {
|
||||
method = strings.ToUpper(m)
|
||||
}
|
||||
|
||||
var bodyReader io.Reader
|
||||
if body, ok := args["body"].(string); ok && body != "" {
|
||||
bodyReader = strings.NewReader(body)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, method, url, bodyReader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
req.Header.Set("User-Agent", "GoClaw-Gateway/1.0")
|
||||
if headers, ok := args["headers"].(map[string]any); ok {
|
||||
for k, v := range headers {
|
||||
req.Header.Set(k, fmt.Sprintf("%v", v))
|
||||
}
|
||||
}
|
||||
if bodyReader != nil && req.Header.Get("Content-Type") == "" {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
|
||||
resp, err := e.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
respBody, _ := io.ReadAll(resp.Body)
|
||||
text := string(respBody)
|
||||
if len(text) > 10000 {
|
||||
text = text[:10000] + "\n...[truncated]"
|
||||
}
|
||||
|
||||
return map[string]any{
|
||||
"status": resp.StatusCode,
|
||||
"statusText": resp.Status,
|
||||
"body": text,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) dockerExec(ctx context.Context, args map[string]any) (any, error) {
|
||||
command, _ := args["command"].(string)
|
||||
if command == "" {
|
||||
return nil, fmt.Errorf("command is required")
|
||||
}
|
||||
|
||||
ctx2, cancel := context.WithTimeout(ctx, 15*time.Second)
|
||||
defer cancel()
|
||||
|
||||
parts := strings.Fields("docker " + command)
|
||||
cmd := exec.CommandContext(ctx2, parts[0], parts[1:]...)
|
||||
out, err := cmd.CombinedOutput()
|
||||
output := string(out)
|
||||
if len(output) > 10000 {
|
||||
output = output[:10000] + "\n...[truncated]"
|
||||
}
|
||||
if err != nil {
|
||||
return map[string]any{"output": output, "error": err.Error()}, nil
|
||||
}
|
||||
return map[string]any{"output": output}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) listAgents() (any, error) {
|
||||
if e.agentListFn == nil {
|
||||
return map[string]any{"agents": []any{}, "note": "DB not connected"}, nil
|
||||
}
|
||||
agents, err := e.agentListFn()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"agents": agents, "count": len(agents)}, nil
|
||||
}
|
||||
|
||||
func (e *Executor) delegateToAgent(args map[string]any) (any, error) {
|
||||
agentID, _ := args["agentId"].(float64)
|
||||
message, _ := args["message"].(string)
|
||||
if message == "" {
|
||||
return nil, fmt.Errorf("message is required")
|
||||
}
|
||||
// Delegation is handled at orchestrator level; here we return a placeholder
|
||||
return map[string]any{
|
||||
"delegated": true,
|
||||
"agentId": int(agentID),
|
||||
"message": message,
|
||||
"note": "Agent delegation queued — response will be processed in next iteration",
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ─── Helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
func (e *Executor) resolvePath(path string) string {
|
||||
if filepath.IsAbs(path) {
|
||||
return path
|
||||
}
|
||||
return filepath.Join(e.projectRoot, path)
|
||||
}
|
||||
|
||||
func ms(start time.Time) int64 {
|
||||
return time.Since(start).Milliseconds()
|
||||
}
|
||||
159
server/gateway-proxy.ts
Normal file
159
server/gateway-proxy.ts
Normal file
@@ -0,0 +1,159 @@
|
||||
/**
|
||||
* GoClaw Gateway Proxy
|
||||
*
|
||||
* Forwards orchestrator/agent/tool requests from the Node.js tRPC server
|
||||
* to the Go Gateway running on :18789.
|
||||
*
|
||||
* The Go Gateway handles:
|
||||
* - LLM orchestration (tool-use loop)
|
||||
* - Tool execution (shell, file, docker, http)
|
||||
* - Agent listing from DB
|
||||
* - Model listing from Ollama
|
||||
*/
|
||||
|
||||
import { ENV } from "./_core/env";
|
||||
|
||||
const GATEWAY_BASE_URL = process.env.GATEWAY_URL ?? "http://localhost:18789";
|
||||
const GATEWAY_TIMEOUT_MS = 180_000; // 3 min — LLM can be slow
|
||||
|
||||
export interface GatewayMessage {
|
||||
role: "user" | "assistant" | "system";
|
||||
content: string;
|
||||
}
|
||||
|
||||
export interface GatewayToolCallStep {
|
||||
tool: string;
|
||||
args: Record<string, any>;
|
||||
result?: any;
|
||||
error?: string;
|
||||
success: boolean;
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
export interface GatewayChatResult {
|
||||
success: boolean;
|
||||
response: string;
|
||||
toolCalls: GatewayToolCallStep[];
|
||||
model?: string;
|
||||
usage?: {
|
||||
prompt_tokens: number;
|
||||
completion_tokens: number;
|
||||
total_tokens: number;
|
||||
};
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface GatewayOrchestratorConfig {
|
||||
id: number | null;
|
||||
name: string;
|
||||
model: string;
|
||||
temperature: number;
|
||||
maxTokens: number;
|
||||
allowedTools: string[];
|
||||
systemPromptPreview: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the Go Gateway is running and healthy.
|
||||
*/
|
||||
export async function checkGatewayHealth(): Promise<{
|
||||
connected: boolean;
|
||||
latencyMs: number;
|
||||
ollama?: { connected: boolean; latencyMs: number };
|
||||
error?: string;
|
||||
}> {
|
||||
const start = Date.now();
|
||||
try {
|
||||
const res = await fetch(`${GATEWAY_BASE_URL}/health`, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
const latencyMs = Date.now() - start;
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
return { connected: true, latencyMs, ollama: data.ollama };
|
||||
}
|
||||
return { connected: false, latencyMs, error: `HTTP ${res.status}` };
|
||||
} catch (err: any) {
|
||||
return { connected: false, latencyMs: Date.now() - start, error: err.message };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Send a chat message to the Go Orchestrator.
|
||||
*/
|
||||
export async function gatewayChat(
|
||||
messages: GatewayMessage[],
|
||||
model?: string,
|
||||
maxIter?: number
|
||||
): Promise<GatewayChatResult> {
|
||||
try {
|
||||
const res = await fetch(`${GATEWAY_BASE_URL}/api/orchestrator/chat`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ messages, model, maxIter }),
|
||||
signal: AbortSignal.timeout(GATEWAY_TIMEOUT_MS),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const text = await res.text();
|
||||
return {
|
||||
success: false,
|
||||
response: "",
|
||||
toolCalls: [],
|
||||
error: `Gateway error (${res.status}): ${text}`,
|
||||
};
|
||||
}
|
||||
return res.json();
|
||||
} catch (err: any) {
|
||||
return {
|
||||
success: false,
|
||||
response: "",
|
||||
toolCalls: [],
|
||||
error: `Gateway unreachable: ${err.message}. Is the Go Gateway running on ${GATEWAY_BASE_URL}?`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get orchestrator config from Go Gateway.
|
||||
*/
|
||||
export async function getGatewayOrchestratorConfig(): Promise<GatewayOrchestratorConfig | null> {
|
||||
try {
|
||||
const res = await fetch(`${GATEWAY_BASE_URL}/api/orchestrator/config`, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
return res.json();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get list of models from Go Gateway (proxied from Ollama).
|
||||
*/
|
||||
export async function getGatewayModels(): Promise<{ data: { id: string }[] } | null> {
|
||||
try {
|
||||
const res = await fetch(`${GATEWAY_BASE_URL}/api/models`, {
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
return res.json();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get list of agents from Go Gateway.
|
||||
*/
|
||||
export async function getGatewayAgents(): Promise<{ agents: any[]; count: number } | null> {
|
||||
try {
|
||||
const res = await fetch(`${GATEWAY_BASE_URL}/api/agents`, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
return res.json();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
20
todo.md
20
todo.md
@@ -123,3 +123,23 @@
|
||||
- [x] Fix: Chat.tsx shows which model is being used from orchestrator config
|
||||
- [x] Fix: Streamdown markdown rendering for assistant responses
|
||||
- [ ] Add: streaming/SSE for real-time response display
|
||||
|
||||
## Phase 9: Go Gateway Migration (Variant C)
|
||||
- [x] Create gateway/ directory with Go module (git.softuniq.eu/UniqAI/GoClaw/gateway)
|
||||
- [x] Implement config/config.go — env-based configuration
|
||||
- [x] Implement internal/llm/client.go — Ollama API client (chat, models, health)
|
||||
- [x] Implement internal/db/db.go — MySQL connection, agent/config queries
|
||||
- [x] Implement internal/tools/executor.go — Tool Executor (shell_exec, file_read, file_write, file_list, http_request, docker_exec, list_agents)
|
||||
- [x] Implement internal/orchestrator/orchestrator.go — LLM tool-use loop, config from DB
|
||||
- [x] Implement internal/api/handlers.go — REST API handlers
|
||||
- [x] Implement cmd/gateway/main.go — HTTP server with chi router, graceful shutdown
|
||||
- [x] Go Gateway compiles successfully (10.8MB binary)
|
||||
- [x] Create server/gateway-proxy.ts — Node.js proxy client to Go Gateway
|
||||
- [x] Create docker/docker-compose.yml — local dev (control-center + gateway + ollama + db)
|
||||
- [x] Create docker/docker-stack.yml — Docker Swarm production (2 replicas, rolling updates)
|
||||
- [x] Create docker/Dockerfile.gateway — multi-stage Go build
|
||||
- [x] Create docker/Dockerfile.control-center — multi-stage Node.js build
|
||||
- [ ] Update server/routers.ts: replace orchestrator.ts calls with gateway-proxy.ts calls
|
||||
- [ ] Write Go unit tests (gateway/internal/tools/executor_test.go)
|
||||
- [ ] Write Go integration test for orchestrator chat loop
|
||||
- [ ] Push to Gitea (NW)
|
||||
|
||||
Reference in New Issue
Block a user