add hosted app gateway

2025-04-06 14:22:29 +00:00 · 2025-03-12 09:18:07 +02:00 · 2025-03-12 09:18:07 +02:00 · b971dcaff9
commit b971dcaff9
parent 0d980cfc29
1 changed files with 156 additions and 0 deletions
--- a/docs/deploying_clearml/enterprise_deploy/appgw_intall_compose_hosted.md
+++ b/docs/deploying_clearml/enterprise_deploy/appgw_intall_compose_hosted.md
@ -0,0 +1,156 @@
+---
+title: Installing AI Application Gateway with docker-compose - hosted server
+---
+
+:::important Enterprise Feature 
+The Application Gateway is available under the ClearML Enterprise plan. 
+:::
+
+The AI Application Gateway enables external access to ClearML tasks, and applications running on workload nodes that
+require HTTP or TCP access. The gateway is configured with an endpoint or external address, making these services 
+accessible from the user's machine, outside the workload nodes’ network.
+
+This guide details the installation of the ClearML AI Application Gateway for ClearML users who use ClearML’s SaaS control 
+plane while hosting their own workload nodes.
+
+## Requirements
+
+* Linux OS (x86) machine with root access  
+* The machine needs to be reachable from your user network  
+* The machine needs to have network reachability to workload nodes  
+* Credentials for the ClearML docker repository  
+* A valid ClearML Server installation
+
+## Recommendations
+
+* For a secure connection, we recommend having a DNS entry and a valid SSL Certificate assigned to the machine IP.
+
+## Host Configuration
+
+### Docker Installation
+
+Installing `docker` and `docker-compose` might vary depending on the specific operating system you're using. Here is an 
+example for AmazonLinux:
+
+```
+sudo dnf -y install docker
+DOCKER_CONFIG="/usr/local/lib/docker"
+sudo mkdir -p $DOCKER_CONFIG/cli-plugins
+sudo curl -SL https://github.com/docker/compose/releases/download/v2.17.3/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
+sudo chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
+sudo systemctl enable docker
+sudo systemctl start docker
+ 
+sudo docker login
+```
+
+Use the ClearML docker hub credentials when prompted by docker login.
+
+### Docker-compose File
+
+This is an example of the docker-compose file you will need to create:
+
+```
+version: '3.5'
+services:
+  task_traffic_webserver:
+    image: clearml/ai-gateway-proxy:${PROXY_TAG:?err}
+    network_mode: "host"
+    restart: unless-stopped
+    container_name: task_traffic_webserver
+    volumes:
+    - ./task_traffic_router/config/nginx:/etc/nginx/conf.d:ro
+    - ./task_traffic_router/config/lua:/usr/local/openresty/nginx/lua:ro
+  task_traffic_router:
+    image: clearml/ai-gateway-router:${ROUTER_TAG:?err}
+    restart: unless-stopped
+    container_name: task_traffic_router
+    volumes:
+    - /var/run/docker.sock:/var/run/docker.sock
+    - ./task_traffic_router/config/nginx:/etc/nginx/conf.d:rw
+    - ./task_traffic_router/config/lua:/usr/local/openresty/nginx/lua:rw
+    environment:
+    - LOGGER_LEVEL=INFO
+    - ROUTER__WEBSERVER__SERVER_PORT="8010"
+    - ROUTER_NAME=${ROUTER_NAME:?err}
+    - ROUTER_URL=${ROUTER_URL:?err}
+    - CLEARML_API_HOST=${CLEARML_API_HOST:?err}
+    - CLEARML_API_ACCESS_KEY=${CLEARML_API_ACCESS_KEY:?err}
+    - CLEARML_API_SECRET_KEY=${CLEARML_API_SECRET_KEY:?err}
+    - AUTH_COOKIE_NAME=${AUTH_COOKIE_NAME:?err}
+    - AUTH_SECURE_ENABLED=${AUTH_SECURE_ENABLED}
+    - TCP_ROUTER_ADDRESS=${TCP_ROUTER_ADDRESS}
+    - TCP_PORT_START=${TCP_PORT_START}
+    - TCP_PORT_END=${TCP_PORT_END}
+```
+
+### Configuration File
+
+You will be provided with a prefilled `runtime.env` file containing the following entries:
+
+```
+# PREFILLED SECTION, PROVIDED BY CLEARML
+PROXY_TAG=
+ROUTER_TAG=
+CLEARML_API_HOST=https://api.
+AUTH_COOKIE_NAME=
+
+# TO BE FILLED BY USER
+ROUTER_NAME=main-router
+ROUTER_URL=http://<ROUTER-HOST-PUBLIC-IP>:8010
+CLEARML_API_ACCESS_KEY=
+CLEARML_API_SECRET_KEY=
+AUTH_SECURE_ENABLED=true
+TCP_ROUTER_ADDRESS=<ROUTER-HOST-PUBLIC-IP>
+TCP_PORT_START=
+TCP_PORT_END=
+```
+
+Edit it according to the following guidelines:
+
+* `ROUTER_NAME`: The name of the Router, which needs to be unique for each tenant.  
+* `CLEARML_API_ACCESS_KEY, CLEARML_API_SECRET_KEY:` API credentials created in the ClearML web UI, for Admin user or Service Account with admin privileges. Make sure to label these credentials clearly, so that they will not be revoked by mistake.  
+* `ROUTER_URL`: The URL for this router. This URL will be shown in the UI of any application for users to access (Like hosted Jupyter or LLM UI).  
+* `TCP_ROUTER_ADDRESS`: The TCP Router external address, which is an IP of the host machine or a load balancer hostname, depending on the customer network configuration.  
+* `TCP_PORT_START`: The start port for the TCP Tasks, chosen by the customer. Ensure that ports are open and can be allocated on the host.  
+* `TCP_PORT_END`: The end port for the TCP Tasks, chosen by the customer. Ensure that ports are open and can be allocated on the host.
+
+### Installation
+
+Run the following command to start the router:
+
+```
+sudo docker compose --env-file runtime.env up -d
+```
+
+### Advanced
+
+#### Running without Certificates
+
+When running on docker-compose with an HTTP interface and without certificates please set the following entry in the *runtime.env* as below:
+
+```
+AUTH_SECURE_ENABLED=false
+```
+
+#### Install Multiple Routers for the Same Tenant
+
+To install multiple routers in the same tenant it is necessary to set parameters to identify and split the workload. Using this setting, each router will only handle routing to tasks that have originated from the specific queues it was assigned. This is important in case you have two different networks with two different agents, and tasks started by Agent A can only be reached by Router A (in the same network), but simply cannot be reached by Router B. The assumption in this case is that Agent A and Agent B will service different queues, and the Routers handling routing to the tasks executed by each agent will need to match the queue definitions.  
+Multiple routers in the same tenant must have different `ROUTER_NAME` and listen to different queues (`LISTEN_QUEUE_NAME`).
+
+**Router-A** *runtime.env*
+
+```
+ROUTER_NAME=router-a
+LISTEN_QUEUE_NAME=queue1,queue2
+```
+
+**Router-2** *runtime.env*
+
+```
+ROUTER_NAME=router-b
+LISTEN_QUEUE_NAME=queue3,queue4
+```
+
+The environment variable `LISTEN_QUEUE_NAME` needs to be specified in the docker-compose file in case.  
+The `LISTEN_QUEUE_NAME` is a list of string names split by a comma. It supports wildcards.