ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-03-22 13:07:59 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Edits

Browse Source
This commit is contained in:
revital 2025-03-17 07:35:38 +02:00
parent 4699667939
commit 5f246a9d84
9 changed files with 72 additions and 62 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/deploying_clearml/enterprise_deploy/appgw.md
View File

@ -30,12 +30,12 @@ their instances:
* [Embedding Model Deployment](../../webapp/applications/apps_embed_model_deployment.md)
* [Llama.cpp Model Deployment](../../webapp/applications/apps_llama_deployment.md)
The AI Application Gateway requires an additional component to the ClearML Server deployment: **The ClearML AI App Gateway Router**.
If your ClearML Deployment does not have the AI App Gateway Router properly installed, these application instances may not be accessible.
The AI Application Gateway requires an additional component to the ClearML Server deployment: the **ClearML App Gateway Router**.
If your ClearML Deployment does not have the App Gateway Router properly installed, these application instances may not be accessible.
#### Installation
The AI Application Gateway Router supports two deployment options:
The App Gateway Router supports two deployment options:
* [Docker Compose](appgw_install_compose.md)
* [Kubernetes](appgw_install_k8s.md)

78
docs/deploying_clearml/enterprise_deploy/appgw_install_compose.md
View File

@ -89,15 +89,21 @@ TCP_PORT_END=
```
Edit it according to the following guidelines:
* `PROXY_TAG`: AI Application Gateway proxy tag.
* `ROUTER_TAG`: AI Application Gateway Router tag.
* `ROUTER_NAME`: Unique name for this router, needed in case of [multiple routers on the same tenant](#install-multiple-routers-for-the-same-tenant).
* `ROUTER__WEBSERVER__SERVER_PORT`: Webserver port. Default is 8080 but can be set differently based on network needs.
* `ROUTER_URL`: URL for this router that was previously configured in the load balancer starting with `https://`.
* `CLEARML_API_HOST`: ClearML API server URL usually starting with `https://api.`
* `PROXY_TAG`: AI Application Gateway proxy tag. The Docker image tag for the proxy component, which needs to be
specified during installation. This tag is provided by ClearML to ensure compatibility with the recommended version.
* `ROUTER_TAG`: App Gateway Router tag. The Docker image tag for the router component. It defines the specific version
to be installed and is provided by ClearML as part of the setup process.
* `ROUTER_NAME`: Each router needs to have a unique name across ClearML server tenant (in case of [multiple routers on the same tenant](#multiple-router-in-the-same-tenant)).
* `ROUTER__WEBSERVER__SERVER_PORT`: Webserver port. The default port is 8080, but it can be adjusted to meet specific network requirements.
* `ROUTER_URL`: External address to access the router. This can be the IP address or DNS of the node where the router
is running, or the address of a load balancer if the router operates behind a proxy/load balancer. Clients use this URL
to access AI workload applications (e.g. remote IDE, model deployment, etc.), so it must be reachable and resolvable for them.
* `CLEARML_API_HOST`: ClearML API server URL starting with `https://api.`
* `CLEARML_API_ACCESS_KEY`: ClearML server API key.
* `CLEARML_API_SECRET_KEY`: ClearML server secret key.
* `AUTH_COOKIE_NAME`: Cookie name used by the ClearML server to store the ClearML authentication cookie. This can usually be found in the `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in the ClearML server installation (`/opt/allegro/config/envoy/envoy.yaml`)
* `AUTH_COOKIE_NAME`: Cookie used by the ClearML server to store the ClearML authentication cookie. This can usually be
found in the `envoy.yaml` file in the ClearML server installation (`/opt/allegro/config/envoy/envoy.yaml`), under the
`value_prefix` key starting with `allegro_token`
* `AUTH_SECURE_ENABLED`: Enable the Set-Cookie `secure` parameter. Set to `false` in case services are exposed with `http`.
* `TCP_ROUTER_ADDRESS`: Router external address, can be an IP or the host machine or a load balancer hostname, depends on network configuration
* `TCP_PORT_START`: Start port for the TCP Session feature
@ -111,46 +117,40 @@ sudo docker compose --env-file runtime.env up -d
### Advanced Configuration
#### Running without Certificates
When running on `docker-compose` with an HTTP interface without certificates, set the following entry in the
`runtime.env`:
#### Using Open HTTP
```
AUTH_SECURE_ENABLED=false
```
To deploy the App Gateway Router on open HTTP (without a certificate), set the `AUTH_SECURE_ENABLED` entry
to `false` in the `runtime.env` file.
#### Install Multiple Routers for the Same Tenant
To deploy multiple routers within the same tenant, you must configure each router to handle specific workloads.
#### Multiple Router in the Same Tenant
Using this setting, each router will only route tasks that originated from its assigned queues. This
is important in case you have multiple networks with different agents. For example:
* Tasks started by Agent A can only be reached by Router A (within the same network), but cannot be reached by Router B
* Agent B will handle a separate set of tasks which can only be reached by Router B
The assumption in this case is that Agent A and Agent B will service different queues, and routers must be configured to
route tasks based on these queue definitions.
Each router in the same tenant must have:
* A unique `ROUTER_NAME`
* Distinct set of queues listed in `LISTEN_QUEUE_NAME`
If you have workloads running in separate networks that cannot communicate with each other, you need to deploy multiple
routers, one for each isolated environment. Each router will only process tasks from designated queues, ensuring that
tasks are correctly routed to agents within the same network.
For example:
* **Router-A** `runtime.env`
* If Agent A and Agent B are in separate networks, each must have its own router to receive tasks.
* Router A will handle tasks from Agent A’s queues. Router B will handle tasks from Agent B’s queues.
To achieve this, each router must be configured with:
* A unique `ROUTER_NAME`
* A distinct set of queues defined in `LISTEN_QUEUE_NAME`.
##### Example Configuration
Each router's `runtime.env` file should include:
* Router A:
```
ROUTER_NAME=router-a
LISTEN_QUEUE_NAME=queue1,queue2
ROUTER_NAME=router-a
LISTEN_QUEUE_NAME=queue1,queue2
```
* **Router-B** `runtime.env`
* Router B:
```
ROUTER_NAME=router-b
LISTEN_QUEUE_NAME=queue3,queue4
````
Ensure that `LISTEN_QUEUE_NAME` is included in the [`docker-compose` environment variables](#docker-compose-file) for each router
instance.
ROUTER_NAME=router-b
LISTEN_QUEUE_NAME=queue3,queue4
```
Make sure `LISTEN_QUEUE_NAME` is set in the [`docker-compose` environment variables](#docker-compose-file) for each router instance.

34
docs/deploying_clearml/enterprise_deploy/appgw_install_k8s.md
View File

@ -6,13 +6,14 @@ title: Kubernetes Deployment
The AI Application Gateway is available under the ClearML Enterprise plan.
:::
This guide details the installation of the ClearML AI App Gateway Router.
The AI App Gateway Router enables access to session-based applications like VSCode and Jupyter.
It acts as a proxy, discovering ClearML Tasks running within its namespace and configuring them for user access.
This guide details the installation of the ClearML App Gateway Router.
The App Gateway Router enables access to your AI workloads (e.g. remote IDEs like VSCode and Jupyter, model API interface, etc.).
It acts as a proxy, identifying ClearML Tasks running within its [K8s namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/)
and making them available for network access.
:::important
The AI App Gateway Router must be installed in the same namespace as a dedicated ClearML Agent.
It can only discover ClearML Tasks within its own namespace.
The App Gateway Router must be installed in the same K8s namespace as a dedicated ClearML Agent.
It can only configure access for ClearML Tasks within its own namespace.
:::
@ -44,7 +45,7 @@ Replace `<GITHUB_TOKEN>` with your valid GitHub token that has access to the Cle
### Prepare Values
Before installing the AI App Gateway Router, create a Helm override file:
Before installing the App Gateway Router, create a Helm override file:
```
imageCredentials:
@ -67,13 +68,13 @@ tcpSession:
end:
```
Edit it according to these guidelines:
Configuration options:
* `imageCredentials.password`: ClearML DockerHub Access Token.
* `clearml.apiServerKey`: ClearML server API key.
* `clearml.apiServerSecret`: ClearML server secret key.
* `clearml.apiServerUrlReference`: ClearML API server URL usually starting with `https://api.`.
* `clearml.authCookieName`: Cookie name used by the ClearML server to store the ClearML authentication cookie.
* `clearml.apiServerUrlReference`: ClearML API server URL starting with `https://api.`.
* `clearml.authCookieName`: Cookie used by the ClearML server to store the ClearML authentication cookie.
* `clearml.sslVerify`: Enable or disable SSL certificate validation on `apiserver` calls check.
* `ingress.hostName`: Hostname of router used by the ingress controller to access it.
* `tcpSession.routerAddress`: The external router address (can be an IP, hostname, or load balancer address) depending on your network setup. Ensure this address is accessible for TCP connections.
@ -82,7 +83,7 @@ Edit it according to these guidelines:
* `tcpSession.portRange.end`: End port for the TCP Session feature.
The whole list of supported configuration is available with the command:
The full list of supported configuration is available with the command:
```
helm show readme allegroai-enterprise/clearml-enterprise-task-traffic-router
@ -103,6 +104,15 @@ allegroai-enterprise/clearml-enterprise-task-traffic-router \
Replace the placeholders with the following values:
* `<RELEASE_NAME>` - Unique name for the AI App Gateway Router within the namespace. This name will appear in the UI and be used for the redirection URL.
* `<WORKLOAD_NAMESPACE>` - Namespace that will be shared with a dedicated ClearML Agent.
* `<RELEASE_NAME>` - Unique name for the App Gateway Router within the K8s namespace. This is a required parameter in
Helm, which identifies a specific installation of the chart. The release name also defines the router’s name and
appears in the UI within session app URLs in the IDE. While we typically recommend using a fixed string (e.g.
`clearml-enterprise` or `clearml-agent`), it can be customized to support multiple installations within the same
namespace by assigning different release names.
* `<WORKLOAD_NAMESPACE>` - [Kubernetes Namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/)
where workloads will be executed. This namespace must be shared between a dedicated ClearML Agent and an App
Gateway Router. The agent is responsible for monitoring its assigned task queues and spawning workloads within this
namespace (unless otherwise configured). Meanwhile, the router monitors the same namespace for AI workloads, such as
session-based tasks. The router has a namespace-limited scope, meaning it can only detect and manage tasks within its
assigned namespace.
* `<CHART_VERSION>` - Version recommended by the ClearML Support Team.

6
docs/deploying_clearml/enterprise_deploy/multi_tenant_k8s.md
View File

@ -513,9 +513,9 @@ Create a `NetworkPolicy` in the tenant namespace with the following configuratio
- podSelector: {}
```
### Install AI App Gateway Router Chart
### Install the App Gateway Router Chart
Install the AI App Gateway Router in your Kubernetes cluster, allowing it to manage and route tasks:
Install the App Gateway Router in your Kubernetes cluster, allowing it to manage and route tasks:
1. Prepare the `overrides.yaml` file with the following content:
@ -531,7 +531,7 @@ Install the AI App Gateway Router in your Kubernetes cluster, allowing it to man
hostName: "<unique url in same domain as apiserver/webserver>"
```
2. Install AI App Gateway Router in the specified tenant namespace:
2. Install App Gateway Router in the specified tenant namespace:
```
helm install -n <TENANT_NAMESPACE> \\

2
docs/webapp/applications/apps_embed_model_deployment.md
View File

@ -13,7 +13,7 @@ running, it serves your embedding model through a secure, publicly accessible ne
endpoint activity and shuts down if the model remains inactive for a specified maximum idle time.
:::info AI Application Gateway
The Embedding Model Deployment app makes use of the AI App Gateway Router which implements a secure, authenticated
The Embedding Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
network endpoint for the model.
If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.

2
docs/webapp/applications/apps_gradio.md
View File

@ -16,7 +16,7 @@ The Gradio launcher monitors the Gradio app activity and shuts down if it is ina
<a id="traffic_router"/>
:::important AI Application Gateway
The Gradio Launcher relies on the ClearML AI App Gateway Router which implements user authentication, and redirects requests
The Gradio Launcher relies on the ClearML App Gateway Router which implements user authentication, and redirects requests
to the IP/port served by the Gradio app.
If the ClearML AI application Gateway is not available, the Gradio app might not be accessible.

2
docs/webapp/applications/apps_llama_deployment.md
View File

@ -12,7 +12,7 @@ running, it serves your model through a secure, publicly accessible network endp
and shuts down if the model remains inactive for a specified maximum idle time.
:::important AI Application Gateway
The llama.cpp Model Deployment app makes use of the AI App Gateway Router which implements a secure, authenticated
The llama.cpp Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
network endpoint for the model.
If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.

2
docs/webapp/applications/apps_model_deployment.md
View File

@ -13,7 +13,7 @@ it serves your model through a secure, publicly accessible network endpoint. The
shuts down if the model remains inactive for a specified maximum idle time.
:::info AI Application Gateway
The vLLM Model Deployment app makes use of the AI App Gateway Router which implements a secure, authenticated
The vLLM Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
network endpoint for the model.
If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.

2
docs/webapp/applications/apps_streamlit.md
View File

@ -17,7 +17,7 @@ time.
<a id="traffic_router"/>
:::important AI Application Gateway
The Streamlit Launcher relies on the ClearML AI App Gateway Router which implements user authentication, and redirects requests
The Streamlit Launcher relies on the ClearML App Gateway Router which implements user authentication, and redirects requests
to the IP/port served by the Streamlit app.
If the ClearML AI application Gateway is not available, the Streamlit app might not be accessible.