ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Small edits

Browse Source
This commit is contained in:
revital
2025-03-16 09:40:44 +02:00
parent 0d980cfc29
commit 100c0ac900
2 changed files with 24 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
docs/clearml_serving/clearml_serving_setup.md
View File

@@ -15,7 +15,7 @@ The following page goes over how to set up and upgrade `clearml-serving`.
[free hosted service](https://app.clear.ml)
1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml)
1. Install clearml-serving CLI:
1. Install the `clearml-serving` CLI:
```bash
pip3 install clearml-serving
@@ -27,21 +27,22 @@ The following page goes over how to set up and upgrade `clearml-serving`.
clearml-serving create --name "serving example"
```
The new serving service UID should be printed
This command prints the Serving Service UID:
```console
New Serving Service created: id=aa11bb22aa11bb22
```
Write down the Serving Service UID
Copy the Serving Service UID (e.g., `aa11bb22aa11bb22`), as you will need it in the next steps.
1. Clone the `clearml-serving` repository:
```bash
git clone https://github.com/clearml/clearml-serving.git
```
1. Edit the environment variables file (docker/example.env) with your clearml-server credentials and Serving Service UID.
For example, you should have something like
1. Edit the environment variables file (`docker/example.env`) with your `clearml-server` API credentials and Serving Service UID.
For example:
```bash
cat docker/example.env
```
@@ -55,31 +56,30 @@ The following page goes over how to set up and upgrade `clearml-serving`.
CLEARML_SERVING_TASK_ID="<serving_service_id_here>"
```
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart)
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart):
```bash
cd docker && docker-compose --env-file example.env -f docker-compose.yml up
```
If you need Triton support (keras/pytorch/onnx etc.), use the triton docker-compose file
If you need Triton support (Keras/PyTorch/ONNX etc.), use the triton `docker-compose` file:
```bash
cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up
```
If running on a GPU instance with Triton support (keras/pytorch/onnx etc.), use the triton gpu docker-compose file:
If running on a GPU instance with Triton support (Keras/PyTorch/ONNX etc.), use the triton gpu docker-compose file:
```bash
cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up
```
:::note
Any model that registers with Triton engine will run the pre/post-processing code on the Inference service container,
Any model that registers with Triton engine will run the pre/post-processing code in the Inference service container,
and the model inference itself will be executed on the Triton Engine container.
:::
## Advanced Setup - S3/GS/Azure Access (Optional)
To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage,
add the respective environment variables to your env files (example.env). For further details, see
[Configuring Storage](../integrations/storage.md#configuring-storage).
To enable inference containers to download models from S3, Google Cloud Storage (GS), or Azure,
add access credentials in the respective environment variables to your env files (`example.env`):
```
AWS_ACCESS_KEY_ID
@@ -92,14 +92,21 @@ AZURE_STORAGE_ACCOUNT
AZURE_STORAGE_KEY
```
For further details, see [Configuring Storage](../integrations/storage.md#configuring-storage).
## Upgrading ClearML Serving
**Upgrading to v1.1**
1. Take down the serving containers (`docker-compose` or k8s)
1. Update the `clearml-serving` CLI `pip3 install -U clearml-serving`
1. Shut down the serving containers (`docker-compose` or k8s)
1. Update the `clearml-serving` CLI:
```
pip3 install -U clearml-serving
```
1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the
`clearml-serving` session definitions
`clearml-serving` session definitions.
1. Pull the latest serving containers (`docker-compose pull ...` or k8s)
1. Re-spin serving containers (`docker-compose` or k8s)