mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-19 11:38:50 +00:00
Small edits
This commit is contained in:
commit
b440974b78
@ -15,7 +15,7 @@ The following page goes over how to set up and upgrade `clearml-serving`.
|
|||||||
[free hosted service](https://app.clear.ml)
|
[free hosted service](https://app.clear.ml)
|
||||||
1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml)
|
1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml)
|
||||||
|
|
||||||
1. Install clearml-serving CLI:
|
1. Install the `clearml-serving` CLI:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip3 install clearml-serving
|
pip3 install clearml-serving
|
||||||
@ -27,21 +27,22 @@ The following page goes over how to set up and upgrade `clearml-serving`.
|
|||||||
clearml-serving create --name "serving example"
|
clearml-serving create --name "serving example"
|
||||||
```
|
```
|
||||||
|
|
||||||
The new serving service UID should be printed
|
This command prints the Serving Service UID:
|
||||||
|
|
||||||
```console
|
```console
|
||||||
New Serving Service created: id=aa11bb22aa11bb22
|
New Serving Service created: id=aa11bb22aa11bb22
|
||||||
```
|
```
|
||||||
|
|
||||||
Write down the Serving Service UID
|
Copy the Serving Service UID (e.g., `aa11bb22aa11bb22`), as you will need it in the next steps.
|
||||||
|
|
||||||
1. Clone the `clearml-serving` repository:
|
1. Clone the `clearml-serving` repository:
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/clearml/clearml-serving.git
|
git clone https://github.com/clearml/clearml-serving.git
|
||||||
```
|
```
|
||||||
|
|
||||||
1. Edit the environment variables file (docker/example.env) with your clearml-server credentials and Serving Service UID.
|
1. Edit the environment variables file (`docker/example.env`) with your `clearml-server` API credentials and Serving Service UID.
|
||||||
For example, you should have something like
|
For example:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cat docker/example.env
|
cat docker/example.env
|
||||||
```
|
```
|
||||||
@ -55,31 +56,30 @@ The following page goes over how to set up and upgrade `clearml-serving`.
|
|||||||
CLEARML_SERVING_TASK_ID="<serving_service_id_here>"
|
CLEARML_SERVING_TASK_ID="<serving_service_id_here>"
|
||||||
```
|
```
|
||||||
|
|
||||||
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart)
|
1. Spin up the `clearml-serving` containers with `docker-compose` (or if running on Kubernetes, use the helm chart):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd docker && docker-compose --env-file example.env -f docker-compose.yml up
|
cd docker && docker-compose --env-file example.env -f docker-compose.yml up
|
||||||
```
|
```
|
||||||
|
|
||||||
If you need Triton support (keras/pytorch/onnx etc.), use the triton docker-compose file
|
If you need Triton support (Keras/PyTorch/ONNX etc.), use the triton `docker-compose` file:
|
||||||
```bash
|
```bash
|
||||||
cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up
|
cd docker && docker-compose --env-file example.env -f docker-compose-triton.yml up
|
||||||
```
|
```
|
||||||
|
|
||||||
If running on a GPU instance with Triton support (keras/pytorch/onnx etc.), use the triton gpu docker-compose file:
|
If running on a GPU instance with Triton support (Keras/PyTorch/ONNX etc.), use the triton gpu docker-compose file:
|
||||||
```bash
|
```bash
|
||||||
cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up
|
cd docker && docker-compose --env-file example.env -f docker-compose-triton-gpu.yml up
|
||||||
```
|
```
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
Any model that registers with Triton engine will run the pre/post-processing code on the Inference service container,
|
Any model that registers with Triton engine will run the pre/post-processing code in the Inference service container,
|
||||||
and the model inference itself will be executed on the Triton Engine container.
|
and the model inference itself will be executed on the Triton Engine container.
|
||||||
:::
|
:::
|
||||||
|
|
||||||
## Advanced Setup - S3/GS/Azure Access (Optional)
|
## Advanced Setup - S3/GS/Azure Access (Optional)
|
||||||
To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage,
|
To enable inference containers to download models from S3, Google Cloud Storage (GS), or Azure,
|
||||||
add the respective environment variables to your env files (example.env). For further details, see
|
add access credentials in the respective environment variables to your env files (`example.env`):
|
||||||
[Configuring Storage](../integrations/storage.md#configuring-storage).
|
|
||||||
|
|
||||||
```
|
```
|
||||||
AWS_ACCESS_KEY_ID
|
AWS_ACCESS_KEY_ID
|
||||||
@ -92,14 +92,21 @@ AZURE_STORAGE_ACCOUNT
|
|||||||
AZURE_STORAGE_KEY
|
AZURE_STORAGE_KEY
|
||||||
```
|
```
|
||||||
|
|
||||||
|
For further details, see [Configuring Storage](../integrations/storage.md#configuring-storage).
|
||||||
|
|
||||||
## Upgrading ClearML Serving
|
## Upgrading ClearML Serving
|
||||||
|
|
||||||
**Upgrading to v1.1**
|
**Upgrading to v1.1**
|
||||||
|
|
||||||
1. Take down the serving containers (`docker-compose` or k8s)
|
1. Shut down the serving containers (`docker-compose` or k8s)
|
||||||
1. Update the `clearml-serving` CLI `pip3 install -U clearml-serving`
|
1. Update the `clearml-serving` CLI:
|
||||||
|
|
||||||
|
```
|
||||||
|
pip3 install -U clearml-serving
|
||||||
|
```
|
||||||
|
|
||||||
1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the
|
1. Re-add a single existing endpoint with `clearml-serving model add ...` (press yes when asked). It will upgrade the
|
||||||
`clearml-serving` session definitions
|
`clearml-serving` session definitions.
|
||||||
1. Pull the latest serving containers (`docker-compose pull ...` or k8s)
|
1. Pull the latest serving containers (`docker-compose pull ...` or k8s)
|
||||||
1. Re-spin serving containers (`docker-compose` or k8s)
|
1. Re-spin serving containers (`docker-compose` or k8s)
|
||||||
|
|
||||||
|
@ -77,12 +77,12 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents:
|
|||||||
and shuts down instances as needed, according to a resource budget that you set.
|
and shuts down instances as needed, according to a resource budget that you set.
|
||||||
|
|
||||||
|
|
||||||
### Reproducing Tasks
|
### Reproducing Task Runs
|
||||||
|
|
||||||

|

|
||||||

|

|
||||||
|
|
||||||
Use ClearML's web interface to reproduce tasks and edit their details, like hyperparameters or input models, then execute the tasks
|
Use ClearML's web interface to reproduce task runs and edit their details, like hyperparameters or input models, then execute the tasks
|
||||||
with the new configuration on a remote machine.
|
with the new configuration on a remote machine.
|
||||||
|
|
||||||
When ClearML is integrated into a script, it captures and stores configurations, such as hyperparameters
|
When ClearML is integrated into a script, it captures and stores configurations, such as hyperparameters
|
||||||
|
Loading…
Reference in New Issue
Block a user