diff --git a/docs/clearml_serving/clearml_serving_tutorial.md b/docs/clearml_serving/clearml_serving_tutorial.md index e6907c31..f7cf1d93 100644 --- a/docs/clearml_serving/clearml_serving_tutorial.md +++ b/docs/clearml_serving/clearml_serving_tutorial.md @@ -7,7 +7,7 @@ In this tutorial, you will go over the model lifecycle -- from training to servi * Serving the model using **ClearML Serving** * Spinning the inference container -The tutorial will also go over these additional options that you can use with `clearml-serving`: +The tutorial also covers the following`clearml-serving` features: * Automatic model deployment * Canary endpoints * Model performance monitoring @@ -19,18 +19,17 @@ Before executing the steps below, make sure you have completed `clearml-serving` ## Steps ### Step 1: Train Model -Train a model. Work from your local `clearml-serving` repository's root. - - Create a python virtual environment - - Install the script requirements `pip3 install -r examples/sklearn/requirements.txt` - - Execute the [training script](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/train_model.py) - `python3 examples/sklearn/train_model.py`. +Train a model using the example script. Start from the root directory of your local `clearml-serving` repository. +1. Create a Python virtual environment +1. Install the script requirements: `pip3 install -r examples/sklearn/requirements.txt` +1. Execute the [training script](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/train_model.py): `python3 examples/sklearn/train_model.py`. -During execution, ClearML automatically registers the sklearn model and uploads it into the model repository. +During execution, ClearML automatically registers the sklearn model and uploads it to the model repository. For information about explicit model registration, see [Registering and Deploying New Models Manually](#registering-and-deploying-new-models-manually). ### Step 2: Register Model -Register the new Model on the Serving Service. +Register the new Model on the Serving Service: ```bash clearml-serving --id model add --engine sklearn --endpoint "test_model_sklearn" --preprocess "examples/sklearn/preprocess.py" --name "train sklearn model - sklearn-model" --project "serving examples" @@ -44,42 +43,43 @@ The Serving Service's ID is required to register a model, and to execute `clearm :::note -The preprocessing python code is packaged and uploaded to the Serving Service, to be used by any inference container, -and downloaded in realtime when updated +The preprocessing Python code is packaged and uploaded to the Serving Service, to be used by any inference container, +and downloaded in real time when updated ::: ### Step 3: Spin Inference Container -Spin the Inference Container - - Customize container [Dockerfile](https://github.com/allegroai/clearml-serving/blob/main/clearml_serving/serving/Dockerfile) if needed - - Build container: +Spin the Inference Container: +1. Customize container [Dockerfile](https://github.com/allegroai/clearml-serving/blob/main/clearml_serving/serving/Dockerfile) if needed +1. Build container: - ```bash - docker build --tag clearml-serving-inference:latest -f clearml_serving/serving/Dockerfile . - ``` + ```bash + docker build --tag clearml-serving-inference:latest -f clearml_serving/serving/Dockerfile . + ``` - - Spin the inference container: +1. Spin the inference container: - ```bash - docker run -v ~/clearml.conf:/root/clearml.conf -p 8080:8080 -e CLEARML_SERVING_TASK_ID= -e CLEARML_SERVING_POLL_FREQ=5 clearml-serving-inference:latest - ``` + ```bash + docker run -v ~/clearml.conf:/root/clearml.conf -p 8080:8080 -e CLEARML_SERVING_TASK_ID= -e CLEARML_SERVING_POLL_FREQ=5 clearml-serving-inference:latest + ``` -Now, test the new model inference endpoint: -```bash -curl -X POST "http://127.0.0.1:8080/serve/test_model_sklearn" -H "accept: application/json" -H "Content-Type: application/json" -d '{"x0": 1, "x1": 2}' -``` +1. Test the new model inference endpoint: + + ```bash + curl -X POST "http://127.0.0.1:8080/serve/test_model_sklearn" -H "accept: application/json" -H "Content-Type: application/json" -d '{"x0": 1, "x1": 2}' + ``` Now that you have an inference container running, you can add new model inference endpoints directly with the CLI. The -inference container will automatically sync once every 5 minutes. On the first few requests the inference container -needs to download the model file and preprocessing python code, this means the request might take a little longer, once -everything is cached, it will return almost immediately. +inference container will automatically sync every 5 minutes. The first few requests may take longer while the inference container +downloads the model file and preprocessing Python code. Once +everything is cached, responses will return almost immediately. :::note Review the model repository in the ClearML web UI, under the "serving examples" Project on your ClearML account/server ([free hosted](https://app.clear.ml) or [self-deployed](https://github.com/allegroai/clearml-server)). Inference services status, console outputs and machine metrics are available in the ClearML UI in the Serving Service -project (default: "DevOps" project) +project (default: "DevOps" project). ::: ## Registering and Deploying New Models Manually @@ -119,18 +119,18 @@ model name The ClearML Serving Service supports automatic model deployment and upgrades, which is connected with the model repository and API. When the model auto-deploy is configured, new model versions will be automatically deployed when you -`publish` or `tag` a new model in the ClearML model repository. This automation interface allows for simpler CI/CD model +`publish` or `tag` a new model in the ClearML model repository. This automation interface allows for a simpler CI/CD model deployment process, as a single API automatically deploys (or removes) a model from the Serving Service. #### Automatic Model Deployment Example -1. Configure the model auto-update on the Serving Service +1. Configure the model auto-update on the Serving Service: ```bash clearml-serving --id model auto-update --engine sklearn --endpoint "test_model_sklearn_auto" --preprocess "preprocess.py" --name "train sklearn model" --project "serving examples" --max-versions 2` ``` 1. Deploy the Inference container (if not already deployed) -1. Publish a new model the model repository in one of the following ways: +1. Publish a new model in the model repository in one of the following ways: - Go to the "serving examples" project in the ClearML web UI, click on the Models Tab, search for "train sklearn model" right-click and select "Publish" - Use the RestAPI (see [details](https://clear.ml/docs/latest/docs/references/api/models#post-modelspublish_many)) - Use Python interface: @@ -251,10 +251,10 @@ You can also specify per-endpoint log frequency with the `clearml-serving` CLI. See examples of ClearML Serving with other supported frameworks: -* [scikit-learn](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/readme.md) - random data -* [scikit-learn Model Ensemble](https://github.com/allegroai/clearml-serving/blob/main/examples/ensemble/readme.md) - random data -* [XGBoost](https://github.com/allegroai/clearml-serving/blob/main/examples/xgboost/readme.md) - iris dataset -* [LightGBM](https://github.com/allegroai/clearml-serving/blob/main/examples/lightgbm/readme.md) - iris dataset -* [PyTorch](https://github.com/allegroai/clearml-serving/blob/main/examples/pytorch/readme.md) - mnist dataset -* [TensorFlow/Keras](https://github.com/allegroai/clearml-serving/blob/main/examples/keras/readme.md) - mnist dataset -* [Model Pipeline](https://github.com/allegroai/clearml-serving/blob/main/examples/pipeline/readme.md) - random data +* [scikit-learn](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/readme.md) - Random data +* [scikit-learn Model Ensemble](https://github.com/allegroai/clearml-serving/blob/main/examples/ensemble/readme.md) - Random data +* [XGBoost](https://github.com/allegroai/clearml-serving/blob/main/examples/xgboost/readme.md) - Iris dataset +* [LightGBM](https://github.com/allegroai/clearml-serving/blob/main/examples/lightgbm/readme.md) - Iris dataset +* [PyTorch](https://github.com/allegroai/clearml-serving/blob/main/examples/pytorch/readme.md) - MNIST dataset +* [TensorFlow/Keras](https://github.com/allegroai/clearml-serving/blob/main/examples/keras/readme.md) - MNIST dataset +* [Model Pipeline](https://github.com/allegroai/clearml-serving/blob/main/examples/pipeline/readme.md) - Random data diff --git a/docs/webapp/webapp_model_viewing.md b/docs/webapp/webapp_model_viewing.md index 53b66a67..ca3f8c90 100644 --- a/docs/webapp/webapp_model_viewing.md +++ b/docs/webapp/webapp_model_viewing.md @@ -39,8 +39,8 @@ Hover over the model configuration area to access the following actions: * Magnifying glass Search * Copy Copy configuration -* TrashCLEAR (for Draft models) - Delete the configuration -* EDIT (for Draft models) - Modify / Add model configuration +* Trash**CLEAR** (for *Draft* models) - Delete the configuration +* **EDIT** (for *Draft* models) - Modify / Add model configuration ## Label Enumeration