From 59c8d343ead061eabf34f8bfc70d06edbe2c770d Mon Sep 17 00:00:00 2001 From: Allegro AI <51604379+allegroai-git@users.noreply.github.com> Date: Sun, 6 Mar 2022 01:39:59 +0200 Subject: [PATCH] Update README.md --- README.md | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index da9d6d3..7566ff7 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ **ClearML Serving - Model deployment made easy** -## **`clearml-serving`
Model Serving (ML/DL) Orchestration and Repository Made Easy** +## **`clearml-serving`
:sparkles: Model Serving (ML/DL) Made Easy :tada:** [![GitHub license](https://img.shields.io/github/license/allegroai/clearml-serving.svg)](https://img.shields.io/github/license/allegroai/clearml-serving.svg) @@ -26,17 +26,17 @@ Features: * Support Machine Learning Models (Scikit Learn, XGBoost, LightGBM) * Support Deep Learning Models (Tensorflow, PyTorch, ONNX) * Customizable RestAPI for serving (i.e. allow per model pre/post-processing for easy integration) -* Flexibility +* Flexible * On-line model deployment * On-line endpoint model/version deployment (i.e. no need to take the service down) * Per model standalone preprocessing and postprocessing python code -* Scalability +* Scalable * Multi model per container * Multi models per serving service * Multi-service support (fully seperated multiple serving service running independently) * Multi cluster support * Out-of-the-box node auto-scaling based on load/usage -* Efficiency +* Efficient * multi-container resource utilization * Support for CPU & GPU nodes * Auto-batching for DL models @@ -55,11 +55,7 @@ Features: ### ClearML Serving Design Principles -* Modular -* Scalable -* Flexible -* Customizable -* Open Source +**Modular** , **Scalable** , **Flexible** , **Customizable** , **Open Source** @@ -67,13 +63,19 @@ Features: ### Concepts -CLI - Secure configuration interface for on-line model upgrade/deployment on running Serving Services -Serving Service Task - Control plane object storing configuration on all the endpoints. Support multiple separated instance, deployed on multiple clusters. -Inference Services - Inference containers, performing model serving pre/post processing. Also support CPU model inferencing. -Serving Engine Services - Inference engine containers (e.g. Nvidia Triton, TorchServe etc.) used by the Inference Services for heavier model inference. -Statistics Service - Single instance per Serving Service collecting and broadcasting model serving & performance statistics -Time-series DB - Statistics collection service used by the Statistics Service, e.g. Prometheus -Dashboard Service - Customizable dashboard-ing solution on top of the collected statistics, e.g. Grafana +**CLI** - Secure configuration interface for on-line model upgrade/deployment on running Serving Services + +**Serving Service Task** - Control plane object storing configuration on all the endpoints. Support multiple separated instance, deployed on multiple clusters. + +**Inference Services** - Inference containers, performing model serving pre/post processing. Also support CPU model inferencing. + +**Serving Engine Services** - Inference engine containers (e.g. Nvidia Triton, TorchServe etc.) used by the Inference Services for heavier model inference. + +**Statistics Service** - Single instance per Serving Service collecting and broadcasting model serving & performance statistics + +**Time-series DB** - Statistics collection service used by the Statistics Service, e.g. Prometheus + +**Dashboards** - Customizable dashboard-ing solution on top of the collected statistics, e.g. Grafana ### prerequisites