clearml-docs/docs/clearml_serving/clearml_serving_cli.md

title

title
CLI

The clearml-serving utility is a CLI tool for model deployment and orchestration.

The following page provides a reference for clearml-serving's CLI commands:

• list - List running Serving Services
• create - Create a new Serving Service
• metrics - Configure inference metrics Service
• config - Configure a new Serving Service
• model - Configure Model endpoints for a running Service

clearml-serving [-h] [--debug] [--id ID] {list,create,metrics,config,model} 

Parameters

Name	Description	Optional
--id	Serving Service (Control plane) Task ID to configure (if not provided automatically detect the running control plane Task)
--debug	Print debug messages

:::info Service ID The Serving Service's ID (--id) is required to execute the metrics, config, and model commands. :::

list

clearml-serving list [-h]

List running Serving Services.

create

clearml-serving create [-h] [--name NAME] [--tags TAGS [TAGS ...]] [--project PROJECT]

Create a new Serving Service

Parameters

Name	Description	Optional
--name	Serving service's name. Default: Serving-Service
--project	Serving service's project. Default: DevOps
--tags	Serving service's user tags. The serving service can be labeled, which can be useful for organizing

metrics

Configure inference metrics Service

clearml-serving metrics [-h] {add,remove,list}

Parameters

Name	Description	Optional
--add	Add/modify metric for a specific endpoint
--remove	Remove metric from a specific endpoint
--list	list metrics logged on all endpoints

config

Configure a new Serving Service.

clearml-serving {base-serving-url, triton-grpc, kafka-metric-server, metric-log-freq}

Parameters

Name	Description	Optional
--base-serving-url	External base serving service url. Example: http://127.0.0.1:8080/serve
--triton-grpc-server	External ClearML-Triton serving container gRPC address. Example: 127.0.0.1:9001
--kafka-metric-server	External Kafka service url. Example: 127.0.0.1:9092
--metric-log-freq	Set default metric logging frequency. 1.0 is 100% of all requests are logged

model

Configure Model endpoints for an already running Service

clearml-serving model [-h] {list,remove,upload,canary,auto-update,add}

Parameters

Name	Description	Optional
--list	List current models
--remove	Remove model by its endpoint name
--upload	Upload and register model files/folder
--canary	Add model Canary/A/B endpoint
--auto-update	Add/Modify model auto update service
--add	Add/Update model