From 7ba665764f40605af9d31f7400df1c1278a5df8a Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Sun, 12 Jan 2025 15:57:06 +0200 Subject: [PATCH 1/2] Add bash script support when creating task via UI (#998) --- docs/faq.md | 2 +- docs/webapp/webapp_exp_table.md | 283 +++++++++++++------------ docs/webapp/webapp_exp_track_visual.md | 2 +- 3 files changed, 145 insertions(+), 142 deletions(-) diff --git a/docs/faq.md b/docs/faq.md index b08136f2..fcad943c 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -165,7 +165,7 @@ add a [custom column](webapp/webapp_model_table.md#customizing-the-models-table) that metric column. Available custom column options depend upon the models in the table and the metrics that have been attached to them (see [Logging Metrics and Plots](clearml_sdk/model_sdk.md#logging-metrics-and-plots)). -ClearML associates models with the experiments that created them, so you can also add a [custom column](webapp/webapp_exp_table.md#customizing-the-experiments-table) +ClearML associates models with the experiments that created them, so you can also add a [custom column](webapp/webapp_exp_table.md#customizing-the-task-table) in an experiments table and sort by that metric column.
diff --git a/docs/webapp/webapp_exp_table.md b/docs/webapp/webapp_exp_table.md index 1c38b19c..977ca67a 100644 --- a/docs/webapp/webapp_exp_table.md +++ b/docs/webapp/webapp_exp_table.md @@ -1,78 +1,81 @@ --- -title: The Experiments Table +title: The Task Table --- -The experiments table is a [customizable](#customizing-the-experiments-table) list of experiments associated with a project. From the experiments -table, view experiment details, and work with experiments (reset, clone, enqueue, create [tracking leaderboards](../guides/ui/building_leader_board.md) -to monitor experimentation, and more). The experiments table's auto-refresh lets users continually monitor experiment progress. +The task table is a [customizable](#customizing-the-task-table) list of tasks associated with a project. From the tasks +table, view task details, and work with tasks (reset, clone, enqueue, create [tracking leaderboards](../guides/ui/building_leader_board.md) +to monitor experimentation, and more). The task table's auto-refresh lets users continually monitor task progress. -View the experiments in table view Table view, +View the tasks in table view Table view, details view Details view, or comparison view Comparison view -using the buttons on the top left of the page. Use the table view for a comparative view of your experiments according -to columns of interest. Use the details view to access a selected experiment's details, while keeping the experiment list -in view. Details view can also be accessed by double-clicking a specific experiment in the table view to open its details view. -Use the [comparison view](#comparing-experiments) to compare your experiments' scalar and plot results (for a more in -depth comparison, see [Comparing Experiments](webapp_exp_comparing.md)). This view compares -the scalars/plots of currently selected experiments. If no experiments are selected, the first 100 -visible experiments in the table are compared. +using the buttons on the top left of the page. Use the table view for a comparative view of your tasks according +to columns of interest. Use the details view to access a selected task's details, while keeping the task list +in view. Details view can also be accessed by double-clicking a specific task in the table view to open its details view. +Use the [comparison view](#comparing-tasks) to compare your tasks' scalar and plot results (for a more in +depth comparison, see [Comparing Tasks](webapp_exp_comparing.md)). This view compares +the scalars/plots of currently selected tasks. If no tasks are selected, the first 100 +visible tasks in the table are compared. -You can archive experiments so the experiments table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the -table to open the archive and view all archived experiments. From the archive, you can restore -experiments to remove them from the archive. You can also permanently delete experiments. +You can archive tasks so the table doesn't get too cluttered. Click **OPEN ARCHIVE** on the top of the +table to open the archive and view all archived tasks. From the archive, you can restore +tasks to remove them from the archive. You can also permanently delete tasks. -You can download the experiments table as a CSV file by clicking Download +You can download the task table as a CSV file by clicking Download and choosing one of these options: -* **Download onscreen items** - Download the values for experiments currently visible on screen -* **Download all items** - Download the values for all experiments in this project that match the current active filters +* **Download onscreen items** - Download the values for tasks currently visible on screen +* **Download all items** - Download the values for all tasks in this project that match the current active filters The downloaded data consists of the currently displayed table columns. -![Experiment table](../img/webapp_experiment_table.png) +![Task table](../img/webapp_experiment_table.png) -## Creating Experiments +## Creating Tasks -You can create experiments by: +You can create tasks by: * Running code instrumented with ClearML (see [Task Creation](../clearml_sdk/task_sdk.md#task-creation)) -* [Cloning an existing experiment](webapp_exp_reproducing.md) -* Through the UI interface: Input the experiment's details, including its source code and python requirements, and then +* [Cloning an existing task](webapp_exp_reproducing.md) +* Via CLI using [`clearml-task`](../apps/clearml_task.md) +* Through the UI interface: Input the task's details, including its source code and python requirements, and then run it through a [ClearML Queue](../fundamentals/agents_and_queues.md#what-is-a-queue) or save it as a *draft*. -To create an experiment through the UI interface: -1. Click `+ New Experiment` -1. In the `Create Experiment` modal, input the following information: - * **Code** - * Experiment name - * Git +To create a task through the UI interface: +1. Click `+ New Task` +1. In the `Create Task` modal, input the following information: + * **Code** - What this task is going to run + * Task name + * Git - Optional fields for checking out the code from a git repository: * Repository URL * Version specification - one of the following: * Tag * Branch * Commit ID - * Execution Entry Point - * Working Directory - * One of the following - * Script name - * Module (see [python module specification](https://docs.python.org/3/using/cmdline.html#cmdoption-m)) - * Add `Task.init` call - If selected, [`Task.init()`](../references/sdk/task.md#taskinit) call is added to the - entry point. Select if it is not already called within your code + * Entry Point - The code to run + * Working Directory + * Script type - Python/Shell + * Binary - The binary executing the script (e.g. python3, bash etc). + * Type – How the code is provided + * Script - The name of the file to run using the above specified binary + * Module - The name of a python module to run (Python only, see [Python module specification](https://docs.python.org/3/using/cmdline.html#cmdoption-m)) + * Custom code - Directly provide the code to run. Write code, or upload a file: + * File name - The script in which your code is stored. Click `Upload` to upload an existing file. + * Content - The actual code. Click `Edit` to modify the script’s contents. + * Add `Task.init` call (Python only) - If selected, a [`Task.init()`](../references/sdk/task.md#taskinit) call is automatically added to + your script (Use if if you script does not yet make use of ClearML) * **Arguments** (*optional*) - Add [hyperparameter](../fundamentals/hyperparameters.md) values. - * **Environment** (*optional*) - Set up the experiment’s python execution environment using either of the following - options: - * Use Poetry specification - Requires specifying a docker image for the experiment to be executed in. - * Manually specify the python environment configuration: - * Python binary - The python executable to use - * Preinstalled venv - A specific existing virtual environment to use. Requires specifying a docker image for the - experiment to be executed in. - * Python package specification: - * Skip - Assume system packages are available. Requires specifying a docker image for the experiment to be - executed in. - * Use an existing `requirements.txt` file - * Explicitly specify the required packages - * **Docker** (*optional*) - Specify Docker container configuration for executing the experiment - * Image - Docker image to use for running the experiment - * Arguments - Add Docker arguments as a single string - * Startup Script - Add a bash script to be executed inside the Docker before setting up the experiment's environment + * **Environment** (*optional*) - Set up the task’s execution environment + * Python - Python environment settings + * Use Poetry - Force Poetry instead of pip package manager. Disables additional python settings. + * Preinstalled venv - The name of a virtual environment available in the task’s execution environment to use when + running the task. Additionally, specify how to use the virtual environment: + * Skip - Try to automatically detect an available virtual environment, and use it as is. + * Use `requirements.txt` file - Install packages from a `requirements.txt` file into the specified virtual environment. + * Specify Packages - Install the specified packages into the specified virtual environment + * Environment Variables - Set these environment variables when running the task + * **Container** (*optional*) - Specify container configuration for executing the task + * Image - Image to use for running the task + * Arguments - Add container arguments as a single string + * Startup Script - Add a bash script to be executed inside the container before setting up the task's environment :::important For a task to run in the specified container, the ClearML Agent executing the task must be running in @@ -85,39 +88,39 @@ To create an experiment through the UI interface: ::: * **Run** - * Queue - [ClearML Queue](../fundamentals/agents_and_queues.md#what-is-a-queue) where the experiment should be + * Queue - [ClearML Queue](../fundamentals/agents_and_queues.md#what-is-a-queue) where the task should be enqueued for execution - * Output Destination - A URI where experiment outputs should be stored (ClearML file server by default). + * Output Destination - A URI where task outputs should be stored (ClearML file server by default). 1. Once you have input all the information, click one of the following options - * Save as Draft - Save the experiment as a new draft task. - * Run - Enqueue the experiment for execution in the queue specified in the **Run** tab + * Save as Draft - Save the task as a new draft. + * Run - Enqueue the task for execution in the queue specified in the **Run** tab -Once you have completed the experiment creation wizard, the experiment will be saved in your current project (where -you clicked `+ New Experiment`). See what you can do with your experiment in [Experiment Actions](#experiment-actions). +Once you have completed the task creation wizard, the task will be saved in your current project (where +you clicked `+ New Task`). See what you can do with your task in [Task Actions](#task-actions). -## Experiments Table Columns +## Task Table Columns -The experiments table default and customizable columns are described in the following table. +The task table default and customizable columns are described in the following table. | Column | Description | Type | |---|---|---| -| **TYPE** | Type of experiment. ClearML supports multiple [task types](../fundamentals/task.md#task-types) for experimentation, and a variety of workflows and use cases. | Default | -| **NAME** | Experiment name. | Default | -| **TAGS** | Descriptive, user-defined, color-coded tags assigned to experiments. Use tags to classify experiments, and filter the list. See [tagging experiments](webapp_exp_track_visual.md#tagging-experiments). | Default | -| **STATUS** | Experiment state (status). See a list of the [task states and state transitions](../fundamentals/task.md#task-states). If you programmatically set task progress values, you will also see a progress indicator for Running, Failed, and Aborted tasks. See [here](../clearml_sdk/task_sdk.md#tracking-task-progress). | Default | -| **PROJECT** | Name of experiment's project. | Default | -| **USER** | User who created or cloned the experiment. | Default (hidden) | -| **STARTED** | Elapsed time since the experiment started. To view the date and time of start, hover over the elapsed time. | Default | -| **UPDATED** | Elapsed time since the last update to the experiment. To view the date and time of update, hover over the elapsed time. | Default | -| **ITERATION** | Last or most recent iteration of the experiment. | Default | -| **DESCRIPTION** | A description of the experiment. For cloned experiments, the description indicates it was auto generated with a timestamp. | Default (hidden) | -| **RUN TIME** | The current / total running time of the experiment. | Default (hidden) | -| **_Metrics_** | Add metrics column (last, minimum, and/or maximum values). The metrics depend upon the experiments in the table. See [adding metrics](#to-add-metrics). | Customizable | -| **_Hyperparameters_** | Add hyperparameters. The hyperparameters depend upon the experiments in the table. See [adding hyperparameters](#to-add-hyperparameters). | Customizable | +| **TYPE** | Type of task. ClearML supports multiple [task types](../fundamentals/task.md#task-types) for experimentation, and a variety of workflows and use cases. | Default | +| **NAME** | Task name. | Default | +| **TAGS** | Descriptive, user-defined, color-coded tags assigned to tasks. Use tags to classify tasks, and filter the list. See [tagging tasks](webapp_exp_track_visual.md#tagging-tasks). | Default | +| **STATUS** | Task state (status). See a list of the [task states and state transitions](../fundamentals/task.md#task-states). If you programmatically set task progress values, you will also see a progress indicator for Running, Failed, and Aborted tasks. See [here](../clearml_sdk/task_sdk.md#tracking-task-progress). | Default | +| **PROJECT** | Name of task's project. | Default | +| **USER** | User who created or cloned the task. | Default (hidden) | +| **STARTED** | Elapsed time since the task started. To view the date and time of start, hover over the elapsed time. | Default | +| **UPDATED** | Elapsed time since the last update to the task. To view the date and time of update, hover over the elapsed time. | Default | +| **ITERATION** | Last or most recent iteration of the task. | Default | +| **DESCRIPTION** | A description of the task. For cloned tasks, the description indicates it was auto generated with a timestamp. | Default (hidden) | +| **RUN TIME** | The current / total running time of the task. | Default (hidden) | +| **_Metrics_** | Add metrics column (last, minimum, and/or maximum values). The metrics depend upon the tasks in the table. See [adding metrics](#to-add-metrics). | Customizable | +| **_Hyperparameters_** | Add hyperparameters. The hyperparameters depend upon the tasks in the table. See [adding hyperparameters](#to-add-hyperparameters). | Customizable | -## Customizing the Experiments Table +## Customizing the Task Table Customize the table using any of the following: * Dynamic column order - Drag a column title to a different position. @@ -130,13 +133,13 @@ Customize the table using any of the following: main column list. Added columns are by default displayed in the table. You can remove the custom columns from the main column list or the column addition windows. * [Filter columns](#filtering-columns) -* Sort columns - According to metrics and hyperparameters, type of experiment, experiment name, start and last update elapsed time, and last iteration. +* Sort columns - According to metrics and hyperparameters, type of task, task name, start and last update elapsed time, and last iteration. -Use experiments table customization for various use cases, including: +Use task table customization for various use cases, including: -* Creating a [leaderboard](#creating-an-experiment-leaderboard) that will update in real time with experiment +* Creating a [leaderboard](#creating-a-task-leaderboard) that will update in real time with task performance, which can be shared and stored. -* Sorting models by metrics - Models are associated with the experiments that created them. For each metric, use the last +* Sorting models by metrics - Models are associated with the tasks that created them. For each metric, use the last value, the minimal value, and/or the maximal value. * Tracking hyperparameters - Track hyperparameters by adding them as columns, and applying filters and sorting. @@ -144,25 +147,25 @@ Changes are persistent (cached in the browser), and represented in the URL so cu bookmark and shared with other ClearML users to collaborate. :::note -The following experiments-table customizations are saved on a **per-project** basis: +The following task-table customizations are saved on a **per-project** basis: * Columns order * Column width * Active sort order * Active filters * Custom columns -If a project has subprojects, the experiments can be viewed by their subproject groupings or together with -all the experiments in the project. The customizations of these two views are saved separately. +If a project has subprojects, the tasks can be viewed by their subproject groupings or together with +all the tasks in the project. The customizations of these two views are saved separately. ::: ### Adding Metrics and/or Hyperparameters -![Experiment table customization gif](../img/gif/webapp_exp_table_cust.gif) +![Task table customization gif](../img/gif/webapp_exp_table_cust.gif) -Add metrics and/or hyperparameters columns to the experiments table. The metrics and hyperparameters depend upon the -experiments in the table. +Add metrics and/or hyperparameters columns to the task table. The metrics and hyperparameters depend upon the +tasks in the table. #### To Add Metrics: @@ -175,7 +178,7 @@ experiments in the table. hyperparameter checkboxes. :::note Float Values Display -By default, the experiments table displays rounded up float values. Hover over a float to view its precise value in the +By default, the task table displays rounded up float values. Hover over a float to view its precise value in the tooltip that appears. To view all precise values in a column, hover over a float and click Expand. ::: @@ -202,38 +205,38 @@ in the top right corner of the table. -## Experiment Actions +## Task Actions -The following table describes the actions that can be done from the experiments table, including the [states](../fundamentals/task.md#task-states) +The following table describes the actions that can be done from the task table, including the [states](../fundamentals/task.md#task-states) that allow each operation. Access these actions in any of the following ways: -* In the experiments table, right-click an experiment or hover over an experiment and click Dot menu +* In the task table, right-click a task or hover over a task and click Dot menu to open the context menu -* In an experiment info panel, click the menu button Bar menu -* Through the batch action bar: available at screen bottom when multiple experiments are selected +* In a task info panel, click the menu button Bar menu +* Through the batch action bar: available at screen bottom when multiple tasks are selected -| Action | Description | States Valid for the Action | State Transition | -|---|---|---|---| -| Details | Open the experiment's [info panel](webapp_exp_track_visual.md#info-panel) (keeps the experiments list in view). Can also be accessed by double-clicking an experiment in the experiments table. | Any state | None | -| View Full Screen | View experiment details in [full screen](webapp_exp_track_visual.md#full-screen-details-view). | Any state | None | -| Manage Queue | If an experiment is *Pending* in a queue, view the utilization of that queue, manage that queue (remove experiments and change the order of experiments), and view information about the worker(s) listening to the queue. See the [Orchestration](webapp_workers_queues.md) page. | *Enqueued* | None | -| View Worker | If an experiment is *Running*, view resource utilization, worker details, and queues to which a worker is listening. | *Running* | None | -| Share | For **ClearML Hosted Service** users only, [share](webapp_exp_sharing.md) an experiment and its model with a **ClearML Hosted Service** user in another workspace. | Any state | None | -| Archive | Move experiment to the project's archive. If it is shared (ClearML Hosted Service only), the experiment becomes private. | Any state | *Pending* to *Draft* | -| Restore |Action available in the archive. Restore an experiment to the active experiments table.| Any State | None | -| Delete | Action available in the archive. Delete an experiment, which will also remove all their logs, results, artifacts and debug samples. | Any State | N/A | -| Enqueue | Add an experiment to a queue for a worker or workers (listening to the queue) to execute. | *Draft*, *Aborted* | *Pending* | -| Dequeue | Remove an experiment from a queue. | *Pending* | *Draft* | -| Reset | Delete the log and output from a previous run of an experiment (for example, before rerunning it). | *Completed*, *Aborted*, or *Failed* | *Draft* | -| Abort | Manually terminate a *Running* experiment. | *Running* | *Aborted* | -| Abort All Children | Manually terminate all *Running* experiments which have this task as a parent | *Running* or *Aborted* | None for parent experiment, *Aborted* for child experiments | -| Retry | Enqueue a failed experiment in order to rerun it. Make sure you have resolved the external problem which previously prevented the experiment’s completion. | *Failed* | *Pending* | -| Publish | Publish an experiment to prevent changes to its tracking data, inputs, and outputs. Published experiments and their models are read-only. *Published* experiments cannot be enqueued, but they can be cloned, and their clones can be edited, tuned, and enqueued. | *Completed*, *Aborted*, or *Failed*. | *Published* | -| Add Tag | Tag experiments with color-coded labels to assist you in organizing your work. See [tagging experiments](webapp_exp_track_visual.md#tagging-experiments). | Any state | None | -| Clone | Make an exact, editable copy of an experiment (for example, to reproduce an experiment, but keep the original). | *Draft* | Newly Cloned Experiment is *Draft* | -| Move to Project | Move an experiment to another project. | Any state | None | -| Compare | Compare selected experiments (see [Comparing Experiments](webapp_exp_comparing.md)) | Any state | None | +| Action | Description | States Valid for the Action | State Transition | +|---|---|---|-------------------------------------------------| +| Details | Open the task's [info panel](webapp_exp_track_visual.md#info-panel) (keeps the tasks list in view). Can also be accessed by double-clicking a task in the task table. | Any state | None | +| View Full Screen | View task details in [full screen](webapp_exp_track_visual.md#full-screen-details-view). | Any state | None | +| Manage Queue | If a task is *Pending* in a queue, view the utilization of that queue, manage that queue (remove tasks and change the order of tasks), and view information about the worker(s) listening to the queue. See the [Orchestration](webapp_workers_queues.md) page. | *Enqueued* | None | +| View Worker | If a task is *Running*, view resource utilization, worker details, and queues to which a worker is listening. | *Running* | None | +| Share | For **ClearML Hosted Service** users only, [share](webapp_exp_sharing.md) a task and its model with a **ClearML Hosted Service** user in another workspace. | Any state | None | +| Archive | Move task to the project's archive. If it is shared (ClearML Hosted Service only), the task becomes private. | Any state | *Pending* to *Draft* | +| Restore |Action available in the archive. Restore a task to the active task table.| Any State | None | +| Delete | Action available in the archive. Delete a task, which will also remove all their logs, results, artifacts and debug samples. | Any State | N/A | +| Enqueue | Add a task to a queue for a worker or workers (listening to the queue) to execute. | *Draft*, *Aborted* | *Pending* | +| Dequeue | Remove a task from a queue. | *Pending* | *Draft* | +| Reset | Delete the log and output from a previous run of a task (for example, before rerunning it). | *Completed*, *Aborted*, or *Failed* | *Draft* | +| Abort | Manually terminate a *Running* task. | *Running* | *Aborted* | +| Abort All Children | Manually terminate all *Running* tasks which have this task as a parent | *Running* or *Aborted* | None for parent task, *Aborted* for child tasks | +| Retry | Enqueue a failed task in order to rerun it. Make sure you have resolved the external problem which previously prevented the task’s completion. | *Failed* | *Pending* | +| Publish | Publish a task to prevent changes to its tracking data, inputs, and outputs. Published tasks and their models are read-only. *Published* tasks cannot be enqueued, but they can be cloned, and their clones can be edited, tuned, and enqueued. | *Completed*, *Aborted*, or *Failed*. | *Published* | +| Add Tag | Tag tasks with color-coded labels to assist you in organizing your work. See [tagging tasks](webapp_exp_track_visual.md#tagging-experiments). | Any state | None | +| Clone | Make an exact, editable copy of a task (for example, to reproduce a task, but keep the original). | *Draft* | Newly cloned task is *Draft* | +| Move to Project | Move a task to another project. | Any state | None | +| Compare | Compare selected tasks (see [Comparing Tasks](webapp_exp_comparing.md)) | Any state | None | :::important Enterprise Feature The ClearML Enterprise Server provides a mechanism to define your own custom actions, which will @@ -241,43 +244,43 @@ appear in the context menu. Create a custom action by defining an HTTP request t action. For more information see [Custom UI Context Menu Actions](../deploying_clearml/clearml_server_config.md#custom-ui-context-menu-actions). ::: -Most of the actions mentioned in the chart above can be performed on multiple experiments at once. -[Select multiple experiments](#selecting-multiple-experiments), then use either the context menu, or the batch action bar +Most of the actions mentioned in the chart above can be performed on multiple tasks at once. +[Select multiple tasks](#selecting-multiple-tasks), then use either the context menu, or the batch action bar that appears at the bottom of the page, to perform -operations on the selected experiments. Actions can be performed only on the experiments that match the action criteria -(for example, only *Running* experiments can be aborted). The context menu shows the number -of experiments that can be affected by each action. The same information can be found in the batch action bar, in a tooltip that +operations on the selected tasks. Actions can be performed only on the tasks that match the action criteria +(for example, only *Running* tasks can be aborted). The context menu shows the number +of tasks that can be affected by each action. The same information can be found in the batch action bar, in a tooltip that appears when hovering over an action icon. -![Experiment table batch operations](../img/webapp_experiment_table_context_menu.png) +![Task table batch operations](../img/webapp_experiment_table_context_menu.png) -## Selecting Multiple Experiments +## Selecting Multiple Tasks -Select multiple experiments by clicking the checkbox on the left of each relevant experiment. Clear any existing selection +Select multiple tasks by clicking the checkbox on the left of each relevant task. Clear any existing selection by clicking the checkbox in the top left corner of the table. Click the checkbox in the top left corner of the table to select all items currently visible. An extended bulk selection tool is available through the down arrow next to the checkbox in the top left corner, enabling selecting items beyond the items currently on-screen: -* **All** - Select all experiments in the project +* **All** - Select all tasks in the project * **None** - Clear selection -* **Filtered** - Select **all experiments in the project** that match the current active filters in the project +* **Filtered** - Select **all tasks in the project** that match the current active filters in the project -## Comparing Experiments +## Comparing Tasks -The comparison view compares experiment scalar and plot results (for a more in depth comparison, see [Comparing Experiments](webapp_exp_comparing.md)). -When selected, the view presents a comparison of all [selected experiments](#selecting-multiple-experiments). If no -experiments are selected, the first 100 visible experiments in the table are displayed in the comparison. +The comparison view compares task scalar and plot results (for a more in depth comparison, see [Comparing Experiments](webapp_exp_comparing.md)). +When selected, the view presents a comparison of all [selected tasks](#selecting-multiple-tasks). If no +tasks are selected, the first 100 visible tasks in the table are displayed in the comparison. In the dropdown menu, select to view **Scalars** or **Plots**. -**Scalars** shows experiment scalar results as time series line graphs. +**Scalars** shows task scalar results as time series line graphs. ![Merged comparison plots](../img/webapp_compare_view_1.png) All single value scalars are plotted into a single clustered bar chart under the "Summary" title, where each cluster -represents a reported metric, and each bar in the cluster represents an experiment. +represents a reported metric, and each bar in the cluster represents a task. ![Single scalar comparison](../img/webapp_compare_view_3.png) @@ -288,34 +291,34 @@ Click Tune -|Name | Description| Optional | +|Name | Description| Mandatory | |---|----|---| -|`--args`| List of `=` strings to pass to the remote execution. Currently only argparse/click/hydra/fire arguments are supported. Example: `--args lr=0.003 batch_size=64`|Yes| -|`--compute-time-limit`|The maximum compute time in minutes that experiment can consume. If this time limit is exceeded, all jobs are aborted.|Yes| -|`--max-iteration-per-job`|The maximum iterations (of the objective metric) per single job. When iteration maximum is exceeded, the job is aborted.|Yes| -|`--max-number-of-concurrent-tasks`|The maximum number of concurrent Tasks (experiments) running at the same time|Yes| -|`--min-iteration-per-job`|The minimum iterations (of the objective metric) per single job.|Yes| -|`--local`| If set, run the experiments locally. Notice that no new python environment will be created. The `--script` parameter must point to a local file entry point and all arguments must be passed with `--args`| Yes| -|`--objective-metric-series`| Objective metric series to maximize/minimize (e.g. 'loss').|No| -|`--objective-metric-sign`| Optimization target, whether to maximize or minimize the value of the objective metric specified. Possible values: "min", "max", "min_global", "max_global". For more information, see [Optimization Objective](#optimization-objective). |No| -|`--objective-metric-title`| Objective metric title to maximize/minimize (e.g. 'validation').|No| -|`--optimization-time-limit`|The maximum time (minutes) for the optimization to run. The default is `None`, indicating no time limit.|Yes| -|`--optimizer-class`|The optimizer to use. Possible values are: OptimizerOptuna (default), OptimizerBOHB, GridSearch, RandomSearch. For more information, see [Supported Optimizers](../fundamentals/hpo.md#supported-optimizers). |No| -|`--params-search`|Parameters space for optimization. See more information in [Specifying the Parameter Space](#specifying-the-parameter-space). |No| -|`--params-override`|Additional parameters of the base task to override for this parameter search. Use the following JSON format for each parameter: `{"name": "param_name", "value": }`. Windows users, see [JSON format note](#json_note).|Yes| -|`--pool-period-min`|The time between two consecutive polls (minutes).|Yes| -|`--project-name`|Name of the project in which the optimization task will be created. If the project does not exist, it is created. If unspecified, the repository name is used.|Yes| -|`--queue`|Queue to enqueue the experiments on.|Yes| -|`--save-top-k-tasks-only`| Keep only the top \ performing tasks, and archive the rest of the experiments. Input `-1` to keep all tasks. Default: `10`.|Yes| -|`--script`|Script to run the parameter search on. Required unless `--task-id` is specified.|Yes| -|`--task-id`|ID of a ClearML task whose hyperparameters will be optimized. Required unless `--script` is specified.|Yes| -|`--task-name`|Name of the optimization task. If unspecified, the base Python script's file name is used.|Yes| -|`--time-limit-per-job`|Maximum execution time per single job in minutes. When the time limit is exceeded, the job is aborted. Default: no time limit.|Yes| -|`--total-max-jobs`|The total maximum jobs for the optimization process. The default value is `None` for unlimited.|Yes| +|`--args`| List of `=` strings to pass to the remote execution. Currently only argparse/click/hydra/fire arguments are supported. Example: `--args lr=0.003 batch_size=64`|No| +|`--compute-time-limit`|The maximum compute time in minutes that experiment can consume. If this time limit is exceeded, all jobs are aborted.|No| +|`--max-iteration-per-job`|The maximum iterations (of the objective metric) per single job. When iteration maximum is exceeded, the job is aborted.|No| +|`--max-number-of-concurrent-tasks`|The maximum number of concurrent Tasks (experiments) running at the same time|No| +|`--min-iteration-per-job`|The minimum iterations (of the objective metric) per single job.|No| +|`--local`| If set, run the experiments locally. Notice that no new python environment will be created. The `--script` parameter must point to a local file entry point and all arguments must be passed with `--args`| No| +|`--objective-metric-series`| Objective metric series to maximize/minimize (e.g. 'loss').|Yes| +|`--objective-metric-sign`| Optimization target, whether to maximize or minimize the value of the objective metric specified. Possible values: "min", "max", "min_global", "max_global". For more information, see [Optimization Objective](#optimization-objective). |Yes| +|`--objective-metric-title`| Objective metric title to maximize/minimize (e.g. 'validation').|Yes| +|`--optimization-time-limit`|The maximum time (minutes) for the optimization to run. The default is `None`, indicating no time limit.|No| +|`--optimizer-class`|The optimizer to use. Possible values are: OptimizerOptuna (default), OptimizerBOHB, GridSearch, RandomSearch. For more information, see [Supported Optimizers](../fundamentals/hpo.md#supported-optimizers). |Yes| +|`--params-search`|Parameters space for optimization. See more information in [Specifying the Parameter Space](#specifying-the-parameter-space). |Yes| +|`--params-override`|Additional parameters of the base task to override for this parameter search. Use the following JSON format for each parameter: `{"name": "param_name", "value": }`. Windows users, see [JSON format note](#json_note).|No| +|`--pool-period-min`|The time between two consecutive polls (minutes).|No| +|`--project-name`|Name of the project in which the optimization task will be created. If the project does not exist, it is created. If unspecified, the repository name is used.|No| +|`--queue`|Queue to enqueue the experiments on.|No| +|`--save-top-k-tasks-only`| Keep only the top \ performing tasks, and archive the rest of the experiments. Input `-1` to keep all tasks. Default: `10`.|No| +|`--script`|Script to run the parameter search on. Required unless `--task-id` is specified.|No| +|`--task-id`|ID of a ClearML task whose hyperparameters will be optimized. Required unless `--script` is specified.|No| +|`--task-name`|Name of the optimization task. If unspecified, the base Python script's file name is used.|No| +|`--time-limit-per-job`|Maximum execution time per single job in minutes. When the time limit is exceeded, the job is aborted. Default: no time limit.|No| +|`--total-max-jobs`|The total maximum jobs for the optimization process. The default value is `None` for unlimited.|No| diff --git a/docs/apps/clearml_task.md b/docs/apps/clearml_task.md index 1d50f260..cd390f6c 100644 --- a/docs/apps/clearml_task.md +++ b/docs/apps/clearml_task.md @@ -53,31 +53,31 @@ errors in identifying the correct default branch.
-|Name | Description| Optional | +|Name | Description| Mandatory | |---|----|---| -| `--args` | Arguments to pass to the remote task, list of `=` strings. Currently only argparse arguments are supported | Yes | -| `--base-task-id` | Use a pre-existing task in the system, instead of a local repo / script. Essentially clones an existing task and overrides arguments / requirements | Yes | -| `--binary` | Binary executable used to launch the entry point. For example: `--binary python3`, `--binary /bin/bash`. By default, the binary will be auto-detected | Yes | -| `--branch` | Select repository branch / tag. By default, latest commit from the master branch | Yes | -| `--commit` | Select commit ID to use. By default, latest commit, or local commit ID when using local repository | Yes | -| `--cwd` | Working directory to launch the script from. Relative to repo root or local `--folder` | Yes | -| `--docker` | Select the Docker image to use in the remote task | Yes | -| `--docker_bash_setup_script` | Add a bash script to be executed inside the Docker before setting up the task's environment | Yes | -| `--docker_args` | Add Docker arguments. Pass a single string in the following format: `--docker_args ""`. For example: `--docker_args "-v some_dir_1:other_dir_1 -v some_dir_2:other_dir_2"` | Yes | -| `--folder` | Execute the code from a local folder. Notice, it assumes a git repository already exists. Current state of the repo (commit ID and uncommitted changes) is logged and replicated on the remote machine | Yes | -| `--import-offline-session`| Specify the path to the offline session you want to import.| Yes | -| `--name` | Set a target name for the new task | No | -| `--output-uri` | Set the task `output_uri`, upload destination for task models and artifacts | Yes | -| `--packages` | Manually specify a list of required packages. Example: `--packages "tqdm>=2.1" "scikit-learn"` | Yes | -| `--project`| Set the project name for the task (required, unless using `--base-task-id`). If the named project does not exist, it is created on-the-fly | No | -| `--queue` | Select a task's execution queue. If not provided, a task is created but not launched | Yes | -| `--repo` | URL of remote repository. Example: `--repo https://github.com/allegroai/clearml.git` | Yes | -| `--requirements` | Specify `requirements.txt` file to install when setting the session. By default, the` requirements.txt` from the repository will be used | Yes | -| `--script` | Entry point script for the remote execution. When used with `--repo`, input the script's relative path inside the repository. For example: `--script source/train.py`. When used with `--folder`, it supports a direct path to a file inside the local repository itself, for example: `--script ~/project/source/train.py` | No | -| `--skip-task-init` | If set, `Task.init()` call is not added to the entry point, and is assumed to be called within the script | Yes | -| `--tags` | Add tags to the newly created task. For example: `--tags "base" "job"` | Yes | -| `--task-type` | Set the task type. Optional values: training, testing, inference, data_processing, application, monitor, controller, optimizer, service, qc, custom | Yes | -| `--version` | Display the `clearml-task` utility version | Yes | +| `--args` | Arguments to pass to the remote task, list of `=` strings. Currently only argparse arguments are supported | No | +| `--base-task-id` | Use a pre-existing task in the system, instead of a local repo / script. Essentially clones an existing task and overrides arguments / requirements | No | +| `--binary` | Binary executable used to launch the entry point. For example: `--binary python3`, `--binary /bin/bash`. By default, the binary will be auto-detected | No | +| `--branch` | Select repository branch / tag. By default, latest commit from the master branch | No | +| `--commit` | Select commit ID to use. By default, latest commit, or local commit ID when using local repository | No | +| `--cwd` | Working directory to launch the script from. Relative to repo root or local `--folder` | No | +| `--docker` | Select the Docker image to use in the remote task | No | +| `--docker_bash_setup_script` | Add a bash script to be executed inside the Docker before setting up the task's environment | No | +| `--docker_args` | Add Docker arguments. Pass a single string in the following format: `--docker_args ""`. For example: `--docker_args "-v some_dir_1:other_dir_1 -v some_dir_2:other_dir_2"` | No | +| `--folder` | Execute the code from a local folder. Notice, it assumes a git repository already exists. Current state of the repo (commit ID and uncommitted changes) is logged and replicated on the remote machine | No | +| `--import-offline-session`| Specify the path to the offline session you want to import.| No | +| `--name` | Set a target name for the new task | Yes | +| `--output-uri` | Set the task `output_uri`, upload destination for task models and artifacts | No | +| `--packages` | Manually specify a list of required packages. Example: `--packages "tqdm>=2.1" "scikit-learn"` | No | +| `--project`| Set the project name for the task (required, unless using `--base-task-id`). If the named project does not exist, it is created on-the-fly | Yes | +| `--queue` | Select a task's execution queue. If not provided, a task is created but not launched | No | +| `--repo` | URL of remote repository. Example: `--repo https://github.com/allegroai/clearml.git` | No | +| `--requirements` | Specify `requirements.txt` file to install when setting the session. By default, the` requirements.txt` from the repository will be used | No | +| `--script` | Entry point script for the remote execution. When used with `--repo`, input the script's relative path inside the repository. For example: `--script source/train.py`. When used with `--folder`, it supports a direct path to a file inside the local repository itself, for example: `--script ~/project/source/train.py` | Yes | +| `--skip-task-init` | If set, `Task.init()` call is not added to the entry point, and is assumed to be called within the script | No | +| `--tags` | Add tags to the newly created task. For example: `--tags "base" "job"` | No | +| `--task-type` | Set the task type. Optional values: training, testing, inference, data_processing, application, monitor, controller, optimizer, service, qc, custom | No | +| `--version` | Display the `clearml-task` utility version | No |
diff --git a/docs/clearml_agent/clearml_agent_ref.md b/docs/clearml_agent/clearml_agent_ref.md index eff7245d..161ebab3 100644 --- a/docs/clearml_agent/clearml_agent_ref.md +++ b/docs/clearml_agent/clearml_agent_ref.md @@ -32,22 +32,22 @@ clearml-agent build [-h] --id TASK_ID [--target TARGET] ### Parameters -|Name | Description| Optional | +|Name | Description| Mandatory | |---|----|---| -|`--id`| Build a worker environment for this Task ID.|No| -|`--cpu-only`| Disable GPU access for the Docker container.|Yes| -|`--docker`| Run agent in Docker mode. Specify a Docker container that a worker will execute at launch. To specify the image name and optional arguments, use one of the following:
  • `--docker ` on the command line
  • `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Docker containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also lets you execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
To limit GPU visibility for Docker, set the `NVIDIA_VISIBLE_DEVICES` environment variable.| Yes| -|`--entry-point`| Used in conjunction with `--docker`, indicates how to run the Task specified by `--task-id` on Docker startup. The `--entry-point` options are:
  • `reuse` - Overwrite the existing Task data.
  • `clone_task` - Clone the Task, and execute the cloned Task.
|Yes| -|`--force-docker`| Force using the agent-specified docker image (either explicitly in the `--docker` argument or using the agent's default docker image). If provided, the agent will not use any docker container information stored in the task itself (default `False`)|Yes| -|`--git-pass`| Git password for repository access.|Yes| -|`--git-user`| Git username for repository access.|Yes| -|`--gpus`| Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|Yes| -|`-h`, `--help`| Get help for this command.|Yes| -|`--install-globally`| Install the required Python packages before creating the virtual environment. Use `agent.package_manager.system_site_packages` to control the installation of the system packages. When `--docker` is used, `--install-globally` is always true.|Yes| -|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|Yes| -|`--python-version`| Virtual environment Python version to use.|Yes| -|`-O`| Compile optimized pyc code (see [python documentation](https://docs.python.org/3/using/cmdline.html#cmdoption-O)). Repeat for more optimization.|Yes| -|`--target`| The target folder for the virtual environment and source code that will be used at launch.|Yes| +|`--id`| Build a worker environment for this Task ID.|Yes| +|`--cpu-only`| Disable GPU access for the Docker container.|No| +|`--docker`| Run agent in Docker mode. Specify a Docker container that a worker will execute at launch. To specify the image name and optional arguments, use one of the following:
  • `--docker ` on the command line
  • `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Docker containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also lets you execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
To limit GPU visibility for Docker, set the `NVIDIA_VISIBLE_DEVICES` environment variable.| No| +|`--entry-point`| Used in conjunction with `--docker`, indicates how to run the Task specified by `--task-id` on Docker startup. The `--entry-point` options are:
  • `reuse` - Overwrite the existing Task data.
  • `clone_task` - Clone the Task, and execute the cloned Task.
|No| +|`--force-docker`| Force using the agent-specified docker image (either explicitly in the `--docker` argument or using the agent's default docker image). If provided, the agent will not use any docker container information stored in the task itself (default `False`)|No| +|`--git-pass`| Git password for repository access.|No| +|`--git-user`| Git username for repository access.|No| +|`--gpus`| Specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|No| +|`-h`, `--help`| Get help for this command.|No| +|`--install-globally`| Install the required Python packages before creating the virtual environment. Use `agent.package_manager.system_site_packages` to control the installation of the system packages. When `--docker` is used, `--install-globally` is always true.|No| +|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|No| +|`--python-version`| Virtual environment Python version to use.|No| +|`-O`| Compile optimized pyc code (see [python documentation](https://docs.python.org/3/using/cmdline.html#cmdoption-O)). Repeat for more optimization.|No| +|`--target`| The target folder for the virtual environment and source code that will be used at launch.|No| ## config List your ClearML Agent configuration. @@ -80,31 +80,31 @@ clearml-agent daemon [-h] [--foreground] [--queue QUEUES [QUEUES ...]] [--order- ### Parameters -|Name | Description| Optional | +|Name | Description| Mandatory | |---|----|---| -|`--child-report-tags`| List of tags to send with the status reports from the worker that executes a task.|Yes| -|`--cpu-only`| If running in Docker mode (see the `--docker` option), disable GPU access for the Docker container or virtual environment.|Yes| -|`--create-queue`| If the queue name provided with `--queue` does not exist in the server, create it on-the-fly and use it.|Yes| -|`--detached`| Run agent in the background. The `clearml-agent` command returns immediately.|Yes| -|`--docker`| Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, either:
  • Use `--docker ` on the command line, or
  • Use `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Docker containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also lets you execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
|Yes| -|`--downtime`| Specify downtime for clearml-agent in ` ` format. For example, use `09-13 TUE` to set Tuesday's downtime to 09-13.

NOTES:
  • This feature is available under the ClearML Enterprise plan
  • Make sure to configure only `--uptime` or `--downtime`, but not both.
|Yes| -|`--dynamic-gpus`| Allow to dynamically allocate GPUs based on queue properties, configure with `--queue =`. For example: `--dynamic-gpus --queue dual_gpus=2 single_gpu=1`

NOTE: This feature is available under the ClearML Enterprise plan|Yes| -|`--force-current-version`| To use your current version of ClearML Agent when running in Docker mode (the `--docker` argument is specified), instead of the latest ClearML Agent version which is automatically installed, specify `force-current-version`.

For example, if your current ClearML Agent version is `0.13.1`, but the latest version of ClearML Agent is `0.13.3`, use `--force-current-version` and your Task will execute in the Docker container with ClearML Agent version `0.13.1`.|Yes| -|`--foreground`| Pipe full log to stdout/stderr. Do not use this option if running in background.|Yes| -|`--git-pass`| Git password for repository access.|Yes| -|`--git-user`| Git username for repository access|Yes| -|`--gpus`| If running in Docker mode (see the `--docker` option), specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|Yes| -|`-h`, `--help`| Get help for this command.|Yes| -|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|Yes| -|`-O`| Compile optimized pyc code (see [python documentation](https://docs.python.org/3/using/cmdline.html#cmdoption-O)). Repeat for more optimization.|Yes| -|`--order-fairness`| Pull from each queue in a round-robin order, instead of priority order.|Yes| -|`--queue`| Specify the queues which the worker is listening to. The values can be any combination of:
  • One or more queue IDs
  • One or more queue names
  • `default` indicating the default queue
|Yes| -|`--services-mode`| Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is verified. Use in CPU mode (`--cpu-only`) only.
To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the `--services-mode` option (e.g. `--services-mode 5`)|Yes| -|`--standalone-mode`| Do not use any network connects. This assumes all requirements are pre-installed.|Yes| -|`--status`| Print the worker's schedule (uptime properties, server's runtime properties and listening queues)|Yes| -|`--stop`| Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are terminated in lexicographical order.|Yes| -|`--uptime`| Specify uptime for clearml-agent in ` ` format. For example, use `17-20 TUE` to set Tuesday's uptime to 17-20.

NOTES:
  • This feature is available under the ClearML Enterprise plan
  • Make sure to configure only `--uptime` or `--downtime`, but not both.
|Yes| -|`--use-owner-token`| Generate and use the task owner's token for the execution of the task.|Yes| +|`--child-report-tags`| List of tags to send with the status reports from the worker that executes a task.|No| +|`--cpu-only`| If running in Docker mode (see the `--docker` option), disable GPU access for the Docker container or virtual environment.|No| +|`--create-queue`| If the queue name provided with `--queue` does not exist in the server, create it on-the-fly and use it.|No| +|`--detached`| Run agent in the background. The `clearml-agent` command returns immediately.|No| +|`--docker`| Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, either:
  • Use `--docker ` on the command line, or
  • Use `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Docker containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also lets you execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
|No| +|`--downtime`| Specify downtime for clearml-agent in ` ` format. For example, use `09-13 TUE` to set Tuesday's downtime to 09-13.

NOTES:
  • This feature is available under the ClearML Enterprise plan
  • Make sure to configure only `--uptime` or `--downtime`, but not both.
|No| +|`--dynamic-gpus`| Allow to dynamically allocate GPUs based on queue properties, configure with `--queue =`. For example: `--dynamic-gpus --queue dual_gpus=2 single_gpu=1`

NOTE: This feature is available under the ClearML Enterprise plan|No| +|`--force-current-version`| To use your current version of ClearML Agent when running in Docker mode (the `--docker` argument is specified), instead of the latest ClearML Agent version which is automatically installed, specify `force-current-version`.

For example, if your current ClearML Agent version is `0.13.1`, but the latest version of ClearML Agent is `0.13.3`, use `--force-current-version` and your Task will execute in the Docker container with ClearML Agent version `0.13.1`.|No| +|`--foreground`| Pipe full log to stdout/stderr. Do not use this option if running in background.|No| +|`--git-pass`| Git password for repository access.|No| +|`--git-user`| Git username for repository access|No| +|`--gpus`| If running in Docker mode (see the `--docker` option), specify the active GPUs for the Docker containers to use. These are the same GPUs set in the `NVIDIA_VISIBLE_DEVICES` environment variable. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|No| +|`-h`, `--help`| Get help for this command.|No| +|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|No| +|`-O`| Compile optimized pyc code (see [python documentation](https://docs.python.org/3/using/cmdline.html#cmdoption-O)). Repeat for more optimization.|No| +|`--order-fairness`| Pull from each queue in a round-robin order, instead of priority order.|No| +|`--queue`| Specify the queues which the worker is listening to. The values can be any combination of:
  • One or more queue IDs
  • One or more queue names
  • `default` indicating the default queue
|No| +|`--services-mode`| Launch multiple long-term docker services. Spin multiple, simultaneous Tasks, each in its own Docker container, on the same machine. Each Task will be registered as a new node in the system, providing tracking and transparency capabilities. Start up and shutdown of each Docker is verified. Use in CPU mode (`--cpu-only`) only.
To limit the number of simultaneous tasks run in services mode, pass the maximum number immediately after the `--services-mode` option (e.g. `--services-mode 5`)|No| +|`--standalone-mode`| Do not use any network connects. This assumes all requirements are pre-installed.|No| +|`--status`| Print the worker's schedule (uptime properties, server's runtime properties and listening queues)|No| +|`--stop`| Terminate a running ClearML Agent, if other arguments are the same. If no additional arguments are provided, agents are terminated in lexicographical order.|No| +|`--uptime`| Specify uptime for clearml-agent in ` ` format. For example, use `17-20 TUE` to set Tuesday's uptime to 17-20.

NOTES:
  • This feature is available under the ClearML Enterprise plan
  • Make sure to configure only `--uptime` or `--downtime`, but not both.
|No| +|`--use-owner-token`| Generate and use the task owner's token for the execution of the task.|No| ## execute @@ -123,23 +123,23 @@ clearml-agent execute [-h] --id TASK_ID [--log-file LOG_FILE] [--disable-monitor ### Parameters -|Name | Description| Optional | +|Name | Description| Mandatory | |---|----|---| -|`--id`| The ID of the Task to build|No| -|`--clone`| Clone the Task specified by `--id`, and then execute that cloned Task.|Yes| -|`--cpu-only`| Disable GPU access for the daemon, only use CPU in either docker or virtual environment.|Yes| -|`--docker`| Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, use one of the following:
  • `--docker ` on the command line
  • `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Dockers containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also lets you execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
| Yes| -|`--disable-monitoring`| Disable logging and monitoring, except for stdout.|Yes| -|`--full-monitoring`| Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout.|Yes| -|`--git-pass`| Git password for repository access.|Yes| -|`--git-user`| Git username for repository access.|Yes| -|`--gpus`| Specify active GPUs for the daemon to use (docker / virtual environment). Equivalent to setting `NVIDIA_VISIBLE_DEVICES`. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|Yes| -|`-h`, `--help`| Get help for this command.|Yes| -|`--log-file`| The log file for Task execution output (stdout / stderr) to a text file.|Yes| -|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|Yes| -|`-O`| Compile optimized pyc code (see [python documentation](https://docs.python.org/3/using/cmdline.html#cmdoption-O)). Repeat for more optimization.|Yes| -|`--require-queue`| If the specified task is not queued, the execution will fail (used for 3rd party scheduler integration, e.g. K8s, SLURM, etc.)|Yes| -|`--standalone-mode`| Do not use any network connects, assume everything is pre-installed|Yes| +|`--id`| The ID of the Task to build|Yes| +|`--clone`| Clone the Task specified by `--id`, and then execute that cloned Task.|No| +|`--cpu-only`| Disable GPU access for the daemon, only use CPU in either docker or virtual environment.|No| +|`--docker`| Run in Docker mode. Execute the Task inside a Docker container. To specify the image name and optional arguments, use one of the following:
  • `--docker ` on the command line
  • `--docker` on the command line, and specify the default image name and arguments in the configuration file.
Environment variable settings for Dockers containers:
  • `CLEARML_DOCKER_SKIP_GPUS_FLAG` - Ignore the `--gpus` flag inside the Docker container. This also lets you execute ClearML Agent using Docker versions earlier than 19.03.
  • `NVIDIA_VISIBLE_DEVICES` - Limit GPU visibility for the Docker container.
  • `CLEARML_AGENT_GIT_USER` and `CLEARML_AGENT_GIT_PASS` - Pass these credentials to the Docker container at execution.
| No| +|`--disable-monitoring`| Disable logging and monitoring, except for stdout.|No| +|`--full-monitoring`| Create a full log, including the environment setup log, Task log, and monitoring, as well as stdout.|No| +|`--git-pass`| Git password for repository access.|No| +|`--git-user`| Git username for repository access.|No| +|`--gpus`| Specify active GPUs for the daemon to use (docker / virtual environment). Equivalent to setting `NVIDIA_VISIBLE_DEVICES`. For example:
  • `--gpus 0`
  • `--gpu 0,1,2`
  • `--gpus all`
|No| +|`-h`, `--help`| Get help for this command.|No| +|`--log-file`| The log file for Task execution output (stdout / stderr) to a text file.|No| +|`--log-level`| SDK log level. The values are:
  • `DEBUG`
  • `INFO`
  • `WARN`
  • `WARNING`
  • `ERROR`
  • `CRITICAL`
|No| +|`-O`| Compile optimized pyc code (see [python documentation](https://docs.python.org/3/using/cmdline.html#cmdoption-O)). Repeat for more optimization.|No| +|`--require-queue`| If the specified task is not queued, the execution will fail (used for 3rd party scheduler integration, e.g. K8s, SLURM, etc.)|No| +|`--standalone-mode`| Do not use any network connects, assume everything is pre-installed|No| ## list diff --git a/docs/clearml_data/clearml_data_cli.md b/docs/clearml_data/clearml_data_cli.md index a9c80966..bc205cf8 100644 --- a/docs/clearml_data/clearml_data_cli.md +++ b/docs/clearml_data/clearml_data_cli.md @@ -28,14 +28,14 @@ clearml-data create [-h] [--parents [PARENTS [PARENTS ...]]] [--project PROJECT]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--name` |Dataset's name| No | -|`--project`|Dataset's project| No | -|`--version` |Dataset version. Use the [semantic versioning](https://semver.org) scheme. If not specified a version will automatically be assigned | Yes | -|`--parents`|IDs of the dataset's parents. The dataset inherits all of its parents' content. Multiple parents can be entered, but they are merged in the order they were entered| Yes | -|`--output-uri`| Sets where dataset and its previews are uploaded to| Yes| -|`--tags` |Dataset user tags. The dataset can be labeled, which can be useful for organizing datasets| Yes| +|`--name` |Dataset's name| Yes | +|`--project`|Dataset's project| Yes | +|`--version` |Dataset version. Use the [semantic versioning](https://semver.org) scheme. If not specified a version will automatically be assigned | No | +|`--parents`|IDs of the dataset's parents. The dataset inherits all of its parents' content. Multiple parents can be entered, but they are merged in the order they were entered| No | +|`--output-uri`| Sets where dataset and its previews are uploaded to| No| +|`--tags` |Dataset user tags. The dataset can be labeled, which can be useful for organizing datasets| No|
@@ -63,15 +63,15 @@ clearml-data add [-h] [--id ID] [--dataset-folder DATASET_FOLDER]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id` | Dataset's ID. Default: previously created / accessed dataset| Yes | -|`--files`| Files / folders to add. Items will be uploaded to the dataset's designated storage. | Yes | -|`--wildcard`| Add specific set of files, denoted by these wildcards. For example: `~/data/*.jpg ~/data/json`. Multiple wildcards can be passed. | Yes | -|`--links`| Files / folders link to add. Supports S3, GS, Azure links. Example: `s3://bucket/data` `azure://.blob.core.windows.net/path/to/file`. Items remain in their original location. | Yes | -|`--dataset-folder` | Dataset base folder to add the files to in the dataset. Default: dataset root| Yes | -|`--non-recursive` | Disable recursive scan of files | Yes | -|`--verbose` | Verbose reporting | Yes| +|`--id` | Dataset's ID. Default: previously created / accessed dataset| No | +|`--files`| Files / folders to add. Items will be uploaded to the dataset's designated storage. | No | +|`--wildcard`| Add specific set of files, denoted by these wildcards. For example: `~/data/*.jpg ~/data/json`. Multiple wildcards can be passed. | No | +|`--links`| Files / folders link to add. Supports S3, GS, Azure links. Example: `s3://bucket/data` `azure://.blob.core.windows.net/path/to/file`. Items remain in their original location. | No | +|`--dataset-folder` | Dataset base folder to add the files to in the dataset. Default: dataset root| No | +|`--non-recursive` | Disable recursive scan of files | No | +|`--verbose` | Verbose reporting | No|
@@ -90,12 +90,12 @@ clearml-data remove [-h] [--id ID] [--files [FILES [FILES ...]]]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id` | Dataset's ID. Default: previously created / accessed dataset| Yes | -|`--files` | Files / folders to remove (wildcard selection is supported, for example: `~/data/*.jpg ~/data/json`). Notice: file path is the path within the dataset, not the local path. For links, you can specify their URL (for example, `s3://bucket/data`) | No | -|`--non-recursive` | Disable recursive scan of files | Yes | -|`--verbose` | Verbose reporting | Yes| +|`--id` | Dataset's ID. Default: previously created / accessed dataset| No | +|`--files` | Files / folders to remove (wildcard selection is supported, for example: `~/data/*.jpg ~/data/json`). Notice: file path is the path within the dataset, not the local path. For links, you can specify their URL (for example, `s3://bucket/data`) | Yes | +|`--non-recursive` | Disable recursive scan of files | No | +|`--verbose` | Verbose reporting | No|
@@ -121,12 +121,12 @@ clearml-data upload [-h] [--id ID] [--storage STORAGE] [--chunk-size CHUNK_SIZE]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`| Dataset's ID. Default: previously created / accessed dataset| Yes | -|`--storage`| Remote storage to use for the dataset files. Default: files_server | Yes | -|`--chunk-size`| Set dataset artifact upload chunk size in MB. Default 512, (pass -1 for a single chunk). Example: 512, dataset will be split and uploaded in 512 MB chunks. | Yes| -|`--verbose` | Verbose reporting | Yes| +|`--id`| Dataset's ID. Default: previously created / accessed dataset| No | +|`--storage`| Remote storage to use for the dataset files. Default: files_server | No | +|`--chunk-size`| Set dataset artifact upload chunk size in MB. Default 512, (pass -1 for a single chunk). Example: 512, dataset will be split and uploaded in 512 MB chunks. | No| +|`--verbose` | Verbose reporting | No|
@@ -146,13 +146,13 @@ clearml-data close [-h] [--id ID] [--storage STORAGE] [--disable-upload]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`| Dataset's ID. Default: previously created / accessed dataset| Yes | -|`--storage`| Remote storage to use for the dataset files. Default: files_server | Yes | -|`--disable-upload` | Disable automatic upload when closing the dataset | Yes | -|`--chunk-size`| Set dataset artifact upload chunk size in MB. Default 512, (pass -1 for a single chunk). Example: 512, dataset will be split and uploaded in 512 MB chunks. | Yes| -|`--verbose` | Verbose reporting | Yes| +|`--id`| Dataset's ID. Default: previously created / accessed dataset| No | +|`--storage`| Remote storage to use for the dataset files. Default: files_server | No | +|`--disable-upload` | Disable automatic upload when closing the dataset | No | +|`--chunk-size`| Set dataset artifact upload chunk size in MB. Default 512, (pass -1 for a single chunk). Example: 512, dataset will be split and uploaded in 512 MB chunks. | No| +|`--verbose` | Verbose reporting | No|
@@ -179,20 +179,20 @@ clearml-data sync [-h] [--id ID] [--dataset-folder DATASET_FOLDER] --folder FOLD
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`| Dataset's ID. Default: previously created / accessed dataset| Yes | -|`--dataset-folder`|Dataset base folder to add the files to (default: Dataset root)|Yes| -|`--folder`|Local folder to sync. Wildcard selection is supported, for example: `~/data/*.jpg ~/data/json`|No| -|`--storage`|Remote storage to use for the dataset files. Default: files server |Yes| -|`--parents`|IDs of the dataset's parents (i.e. merge all parents). All modifications made to the folder since the parents were synced will be reflected in the dataset|Yes| -|`--project`|If creating a new dataset, specify the dataset's project name|Yes| -|`--name`|If creating a new dataset, specify the dataset's name|Yes| -|`--version`|Specify the dataset's version using the [semantic versioning](https://semver.org) scheme. Default: `1.0.0`|Yes| -|`--tags`|Dataset user tags|Yes| -|`--skip-close`|Do not auto close dataset after syncing folders|Yes| -|`--chunk-size`| Set dataset artifact upload chunk size in MB. Default 512, (pass -1 for a single chunk). Example: 512, dataset will be split and uploaded in 512 MB chunks. |Yes| -|`--verbose` | Verbose reporting |Yes| +|`--id`| Dataset's ID. Default: previously created / accessed dataset| No | +|`--dataset-folder`|Dataset base folder to add the files to (default: Dataset root)|No| +|`--folder`|Local folder to sync. Wildcard selection is supported, for example: `~/data/*.jpg ~/data/json`|Yes| +|`--storage`|Remote storage to use for the dataset files. Default: files server |No| +|`--parents`|IDs of the dataset's parents (i.e. merge all parents). All modifications made to the folder since the parents were synced will be reflected in the dataset|No| +|`--project`|If creating a new dataset, specify the dataset's project name|No| +|`--name`|If creating a new dataset, specify the dataset's name|No| +|`--version`|Specify the dataset's version using the [semantic versioning](https://semver.org) scheme. Default: `1.0.0`|No| +|`--tags`|Dataset user tags|No| +|`--skip-close`|Do not auto close dataset after syncing folders|No| +|`--chunk-size`| Set dataset artifact upload chunk size in MB. Default 512, (pass -1 for a single chunk). Example: 512, dataset will be split and uploaded in 512 MB chunks. |No| +|`--verbose` | Verbose reporting |No|
@@ -211,14 +211,14 @@ clearml-data list [-h] [--id ID] [--project PROJECT] [--name NAME] [--version VE
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`|Dataset ID whose contents will be shown (alternatively, use project / name combination). Default: previously accessed dataset|Yes| -|`--project`|Specify dataset project name (if used instead of ID, dataset name is also required)|Yes| -|`--name`|Specify dataset name (if used instead of ID, dataset project is also required)|Yes| -|`--version`|Specify dataset version. Default: most recent version |Yes| -|`--filter`|Filter files based on folder / wildcard. Multiple filters are supported. Example: `folder/date_*.json folder/subfolder`|Yes| -|`--modified`|Only list file changes (add / remove / modify) introduced in this version|Yes| +|`--id`|Dataset ID whose contents will be shown (alternatively, use project / name combination). Default: previously accessed dataset|No| +|`--project`|Specify dataset project name (if used instead of ID, dataset name is also required)|No| +|`--name`|Specify dataset name (if used instead of ID, dataset project is also required)|No| +|`--version`|Specify dataset version. Default: most recent version |No| +|`--filter`|Filter files based on folder / wildcard. Multiple filters are supported. Example: `folder/date_*.json folder/subfolder`|No| +|`--modified`|Only list file changes (add / remove / modify) introduced in this version|No|
@@ -236,10 +236,10 @@ clearml-data set-description [-h] [--id ID] [--description DESCRIPTION]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`|Dataset's ID|No| -|`--description`|Description to be set|No| +|`--id`|Dataset's ID|Yes| +|`--description`|Description to be set|Yes|
@@ -268,14 +268,14 @@ clearml-data delete [-h] [--id ID] [--project PROJECT] [--name NAME]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`|ID of the dataset to delete (alternatively, use project / name combination).|Yes| -|`--project`|Specify dataset project name (if used instead of ID, dataset name is also required)|Yes| -|`--name`|Specify dataset name (if used instead of ID, dataset project is also required)|Yes| -|`--version`|Specify dataset version|Yes| -|`-–force`|Force dataset deletion even if other dataset versions depend on it. Must also be used if `--entire-dataset` flag is used|Yes| -|`--entire-dataset`|Delete all found datasets|Yes| +|`--id`|ID of the dataset to delete (alternatively, use project / name combination).|No| +|`--project`|Specify dataset project name (if used instead of ID, dataset name is also required)|No| +|`--name`|Specify dataset name (if used instead of ID, dataset project is also required)|No| +|`--version`|Specify dataset version|No| +|`-–force`|Force dataset deletion even if other dataset versions depend on it. Must also be used if `--entire-dataset` flag is used|No| +|`--entire-dataset`|Delete all found datasets|No|
@@ -293,11 +293,11 @@ clearml-data rename [-h] --new-name NEW_NAME --project PROJECT --name NAME
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--new-name`|The new name of the dataset|No| -|`--project`|The project the dataset to be renamed belongs to|No| -|`--name`|The current name of the dataset(s) to be renamed|No| +|`--new-name`|The new name of the dataset|Yes| +|`--project`|The project the dataset to be renamed belongs to|Yes| +|`--name`|The current name of the dataset(s) to be renamed|Yes|
@@ -316,11 +316,11 @@ clearml-data move [-h] --new-project NEW_PROJECT --project PROJECT --name NAME
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--new-project`|The new project of the dataset|No| -|`--project`|The current project the dataset to be move belongs to|No| -|`--name`|The name of the dataset to be moved|No| +|`--new-project`|The new project of the dataset|Yes| +|`--project`|The current project the dataset to be move belongs to|Yes| +|`--name`|The name of the dataset to be moved|Yes|
@@ -341,12 +341,12 @@ clearml-data search [-h] [--ids [IDS [IDS ...]]] [--project PROJECT]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--ids`|A list of dataset IDs|Yes| -|`--project`|The project name of the datasets|Yes| -|`--name`|A dataset name or a partial name to filter datasets by|Yes| -|`--tags`|A list of dataset user tags|Yes| +|`--ids`|A list of dataset IDs|No| +|`--project`|The project name of the datasets|No| +|`--name`|A dataset name or a partial name to filter datasets by|No| +|`--tags`|A list of dataset user tags|No|
@@ -367,11 +367,11 @@ clearml-data compare [-h] --source SOURCE --target TARGET [--verbose]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--source`|Source dataset ID (used as baseline)|No| -|`--target`|Target dataset ID (compare against the source baseline dataset)|No| -|`--verbose`|Verbose report all file changes (instead of summary)|Yes| +|`--source`|Source dataset ID (used as baseline)|Yes| +|`--target`|Target dataset ID (compare against the source baseline dataset)|Yes| +|`--verbose`|Verbose report all file changes (instead of summary)|No|
@@ -387,12 +387,12 @@ clearml-data squash [-h] --name NAME --ids [IDS [IDS ...]] [--storage STORAGE] [
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--name`|Create squashed dataset name|No| -|`--ids`|Source dataset IDs to squash (merge down)|No| -|`--storage`|Remote storage to use for the dataset files. Default: files_server |Yes| -|`--verbose`|Verbose report all file changes (instead of summary)|Yes| +|`--name`|Create squashed dataset name|Yes| +|`--ids`|Source dataset IDs to squash (merge down)|Yes| +|`--storage`|Remote storage to use for the dataset files. Default: files_server |No| +|`--verbose`|Verbose report all file changes (instead of summary)|No|
@@ -408,12 +408,12 @@ clearml-data verify [-h] [--id ID] [--folder FOLDER] [--filesize] [--verbose]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`|Specify dataset ID. Default: previously created/accessed dataset|Yes| -|`--folder`|Specify dataset local copy (if not provided the local cache folder will be verified)|Yes| -|`--filesize`| If `True`, only verify file size and skip hash checks (default: `False`)|Yes| -|`--verbose`|Verbose report all file changes (instead of summary)|Yes| +|`--id`|Specify dataset ID. Default: previously created/accessed dataset|No| +|`--folder`|Specify dataset local copy (if not provided the local cache folder will be verified)|No| +|`--filesize`| If `True`, only verify file size and skip hash checks (default: `False`)|No| +|`--verbose`|Verbose report all file changes (instead of summary)|No|
@@ -431,15 +431,15 @@ clearml-data get [-h] [--id ID] [--copy COPY] [--link LINK] [--part PART]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`| Specify dataset ID. Default: previously created / accessed dataset|Yes| -|`--copy`| Get a writable copy of the dataset to a specific output folder|Yes| -|`--link`| Create a soft link (not supported on Windows) to a read-only cached folder containing the dataset|Yes| -|`--part`|Retrieve a partial copy of the dataset. Part number (0 to `--num-parts`-1) of total parts `--num-parts`.|Yes| -|`--num-parts`|Total number of parts to divide the dataset into. Notice, minimum retrieved part is a single chunk in a dataset (or its parents). Example: Dataset gen4, with 3 parents, each with a single chunk, can be divided into 4 parts |Yes| -|`--overwrite`| If `True`, overwrite the target folder|Yes| -|`--verbose`| Verbose report all file changes (instead of summary)| Yes| +|`--id`| Specify dataset ID. Default: previously created / accessed dataset|No| +|`--copy`| Get a writable copy of the dataset to a specific output folder|No| +|`--link`| Create a soft link (not supported on Windows) to a read-only cached folder containing the dataset|No| +|`--part`|Retrieve a partial copy of the dataset. Part number (0 to `--num-parts`-1) of total parts `--num-parts`.|No| +|`--num-parts`|Total number of parts to divide the dataset into. Notice, minimum retrieved part is a single chunk in a dataset (or its parents). Example: Dataset gen4, with 3 parents, each with a single chunk, can be divided into 4 parts |No| +|`--overwrite`| If `True`, overwrite the target folder|No| +|`--verbose`| Verbose report all file changes (instead of summary)| No|
@@ -455,8 +455,8 @@ clearml-data publish [-h] --id ID
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`| The dataset task ID to be published.| No| +|`--id`| The dataset task ID to be published.| Yes|
diff --git a/docs/clearml_serving/clearml_serving_cli.md b/docs/clearml_serving/clearml_serving_cli.md index 7abfe7db..be377690 100644 --- a/docs/clearml_serving/clearml_serving_cli.md +++ b/docs/clearml_serving/clearml_serving_cli.md @@ -19,11 +19,11 @@ clearml-serving [-h] [--debug] [--yes] [--id ID] {list,create,metrics,config,mod
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--id`|Serving Service (Control plane) Task ID to configure (if not provided, automatically detect the running control plane Task) | No | -|`--debug` | Print debug messages | Yes | -|`--yes` |Always answer YES on interactive inputs| Yes | +|`--id`|Serving Service (Control plane) Task ID to configure (if not provided, automatically detect the running control plane Task) | Yes | +|`--debug` | Print debug messages | No | +|`--yes` |Always answer YES on interactive inputs| No |
@@ -51,11 +51,11 @@ clearml-serving create [-h] [--name NAME] [--tags TAGS [TAGS ...]] [--project PR
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--name` |Serving service's name. Default: `Serving-Service`| No | -|`--project`|Serving service's project. Default: `DevOps`| No | -|`--tags` |Serving service's user tags. The serving service can be labeled, which can be useful for organizing | Yes| +|`--name` |Serving service's name. Default: `Serving-Service`| Yes | +|`--project`|Serving service's project. Default: `DevOps`| Yes | +|`--tags` |Serving service's user tags. The serving service can be labeled, which can be useful for organizing | No|
@@ -81,13 +81,13 @@ clearml-serving metrics add [-h] --endpoint ENDPOINT [--log-freq LOG_FREQ]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--endpoint`|Metric endpoint name including version (e.g. `"model/1"` or a prefix `"model/*"`). Notice: it will override any previous endpoint logged metrics| No| -|`--log-freq`|Logging request frequency, between 0.0 to 1.0. Example: 1.0 means all requests are logged, 0.5 means half of the requests are logged if not specified. To use global logging frequency, see [`config --metric-log-freq`](#config)| Yes| -|`--variable-scalar`|Add float (scalar) argument to the metric logger, `=`. Example: with specific buckets: `"x1=0,0.2,0.4,0.6,0.8,1"` or with min/max/num_buckets `"x1=0.0/1.0/5"`. Notice: In cases where 1000s of requests per second reach the serving, it makes no sense to display every datapoint. So scalars can be divided in buckets, and for each minute for example. Then it's possible to calculate what % of the total traffic fell in bucket 1, bucket 2, bucket 3 etc. The Y axis represents the buckets, color is the value in % of traffic in that bucket, and X is time. | Yes| -|`--variable-enum`|Add enum (string) argument to the metric logger, `=`. Example: `"detect=cat,dog,sheep"` |Yes| -|`--variable-value`|Add non-samples scalar argument to the metric logger, ``. Example: `"latency"` |Yes| +|`--endpoint`|Metric endpoint name including version (e.g. `"model/1"` or a prefix `"model/*"`). Notice: it will override any previous endpoint logged metrics| Yes| +|`--log-freq`|Logging request frequency, between 0.0 to 1.0. Example: 1.0 means all requests are logged, 0.5 means half of the requests are logged if not specified. To use global logging frequency, see [`config --metric-log-freq`](#config)| No| +|`--variable-scalar`|Add float (scalar) argument to the metric logger, `=`. Example: with specific buckets: `"x1=0,0.2,0.4,0.6,0.8,1"` or with min/max/num_buckets `"x1=0.0/1.0/5"`. Notice: In cases where 1000s of requests per second reach the serving, it makes no sense to display every datapoint. So scalars can be divided in buckets, and for each minute for example. Then it's possible to calculate what % of the total traffic fell in bucket 1, bucket 2, bucket 3 etc. The Y axis represents the buckets, color is the value in % of traffic in that bucket, and X is time. | No| +|`--variable-enum`|Add enum (string) argument to the metric logger, `=`. Example: `"detect=cat,dog,sheep"` |No| +|`--variable-value`|Add non-samples scalar argument to the metric logger, ``. Example: `"latency"` |No|
@@ -103,10 +103,10 @@ clearml-serving metrics remove [-h] [--endpoint ENDPOINT]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--endpoint`| Metric endpoint name including version (e.g. `"model/1"` or a prefix `"model/*"`) |No| -|`--variable`| Remove (scalar/enum) argument from the metric logger, `` example: `"x1"` |Yes| +|`--endpoint`| Metric endpoint name including version (e.g. `"model/1"` or a prefix `"model/*"`) |Yes| +|`--variable`| Remove (scalar/enum) argument from the metric logger, `` example: `"x1"` |No|
@@ -135,12 +135,12 @@ clearml-serving config [-h] [--base-serving-url BASE_SERVING_URL]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--base-serving-url`|External base serving service url. Example: `http://127.0.0.1:8080/serve`|Yes| -|`--triton-grpc-server`|External ClearML-Triton serving container gRPC address. Example: `127.0.0.1:9001`|Yes| -|`--kafka-metric-server`|External Kafka service url. Example: `127.0.0.1:9092`|Yes| -|`--metric-log-freq`|Set default metric logging frequency between 0.0 to 1.0. 1.0 means that 100% of all requests are logged|Yes| +|`--base-serving-url`|External base serving service url. Example: `http://127.0.0.1:8080/serve`|No| +|`--triton-grpc-server`|External ClearML-Triton serving container gRPC address. Example: `127.0.0.1:9001`|No| +|`--kafka-metric-server`|External Kafka service url. Example: `127.0.0.1:9092`|No| +|`--metric-log-freq`|Set default metric logging frequency between 0.0 to 1.0. 1.0 means that 100% of all requests are logged|No|
@@ -174,9 +174,9 @@ clearml-serving model remove [-h] [--endpoint ENDPOINT]
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--endpoint` | Model endpoint name | No| +|`--endpoint` | Model endpoint name | Yes|
@@ -193,16 +193,16 @@ clearml-serving model upload [-h] --name NAME [--tags TAGS [TAGS ...]] --project
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--name`|Specifying the model name to be registered in| No| -|`--tags`| Add tags to the newly created model| Yes| -|`--project`| Specify the project for the model to be registered in| No| -|`--framework`| Specify the model framework. Options are: 'tensorflow', 'tensorflowjs', 'tensorflowlite', 'pytorch', 'torchscript', 'caffe', 'caffe2', 'onnx', 'keras', 'mknet', 'cntk', 'torch', 'darknet', 'paddlepaddle', 'scikitlearn', 'xgboost', 'lightgbm', 'parquet', 'megengine', 'catboost', 'tensorrt', 'openvino', 'custom' | Yes| -|`--publish`| Publish the newly created model (change model state to "published" (i.e. locked and ready to deploy)|Yes| -|`--path`|Specify a model file/folder to be uploaded and registered| Yes| -|`--url`| Specify an already uploaded model url (e.g. `s3://bucket/model.bin`, `gs://bucket/model.bin`)|Yes| -|`--destination`|Specify the target destination for the model to be uploaded. For example: `s3://bucket/folder/`, `s3://host_addr:port/bucket` (for non-AWS S3-like services like MinIO), `gs://bucket-name/folder`, `azure://.blob.core.windows.net/path/to/file`|Yes| +|`--name`|Specifying the model name to be registered in| Yes| +|`--tags`| Add tags to the newly created model| No| +|`--project`| Specify the project for the model to be registered in| Yes| +|`--framework`| Specify the model framework. Options are: 'tensorflow', 'tensorflowjs', 'tensorflowlite', 'pytorch', 'torchscript', 'caffe', 'caffe2', 'onnx', 'keras', 'mknet', 'cntk', 'torch', 'darknet', 'paddlepaddle', 'scikitlearn', 'xgboost', 'lightgbm', 'parquet', 'megengine', 'catboost', 'tensorrt', 'openvino', 'custom' | No| +|`--publish`| Publish the newly created model (change model state to "published" (i.e. locked and ready to deploy)|No| +|`--path`|Specify a model file/folder to be uploaded and registered| No| +|`--url`| Specify an already uploaded model url (e.g. `s3://bucket/model.bin`, `gs://bucket/model.bin`)|No| +|`--destination`|Specify the target destination for the model to be uploaded. For example: `s3://bucket/folder/`, `s3://host_addr:port/bucket` (for non-AWS S3-like services like MinIO), `gs://bucket-name/folder`, `azure://.blob.core.windows.net/path/to/file`|No|
@@ -221,12 +221,12 @@ clearml-serving model canary [-h] [--endpoint ENDPOINT] [--weights WEIGHTS [WEIG
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--endpoint`| Model canary serving endpoint name (e.g. `my_model/latest`)| Yes| -|`--weights`| Model canary weights (order matching model ep), (e.g. 0.2 0.8) |Yes| -|`--input-endpoints`|Model endpoint prefixes, can also include version (e.g. `my_model`, `my_model/v1`)| Yes| -|`--input-endpoint-prefix`| Model endpoint prefix, lexicographic order or by version `` (e.g. `my_model/1`, `my_model/v1`), where the first weight matches the last version.|Yes| +|`--endpoint`| Model canary serving endpoint name (e.g. `my_model/latest`)| No| +|`--weights`| Model canary weights (order matching model ep), (e.g. 0.2 0.8) |No| +|`--input-endpoints`|Model endpoint prefixes, can also include version (e.g. `my_model`, `my_model/v1`)| No| +|`--input-endpoint-prefix`| Model endpoint prefix, lexicographic order or by version `` (e.g. `my_model/1`, `my_model/v1`), where the first weight matches the last version.|No|
@@ -250,23 +250,23 @@ clearml-serving model auto-update [-h] [--endpoint ENDPOINT] --engine ENGINE
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--endpoint`| Base model endpoint (must be unique)| No| -|`--engine`| Model endpoint serving engine (triton, sklearn, xgboost, lightgbm)| No| -|`--max-versions`|Max versions to store (and create endpoints) for the model. Highest number is the latest version | Yes| -|`--name`| Specify model name to be selected and auto-updated (notice regexp selection use `"$name^"` for exact match) | Yes| -|`--tags`|Specify tags to be selected and auto-updated |Yes| -|`--project`|Specify model project to be selected and auto-updated | Yes| -|`--published`| Only select published model for auto-update |Yes| -|`--preprocess` |Specify Pre/Post processing code to be used with the model (point to local file / folder) - this should hold for all the models |Yes| -|`--input-size`| Specify the model matrix input size [Rows x Columns X Channels etc ...] | Yes| -|`--input-type`| Specify the model matrix input type. Examples: uint8, float32, int16, float16 etc. |Yes| -|`--input-name`|Specify the model layer pushing input into. Example: layer_0 | Yes| -|`--output-size`|Specify the model matrix output size [Rows x Columns X Channels etc ...]|Yes| -|`--output_type`| Specify the model matrix output type. Examples: uint8, float32, int16, float16 etc. | Yes| -|`--output-name`|Specify the model layer pulling results from. Examples: layer_99| Yes| -|`--aux-config`| Specify additional engine specific auxiliary configuration in the form of key=value. Example: `platform=onnxruntime_onnx response_cache.enable=true max_batch_size=8`. Notice: you can also pass a full configuration file (e.g. Triton "config.pbtxt")|Yes| +|`--endpoint`| Base model endpoint (must be unique)| Yes| +|`--engine`| Model endpoint serving engine (triton, sklearn, xgboost, lightgbm)| Yes| +|`--max-versions`|Max versions to store (and create endpoints) for the model. Highest number is the latest version | No| +|`--name`| Specify model name to be selected and auto-updated (notice regexp selection use `"$name^"` for exact match) | No| +|`--tags`|Specify tags to be selected and auto-updated |No| +|`--project`|Specify model project to be selected and auto-updated | No| +|`--published`| Only select published model for auto-update |No| +|`--preprocess` |Specify Pre/Post processing code to be used with the model (point to local file / folder) - this should hold for all the models |No| +|`--input-size`| Specify the model matrix input size [Rows x Columns X Channels etc ...] | No| +|`--input-type`| Specify the model matrix input type. Examples: uint8, float32, int16, float16 etc. |No| +|`--input-name`|Specify the model layer pushing input into. Example: layer_0 | No| +|`--output-size`|Specify the model matrix output size [Rows x Columns X Channels etc ...]|No| +|`--output_type`| Specify the model matrix output type. Examples: uint8, float32, int16, float16 etc. | No| +|`--output-name`|Specify the model layer pulling results from. Examples: layer_99| No| +|`--aux-config`| Specify additional engine specific auxiliary configuration in the form of key=value. Example: `platform=onnxruntime_onnx response_cache.enable=true max_batch_size=8`. Notice: you can also pass a full configuration file (e.g. Triton "config.pbtxt")|No|
@@ -289,24 +289,24 @@ clearml-serving model add [-h] --engine ENGINE --endpoint ENDPOINT [--version VE
-|Name|Description|Optional| +|Name|Description|Mandatory| |---|---|---| -|`--engine`| Model endpoint serving engine (triton, sklearn, xgboost, lightgbm)| No| -|`--endpoint`| Base model endpoint (must be unique)| No| -|`--version`|Model endpoint version (default: None) | Yes| -|`--model-id`|Specify a model ID to be served|No| -|`--preprocess` |Specify Pre/Post processing code to be used with the model (point to local file / folder) - this should hold for all the models |Yes| -|`--input-size`| Specify the model matrix input size [Rows x Columns X Channels etc ...] | Yes| -|`--input-type`| Specify the model matrix input type. Examples: uint8, float32, int16, float16 etc. |Yes| -|`--input-name`|Specify the model layer pushing input into. Example: layer_0 | Yes| -|`--output-size`|Specify the model matrix output size [Rows x Columns X Channels etc ...]|Yes| -|`--output_type`| Specify the model matrix output type. Examples: uint8, float32, int16, float16 etc. | Yes| -|`--output-name`|Specify the model layer pulling results from. Examples: layer_99| Yes| -|`--aux-config`| Specify additional engine specific auxiliary configuration in the form of key=value. Example: `platform=onnxruntime_onnx response_cache.enable=true max_batch_size=8`. Notice: you can also pass a full configuration file (e.g. Triton "config.pbtxt")|Yes| -|`--name`| Instead of specifying `--model-id` select based on model name | Yes| -|`--tags`|Specify tags to be selected and auto-updated |Yes| -|`--project`|Instead of specifying `--model-id` select based on model project | Yes| -|`--published`| Instead of specifying `--model-id` select based on model published |Yes| +|`--engine`| Model endpoint serving engine (triton, sklearn, xgboost, lightgbm)| Yes| +|`--endpoint`| Base model endpoint (must be unique)| Yes| +|`--version`|Model endpoint version (default: None) | No| +|`--model-id`|Specify a model ID to be served|Yes| +|`--preprocess` |Specify Pre/Post processing code to be used with the model (point to local file / folder) - this should hold for all the models |No| +|`--input-size`| Specify the model matrix input size [Rows x Columns X Channels etc ...] | No| +|`--input-type`| Specify the model matrix input type. Examples: uint8, float32, int16, float16 etc. |No| +|`--input-name`|Specify the model layer pushing input into. Example: layer_0 | No| +|`--output-size`|Specify the model matrix output size [Rows x Columns X Channels etc ...]|No| +|`--output_type`| Specify the model matrix output type. Examples: uint8, float32, int16, float16 etc. | No| +|`--output-name`|Specify the model layer pulling results from. Examples: layer_99| No| +|`--aux-config`| Specify additional engine specific auxiliary configuration in the form of key=value. Example: `platform=onnxruntime_onnx response_cache.enable=true max_batch_size=8`. Notice: you can also pass a full configuration file (e.g. Triton "config.pbtxt")|No| +|`--name`| Instead of specifying `--model-id` select based on model name | No| +|`--tags`|Specify tags to be selected and auto-updated |No| +|`--project`|Instead of specifying `--model-id` select based on model project | No| +|`--published`| Instead of specifying `--model-id` select based on model published |No|