This commit is contained in:
revital
2025-03-11 18:03:33 +02:00
141 changed files with 170 additions and 104 deletions

View File

@@ -77,7 +77,7 @@ See more information about explicitly logging information to a ClearML Task:
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -93,7 +93,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -31,7 +31,8 @@ You can view all the task details in the [WebApp](../webapp/webapp_overview.md).
See an example of CatBoost and ClearML in action [here](../guides/frameworks/catboost/catboost.md).
![Task scalars](../img/examples_catboost_scalars.png)
![Task scalars](../img/examples_catboost_scalars.png#light-mode-only)
![Task scalars](../img/examples_catboost_scalars_dark.png#dark-mode-only)
## Automatic Logging Control
By default, when ClearML is integrated into your CatBoost script, it captures models, and
@@ -75,7 +76,7 @@ See more information about explicitly logging information to a ClearML Task:
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -91,7 +92,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -42,7 +42,8 @@ if __name__ == '__main__':
When this code is executed, ClearML logs your command-line arguments, which you can view in the
[WebApp](../webapp/webapp_overview.md), in the task's **Configuration > Hyperparameters > Args** section.
![click configuration](../img/integrations_click_configs.png)
![click configuration](../img/integrations_click_configs.png#light-mode-only)
![click configuration](../img/integrations_click_configs_dark.png#dark-mode-only)
In the UI, you can clone the task multiple times and set the clones' parameter values for execution by the [ClearML Agent](../clearml_agent.md).
When the clone is executed, the executing agent will use the new parameter values as if set by the command-line.

View File

@@ -30,7 +30,8 @@ You can view all the task details in the [WebApp](../webapp/webapp_overview.md).
See an example of `fastai` and ClearML in action [here](../guides/frameworks/fastai/fastai_with_tensorboard.md).
![Task scalars](../img/examples_reporting_fastai_01.png)
![Task scalars](../img/examples_reporting_fastai_01.png#light-mode-only)
![Task scalars](../img/examples_reporting_fastai_01_dark.png#dark-mode-only)
## Automatic Logging Control
By default, when ClearML is integrated into your `fastai` script, it captures models and
@@ -74,7 +75,7 @@ See more information about explicitly logging information to a ClearML Task:
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -90,7 +91,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -22,7 +22,8 @@ task = Task.init(task_name="<task_name>", project_name="<project_name>")
ClearML logs the OmegaConf as a blob and can be viewed in the
[WebApp](../webapp/webapp_overview.md), in the task's **CONFIGURATION > CONFIGURATION OBJECTS > OmegaConf** section.
![Hydra configuration](../img/integrations_hydra_configs.png)
![Hydra configuration](../img/integrations_hydra_configs.png#light-mode-only)
![Hydra configuration](../img/integrations_hydra_configs_dark.png#dark-mode-only)
## Modifying Hydra Values

View File

@@ -87,7 +87,7 @@ and debug samples, plots, and scalars logged to TensorBoard
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -103,7 +103,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -53,16 +53,19 @@ You can view all the task details in the [WebApp](../webapp/webapp_exp_track_vis
ClearML logs the scalars from training each network. They appear in the task's **SCALARS** tab in the Web UI.
![Optimization scalars](../img/integration_keras_tuner_06.png)
![Optimization scalars](../img/integration_keras_tuner_06.png#light-mode-only)
![Optimization scalars](../img/integration_keras_tuner_06_dark.png#dark-mode-only)
ClearML automatically logs the parameters of each task run in the hyperparameter search. They appear in tabular
form in the task's **PLOTS**.
![Optimization plot](../img/integration_keras_tuner_07.png)
![Optimization plot](../img/integration_keras_tuner_07.png#light-mode-only)
![Optimization plot](../img/integration_keras_tuner_07_dark.png#dark-mode-only)
ClearML automatically stores the output model. It appears in the task's **ARTIFACTS** **>** **Output Model**.
![output model](../img/integration_keras_tuner_03.png)
![output model](../img/integration_keras_tuner_03.png#light-mode-only)
![output model](../img/integration_keras_tuner_03_dark.png#dark-mode-only)
## Example

View File

@@ -76,7 +76,7 @@ See more information about explicitly logging information to a ClearML Task:
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -92,7 +92,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -72,7 +72,7 @@ See more information about explicitly logging information to a ClearML Task:
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -88,7 +88,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -28,7 +28,8 @@ And that's it! This creates a [ClearML Task](../fundamentals/task.md) which capt
You can view all the task details in the [WebApp](../webapp/webapp_overview.md).
![Pytorch webapp](../img/examples_pytorch_distributed_example_08.png)
![Pytorch webapp](../img/examples_pytorch_distributed_example_08.png#light-mode-only)
![Pytorch webapp](../img/examples_pytorch_distributed_example_08_dark.png#dark-mode-only)
## Automatic Logging Control
By default, when ClearML is integrated into your PyTorch script, it captures PyTorch models. But, you may want to have
@@ -96,7 +97,7 @@ additional tools, like argparse, TensorBoard, and matplotlib:
* [PyTorch Distributed](../guides/frameworks/pytorch/pytorch_distributed_example.md) - Demonstrates using ClearML with the [PyTorch Distributed Communications Package (`torch.distributed`)](https://pytorch.org/tutorials/beginner/dist_overview.html)
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -112,7 +113,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -102,7 +102,7 @@ See more information about explicitly logging information to a ClearML Task:
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -118,7 +118,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md), to
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -78,7 +78,7 @@ additional tools, like Matplotlib:
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -94,7 +94,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -94,7 +94,7 @@ You can view all of this captured information in the [ClearML Web UI](../webapp/
![TAO UI plots](../img/integrations_nvidia_tao_plots.png)
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is
enqueued, the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -111,7 +111,7 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents:
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -22,9 +22,11 @@ uncommitted code, Python environment, your TensorBoard metrics, plots, images, a
View the TensorBoard outputs in the [WebApp](../webapp/webapp_overview.md), in the task's page.
![TensorBoard WebApp scalars](../img/examples_pytorch_tensorboard_07.png)
![TensorBoard WebApp scalars](../img/examples_pytorch_tensorboard_07.png#light-mode-only)
![TensorBoard WebApp scalars](../img/examples_pytorch_tensorboard_07_dark.png#dark-mode-only)
![Tensorboard WebApp debug samples](../img/examples_tensorboard_toy_pytorch_02.png)
![Tensorboard WebApp debug samples](../img/examples_tensorboard_toy_pytorch_02.png#light-mode-only)
![Tensorboard WebApp debug samples](../img/examples_tensorboard_toy_pytorch_02_dark.png#dark-mode-only)
## Automatic Logging Control
By default, when ClearML is integrated into your script, it captures all of your TensorBoard plots, images, and metrics.

View File

@@ -22,7 +22,8 @@ uncommitted code, Python environment, your TensorboardX metrics, plots, images,
View the TensorboardX outputs in the [WebApp](../webapp/webapp_overview.md), in the task's page.
![TensorboardX WebApp scalars](../img/examples_pytorch_tensorboardx_03.png)
![TensorboardX WebApp scalars](../img/examples_pytorch_tensorboardx_03.png#light-mode-only)
![TensorboardX WebApp scalars](../img/examples_pytorch_tensorboardx_03_dark.png#dark-mode-only)
## Automatic Logging Control
By default, when ClearML is integrated into your script, it captures all of your TensorboardX plots, images, metrics, videos, and text.

View File

@@ -89,7 +89,7 @@ TensorBoard scalars, histograms, images, and text, as well as all console output
ClearML's automatic logging of parameters defined using `absl.flags`
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -105,7 +105,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -60,7 +60,7 @@ You can also select multiple tasks and directly [compare](../webapp/webapp_exp_c
See an example of Transformers and ClearML in action [here](../guides/frameworks/huggingface/transformers.md).
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is
enqueued, the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.

View File

@@ -51,7 +51,8 @@ except ImportError:
You can view all the task details in the [WebApp](../webapp/webapp_overview.md).
![Task scalars](../img/examples_xgboost_metric_scalars.png)
![Task scalars](../img/examples_xgboost_metric_scalars.png#light-mode-only)
![Task scalars](../img/examples_xgboost_metric_scalars_dark.png#dark-mode-only)
## Automatic Logging Control
By default, when ClearML is integrated into your XGBoost script, it captures models, and
@@ -102,7 +103,7 @@ additional tools, like Matplotlib and scikit-learn:
* [XGBoost and scikit-learn](../guides/frameworks/xgboost/xgboost_sample.md) - Demonstrates ClearML automatic logging of XGBoost scalars and models
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is enqueued,
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -118,7 +119,7 @@ Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md) to h
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -150,7 +150,7 @@ python train.py --img 640 --batch 16 --epochs 3 --data clearml://<your_dataset_i
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is
enqueued, the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -167,7 +167,7 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents:
and shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif#light-mode-only)
![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5_dark.gif#dark-mode-only)

View File

@@ -95,7 +95,7 @@ Add custom columns to the table, such as mAP values, so you can easily sort and
You can also select multiple tasks and directly [compare](../webapp/webapp_exp_comparing.md) them.
## Remote Execution
ClearML logs all the information required to reproduce a task on a different machine (installed packages,
ClearML logs all the information required to reproduce a task run on a different machine (installed packages,
uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is
enqueued, the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
task manager.
@@ -112,9 +112,9 @@ cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents:
shuts down instances as needed, according to a resource budget that you set.
### Reproducing Tasks
### Reproducing Task Runs
ClearML logs all the information required to reproduce a task, but you may also want to change a few parameters
ClearML logs all the information required to reproduce a task run, but you may also want to change a few parameters
and task details when you re-run it, which you can do through ClearML's UI.
In order to be able to override parameters via the UI,