mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-26 18:17:44 +00:00
Merge branch 'main' of https://github.com/allegroai/clearml-docs
This commit is contained in:
revital
commit
087825f5bb
@ -1,6 +1,5 @@
|
||||
---
|
||||
title: PyTorch Ignite TensorboardLogger
|
||||
displayed_sidebar: mainSidebar
|
||||
---
|
||||
|
||||
The [cifar_ignite.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py) example
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
---
|
||||
title: PyTorch Ignite ClearMLLogger
|
||||
displayed_sidebar: mainSidebar
|
||||
---
|
||||
|
||||
The `ignite` repository contains the [mnist_with_clearml_logger.py](https://github.com/pytorch/ignite/blob/master/examples/contrib/mnist/mnist_with_clearml_logger.py)
|
||||
|
||||
@ -1,6 +1,5 @@
|
||||
---
|
||||
title: TensorFlow
|
||||
displayed_sidebar: mainSidebar
|
||||
title: TensorFlow MNIST
|
||||
---
|
||||
|
||||
The [tensorflow_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py)
|
||||
|
||||
BIN
docs/img/gif/tensorflow.gif
Normal file
BIN
docs/img/gif/tensorflow.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 380 KiB |
146
docs/integrations/ignite.md
Normal file
146
docs/integrations/ignite.md
Normal file
@ -0,0 +1,146 @@
|
||||
---
|
||||
title: PyTorch Ignite
|
||||
---
|
||||
|
||||
[PyTorch Ignite](https://pytorch.org/ignite/index.html) is a library for training and evaluating neural networks in
|
||||
PyTorch. You can integrate ClearML into your code using Ignite’s built-in loggers: [TensorboardLogger](#tensorboardlogger)
|
||||
and [ClearMLLogger](#clearmllogger).
|
||||
|
||||
## TensorboardLogger
|
||||
|
||||
ClearML integrates seamlessly with TensorboardLogger, and automatically captures all information logged through the
|
||||
handler: metrics, parameters, images, and gradients.
|
||||
|
||||
All you have to do is add two lines of code to your script:
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
task = Task.init(task_name="<task_name>", project_name="<project_name>")
|
||||
```
|
||||
|
||||
This will create a [ClearML Task](../fundamentals/task.md) that captures your script's information, including Git details,
|
||||
uncommitted code, python environment, all information logged through `TensorboardLogger`, and more.
|
||||
|
||||
Visualize all the captured information in the experiment's page in ClearML's [WebApp](#webapp)
|
||||
|
||||
See a code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py).
|
||||
|
||||
## ClearMLLogger
|
||||
PyTorch Ignite supports a ClearML Logger to log metrics, text, model/optimizer parameters, plots, and model checkpoints
|
||||
during training and validation.
|
||||
|
||||
Integrate ClearML with the following steps:
|
||||
1. Create a `ClearMLLogger` object:
|
||||
|
||||
```python
|
||||
from ignite.contrib.handlers.clearml_logger import *
|
||||
|
||||
clearml_logger = ClearMLLogger(task_name="ignite", project_name="examples")
|
||||
```
|
||||
|
||||
This creates a [ClearML Task](../fundamentals/task.md) called `ignite` in the `examples` project, which captures your
|
||||
script's information, including Git details, uncommitted code, python environment.
|
||||
|
||||
You can also pass the following parameters to the `ClearMLLogger` object:
|
||||
* `task_type` – The type of experiment (see [task types](../fundamentals/task.md#task-types)).
|
||||
* `report_freq` – The histogram processing frequency (handles histogram values every X calls to the handler). Affects
|
||||
`GradsHistHandler` and `WeightsHistHandler` (default: 100).
|
||||
* `histogram_update_freq_multiplier` – The histogram report frequency (report first X histograms and once every X
|
||||
reports afterwards) (default: 10).
|
||||
* `histogram_granularity` - Histogram sampling granularity (default: 50).
|
||||
|
||||
1. Attach the C`learMLLogger` to output handlers to log metrics:
|
||||
|
||||
```python
|
||||
# Attach the logger to the trainer to log training loss
|
||||
clearml_logger.attach_output_handler(
|
||||
trainer,
|
||||
event_name=Events.ITERATION_COMPLETED(every=100),
|
||||
tag="training",
|
||||
output_transform=lambda loss: {"batchloss": loss},
|
||||
)
|
||||
|
||||
# Attach the logger to log loss and accuracy for both training and validation
|
||||
for tag, evaluator in [("training metrics", train_evaluator), ("validation metrics", validation_evaluator)]:
|
||||
clearml_logger.attach_output_handler(
|
||||
evaluator,
|
||||
event_name=Events.EPOCH_COMPLETED,
|
||||
tag=tag,
|
||||
metric_names=["loss", "accuracy"],
|
||||
global_step_transform=global_step_from_engine(trainer),
|
||||
)
|
||||
```
|
||||
|
||||
1. Attach the ClearMLLogger object to helper handlers to log experiment outputs. Ignite supports the following helper handlers for ClearML:
|
||||
|
||||
* **ClearMLSaver** - Saves input snapshots as ClearML artifacts.
|
||||
* **GradsHistHandler** and **WeightsHistHandler** - Logs the model's gradients and weights respectively as histograms.
|
||||
* **GradsScalarHandler** and **WeightsScalarHandler** - Logs gradients and weights respectively as scalars.
|
||||
* **OptimizerParamsHandler** - Logs optimizer parameters
|
||||
|
||||
```python
|
||||
# Attach the logger to the trainer to log model's weights norm
|
||||
clearml_logger.attach(
|
||||
trainer, log_handler=WeightsScalarHandler(model), event_name=Events.ITERATION_COMPLETED(every=100)
|
||||
)
|
||||
|
||||
# Attach the logger to the trainer to log model's weights as a histogram
|
||||
clearml_logger.attach(trainer, log_handler=WeightsHistHandler(model), event_name=Events.EPOCH_COMPLETED(every=100))
|
||||
|
||||
# Attach the logger to the trainer to log model’s gradients as scalars
|
||||
clearml_logger.attach(
|
||||
trainer, log_handler=GradsScalarHandler(model), event_name=Events.ITERATION_COMPLETED(every=100)
|
||||
)
|
||||
|
||||
#Attach the logger to the trainer to log model's gradients as a histogram
|
||||
clearml_logger.attach(trainer, log_handler=GradsHistHandler(model), event_name=Events.EPOCH_COMPLETED(every=100))
|
||||
|
||||
handler = Checkpoint(
|
||||
{"model": model},
|
||||
ClearMLSaver(),
|
||||
n_saved=1,
|
||||
score_function=lambda e: e.state.metrics["accuracy"],
|
||||
score_name="val_acc",
|
||||
filename_prefix="best",
|
||||
global_step_transform=global_step_from_engine(trainer),
|
||||
)
|
||||
validation_evaluator.add_event_handler(Events.EPOCH_COMPLETED, handler)
|
||||
|
||||
# Attach the logger to the trainer to log optimizer's parameters, e.g. learning rate at each iteration
|
||||
clearml_logger.attach(
|
||||
trainer,
|
||||
log_handler=OptimizerParamsHandler(optimizer),
|
||||
event_name=Events.ITERATION_STARTED
|
||||
)
|
||||
```
|
||||
|
||||
Visualize all the captured information in the experiment's page in ClearML's [WebApp](#webapp).
|
||||
|
||||
For more information, see the [ignite documentation](https://pytorch.org/ignite/generated/ignite.contrib.handlers.clearml_logger.html).
|
||||
|
||||
See code example [here](https://github.com/pytorch/ignite/blob/master/examples/contrib/mnist/mnist_with_clearml_logger.py)
|
||||
|
||||
## WebApp
|
||||
|
||||
All the experiment information that ClearML captures can be viewed in the [WebApp](../webapp/webapp_overview.md):
|
||||
|
||||
### Models
|
||||
|
||||
View saved model snapshots in the **ARTIFACTS** tab.
|
||||
|
||||
|
||||
|
||||
### Scalars
|
||||
|
||||
View the scalars in the experiment's **SCALARS** tab.
|
||||
|
||||
|
||||
|
||||
|
||||
### Debug Samples
|
||||
|
||||
ClearML automatically tracks images logged to `TensorboardLogger`. They appear in the experiment's **DEBUG SAMPLES**.
|
||||
|
||||
|
||||
|
||||
|
||||
134
docs/integrations/tensorflow.md
Normal file
134
docs/integrations/tensorflow.md
Normal file
@ -0,0 +1,134 @@
|
||||
---
|
||||
title: TensorFlow
|
||||
---
|
||||
|
||||
:::tip
|
||||
If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup
|
||||
instructions.
|
||||
:::
|
||||
|
||||
ClearML integrates with [TensorFlow](https://www.tensorflow.org/) out-of-the-box, automatically logging its models,
|
||||
definitions, scalars, as well as TensorBoard outputs.
|
||||
|
||||
All you have to do is simply add two lines of code to your TensorFlow script:
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
task = Task.init(task_name="<task_name>", project_name="<project_name>")
|
||||
```
|
||||
|
||||
And that’s it! This creates a [ClearML Task](../fundamentals/task.md) which captures:
|
||||
* Source code and uncommitted changes
|
||||
* Installed packages
|
||||
* TensorFlow definitions
|
||||
* TensorFlow model files
|
||||
* [TensorBoard](https://www.tensorflow.org/tensorboard) outputs (see example [here](https://clear.ml/docs/latest/docs/guides/frameworks/tensorflow/tensorboard_toy/))
|
||||
* Scalars (loss, learning rates)
|
||||
* Console output
|
||||
* General details such as machine details, runtime, creation date etc.
|
||||
* And more
|
||||
|
||||
You can view all the task details in the [WebApp](../webapp/webapp_overview.md).
|
||||
|
||||
|
||||
|
||||
## Automatic Logging Control
|
||||
By default, when ClearML is integrated into your TensorFlow script, it captures TensorFlow definitions, models, and
|
||||
scalars. But, you may want to have more control over what your experiment logs.
|
||||
|
||||
To control a task's framework logging, use the `auto_connect_frameworks` parameter of [`Task.init()`](../references/sdk/task.md#taskinit).
|
||||
Completely disable all automatic logging by setting the parameter to `False`. For finer grained control of logged
|
||||
frameworks, input a dictionary, with framework-boolean pairs.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
auto_connect_frameworks={
|
||||
'matplotlib': True, 'tensorflow': False, 'tensorboard': False, 'pytorch': True,
|
||||
'xgboost': False, 'scikit': True, 'fastai': True, 'lightgbm': False,
|
||||
'hydra': True, 'detect_repository': True, 'tfdefines': True, 'joblib': True,
|
||||
'megengine': True, 'jsonargparse': True, 'catboost': True
|
||||
}
|
||||
```
|
||||
|
||||
You can also input wildcards as dictionary values, so ClearML will log a model created by a framework only if its local
|
||||
path matches at least one wildcard.
|
||||
|
||||
For example, in the code below, ClearML will log TensorFlow models only if their paths have the `.pt` extension. The
|
||||
unspecified frameworks' values default to true so all their models are automatically logged.
|
||||
|
||||
```python
|
||||
auto_connect_frameworks={'tensorflow' : '*.pt'}
|
||||
```
|
||||
|
||||
## Manual Logging
|
||||
To augment its automatic logging, ClearML also provides an explicit logging interface.
|
||||
|
||||
See more information about explicitly logging information to a ClearML Task:
|
||||
* [Models](../clearml_sdk/model_sdk.md#manually-logging-models)
|
||||
* [Configuration](../clearml_sdk/task_sdk.md#configuration) (e.g. parameters, configuration files)
|
||||
* [Artifacts](../clearml_sdk/task_sdk.md#artifacts) (e.g. output files or python objects created by a task)
|
||||
* [Scalars](../clearml_sdk/task_sdk.md#scalars)
|
||||
* [Text/Plots/Debug Samples](../fundamentals/logger.md#manual-reporting)
|
||||
|
||||
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
|
||||
|
||||
## Examples
|
||||
|
||||
Take a look at ClearML’s TensorFlow examples. The examples use TensorFlow and ClearML in different configurations with
|
||||
additional tools, like Abseil and TensorBoard:
|
||||
|
||||
* [TensorFlow MNIST](../guides/frameworks/tensorflow/tensorflow_mnist.md) - Demonstrates ClearML's automatic logging of
|
||||
model checkpoints, TensorFlow definitions, and scalars logged using TensorFlow methods
|
||||
* [TensorBoard PR Curve](../guides/frameworks/tensorflow/tensorboard_pr_curve.md) - Demonstrates ClearML’s automatic
|
||||
logging of TensorBoard output and TensorFlow definitions.
|
||||
* [TensorBoard Toy](../guides/frameworks/tensorflow/tensorboard_toy.md) - Demonstrates ClearML’s automatic logging of
|
||||
TensorBoard scalars, histograms, images, and text, as well as all console output and TensorFlow Definitions.
|
||||
* [Absl flags](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/absl_flags.py) - Demonstrates
|
||||
ClearML’s automatic logging of parameters defined using `absl.flags`
|
||||
|
||||
## Remote Execution
|
||||
ClearML logs all the information required to reproduce an experiment on a different machine (installed packages,
|
||||
uncommitted changes etc.). The [ClearML Agent](../clearml_agent) listens to designated queues and when a task is enqueued,
|
||||
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
|
||||
experiment manager.
|
||||
|
||||
Deploy a ClearML Agent onto any machine (e.g. a cloud VM, a local GPU machine, your own laptop) by simply running the
|
||||
following command on it:
|
||||
|
||||
```commandline
|
||||
clearml-agent daemon --queue <queues_to_listen_to> [--docker]
|
||||
```
|
||||
|
||||
Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md), to help you manage cloud workloads in the
|
||||
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
|
||||
and shuts down instances as needed, according to a resource budget that you set.
|
||||
|
||||
### Cloning, Editing, and Enqueuing
|
||||
|
||||
|
||||
|
||||
Use ClearML's web interface to edit task details, like configuration parameters or input models, then execute the task
|
||||
with the new configuration on a remote machine:
|
||||
|
||||
* Clone the experiment
|
||||
* Edit the hyperparameters and/or other details
|
||||
* Enqueue the task
|
||||
|
||||
The ClearML Agent executing the task will use the new values to [override any hard coded values](../clearml_agent).
|
||||
|
||||
### Executing a Task Remotely
|
||||
|
||||
You can set a task to be executed remotely programmatically by adding [`Task.execute_remotely()`](../references/sdk/task.md#execute_remotely)
|
||||
to your script. This method stops the current local execution of the task, and then enqueues it to a specified queue to
|
||||
re-run it on a remote machine.
|
||||
|
||||
```python
|
||||
# If executed locally, process will terminate, and a copy will be executed by an agent instead
|
||||
task.execute_remotely(queue_name='default', exit_process=True)
|
||||
```
|
||||
|
||||
## Hyperparameter Optimization
|
||||
Use ClearML’s [`HyperParameterOptimizer`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md) class to find
|
||||
the hyperparameter values that yield the best performing models. See [Hyperparameter Optimization](../fundamentals/hpo.md)
|
||||
for more information.
|
||||
@ -66,9 +66,9 @@ module.exports = {
|
||||
'guides/frameworks/lightgbm/lightgbm_example', 'guides/frameworks/matplotlib/matplotlib_example',
|
||||
'guides/frameworks/megengine/megengine_mnist', 'integrations/openmmv', 'integrations/optuna',
|
||||
'integrations/python_fire', 'guides/frameworks/pytorch/pytorch_mnist',
|
||||
{'PyTorch Ignite':['guides/frameworks/pytorch_ignite/integration_pytorch_ignite', 'guides/frameworks/pytorch_ignite/pytorch_ignite_mnist']},
|
||||
'integrations/ignite',
|
||||
'guides/frameworks/pytorch_lightning/pytorch_lightning_example', 'guides/frameworks/scikit-learn/sklearn_joblib_example',
|
||||
'guides/frameworks/pytorch/pytorch_tensorboard', 'guides/frameworks/tensorboardx/tensorboardx', 'guides/frameworks/tensorflow/tensorflow_mnist',
|
||||
'guides/frameworks/pytorch/pytorch_tensorboard', 'guides/frameworks/tensorboardx/tensorboardx', 'integrations/tensorflow',
|
||||
'integrations/seaborn', 'integrations/xgboost', 'integrations/yolov5', 'integrations/yolov8'
|
||||
]
|
||||
},
|
||||
|
||||
Loading…
Reference in New Issue
Block a user