clearml-docs/docs/fundamentals/logger.md

124 lines
6.0 KiB
Markdown
Raw Normal View History

2021-05-13 23:48:51 +00:00
---
title: Logger
---
The ClearML Logger class is used to report experiments' results such as metrics, graphs, and debug samples. It is provided
through the ClearML [Task](task.md) object.
A Logger object is used to do the following:
* [Manual reporting](#manual-reporting), complementing ClearML's [automatic reporting](#automatic-reporting)
* [Logging configuration](#logger-configuration)
* Set upload destination for debug sample storage
* Control ClearML's automatic logging
* Set default NaN and Inf values
2021-05-13 23:48:51 +00:00
## Types of Logged Results
2022-01-18 11:23:47 +00:00
ClearML supports four types of reports:
2021-05-13 23:48:51 +00:00
- Text - Mostly captured automatically from stdout and stderr but can be logged manually.
- Scalars - Time series data. X-axis is always a sequential number, usually iterations but can be epochs or others.
2022-09-04 07:17:44 +00:00
- Plots - General graphs and diagrams, such as histograms, confusion matrices, line plots, and custom plotly charts.
2021-05-13 23:48:51 +00:00
- Debug Samples - Images, audio, and videos. Can be reported per iteration.
![image](../img/fundamentals_logger_results.png)
## Automatic Reporting
ClearML automatically captures metrics reported to leading visualization libraries, such as TensorBoard and Matplotlib,
with no additional code necessary.
2021-05-13 23:48:51 +00:00
2023-10-02 15:32:39 +00:00
In addition, ClearML captures and logs everything written to standard output, from debug messages to errors to
2021-05-13 23:48:51 +00:00
library warning messages.
2023-10-02 15:32:39 +00:00
GPU, CPU, Memory, and Network information is also automatically captured.
2021-05-13 23:48:51 +00:00
![image](../img/fundamentals_logger_cpu_monitoring.png)
2021-09-01 06:41:27 +00:00
### Supported Packages
- [TensorBoard](https://www.tensorflow.org/tensorboard)
- [TensorBoardX](https://github.com/lanpa/tensorboardX)
- [Matplotlib](https://matplotlib.org/)
### Automatic Reporting Examples
Check out some of ClearML's automatic reporting examples for supported packages:
* TensorBoard
* [TensorBoard PR Curve](../guides/frameworks/tensorflow/tensorboard_pr_curve.md) - logging TensorBoard outputs and
TensorFlow flags
* [TensorBoard Toy](../guides/frameworks/tensorflow/tensorboard_toy.md) - logging TensorBoard histograms, scalars, images, text, and
TensorFlow flags
* [Tensorboard with PyTorch](../guides/frameworks/pytorch/pytorch_tensorboard.md) - logging TensorBoard scalars, debug samples, and text integrated into
code that uses PyTorch
2021-12-19 10:34:06 +00:00
* TensorBoardX
2023-01-23 13:04:24 +00:00
* [TensorBoardX with PyTorch](../guides/frameworks/tensorboardx/tensorboardx.md) - logging TensorBoardX scalars, debug
2021-09-01 06:41:27 +00:00
samples, and text in code using PyTorch
2021-12-19 10:34:06 +00:00
* [MegEngine MNIST](../guides/frameworks/megengine/megengine_mnist.md) - logging scalars using TensorBoardX's `SummaryWriter`
2021-09-01 06:41:27 +00:00
* Matplotlib
* [Matplotlib](../guides/frameworks/matplotlib/matplotlib_example.md) - logging scatter diagrams plotted with Matplotlib
2021-09-01 06:41:27 +00:00
* [Matplotlib with PyTorch](../guides/frameworks/pytorch/pytorch_matplotlib.md) - logging debug images shown
by Matplotlib
2021-05-13 23:48:51 +00:00
## Manual Reporting
ClearML also supports manually reporting multiple types of metrics and plots, such as line plots, histograms, and even plotly
charts.
2024-02-11 14:57:55 +00:00
The object used for reporting metrics is called **logger** and is obtained by calling [`Task.get_logger()`](../references/sdk/task.md#get_logger).
2021-05-13 23:48:51 +00:00
2021-09-01 06:41:27 +00:00
### Media Reporting
2021-05-13 23:48:51 +00:00
ClearML also supports reporting media (such as audio, video and images) for every iteration.
2022-05-18 08:49:31 +00:00
This section is mostly used for debugging. It's recommended to use [artifacts](task.md#artifacts) for storing script
2021-05-13 23:48:51 +00:00
outputs that would be used later on.
Only the last X results of each title / series are saved to prevent overloading the server.
2024-08-04 15:45:38 +00:00
See details in [`Logger.report_media`](../references/sdk/logger.md#report_media).
2021-05-13 23:48:51 +00:00
![image](../img/fundamentals_logger_reported_images.png)
2021-09-01 06:41:27 +00:00
### Explicit Reporting Examples
Check out ClearML's explicit reporting examples for various types of results:
- [Text](../guides/reporting/text_reporting.md)
- [Scalars](../guides/reporting/scalar_reporting.md)
- Plots
- [2d plots](../guides/reporting/scatter_hist_confusion_mat_reporting.md)
- Histograms
- Confusion matrices
- Scatter plots
- [3d plots](../guides/reporting/3d_plots_reporting.md)
- Surface plots
- Scatter plots
- [Tables](../guides/reporting/pandas_reporting.md)
- Pandas DataFrames
- CSV file
- [Matplotlib figures](../guides/reporting/manual_matplotlib_reporting.md)
- [Plotly figures](../guides/reporting/plotly_reporting.md)
- Debug Samples
- [Images](../guides/reporting/image_reporting.md)
- [HTML](../guides/reporting/html_reporting.md)
- [Media - images, audio, video](../guides/reporting/media_reporting.md)
- Explicit reporting in Jupyter Notebook [example](../guides/reporting/clearml_logging_example.md)
## Logger Configuration
The Logger class provides methods to control aspects of ClearML's logging.
### Upload Destination
Set the default storage URI for uploading debug samples using the [`Logger.set_default_upload_destination`](../references/sdk/logger.md#set_default_upload_destination) method.
The debug samples are uploaded separately. A link to each sample is reported.
:::note DESTINATION STORAGE CREDENTIALS
Credentials for the destination storage are specified in the [ClearML configuration file](../configs/clearml_conf.md#sdk-section).
:::
### Automatic Logging Settings
The Logger class provides methods for fine-tuning ClearML's automatic logging behavior with Matplotlib and Tensorboard.
2023-01-19 15:21:52 +00:00
For example, use the [`Logger.matplotlib_force_report_non_interactive`](../references/sdk/logger.md#loggermatplotlib_force_report_non_interactive)
class method to control how matplotlib objects are logged. See the [`Logger.tensorboard_auto_group_scalars`](../references/sdk/logger.md#loggertensorboard_auto_group_scalars)
2023-10-17 07:46:01 +00:00
class method.
### Set Default NaN and Inf Values
When you report metrics that include NaN or Inf values, ClearML logs them as `0` by default. You can specify
different default values for NaN and Inf using the [`Logger.set_reporting_nan_value`](../references/sdk/logger.md#loggerset_reporting_nan_value)
and the [`Logger.set_reporting_inf_value`](../references/sdk/logger.md#loggerset_reporting_inf_value) class methods respectively.