# ClearML Explicit Logging Using the **ClearML** [Logger](https://github.com/allegroai/clearml/blob/master/clearml/logger.py) module and other **ClearML** features, you can explicitly log any of the following: * Report graphs and images * [Scalar metrics](#scalar-metrics) * [Histograms](#histograms) * [Line plots](#line-plots) * [2D scatter diagrams](#2d-scatter-diagrams) * [3D scatter diagrams](#3d-scatter-diagrams) * [Confusion matrices](#confusion-matrices) * [Surface diagrams](#surface-diagrams) * [Images](#images) * Track hyper-parameters and OS environment variables * Logging experiment parameter [dictionaries](#logging-experiment-parameter-dictionaries) * Specifying [environment variables](#specifying-environment-variables-to-track) to track * Message logging * [Reporting text without formatting](#reporting-text-without-formatting) Additionally, the **ClearML** Logger module provides methods that allow you to do the following: * Get the [current logger]() * Overrride the ClearML configuration file with a [default upload destination]() for images and files ## Graphs and Images ### Scalar Metrics Use to report scalar metrics by iteration as a line plot. First [get the current logger](#get-the-current-logger) and then use it (see an [example script](https://github.com/allegroai/clearml/blob/master/examples/reporting/scalar_reporting.py)) with the following method. **Method**: ```python def report_scalar(self, title, series, value, iteration) ``` **Arguments**:
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
series | string | The data series label. | Yes |
title | string | The plot title. | Yes |
value | float | The scalar metric data value (y-axis). | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
series | string | The data series label. | Yes |
title | string | The plot title. | Yes |
values | Either:
|
The histogram data values (y-axis). | Yes |
labels | list of strings | Labels for each bar group in the histogram. The default value is None .
|
No |
xlabels | list of strings | Labels for each bucket in the histogram. Each label in the xlabels list corresponds to a value in the values list (or numpy array). The default value is None .
|
No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
series | LineSeriesInfo | One (single line plot) or more (multiple line plot) series of data. | Yes |
title | string | The plot title. | Yes |
comment | string | Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None .
|
No |
mode | string | Type of line plot. The values are:
|
No |
reverse_xaxis | boolean | Indicates whether to display the x-axis values in ascending or descending order. The values are:
|
No |
xaxis | string | x-axis title. | No |
yaxis | string | y-axis title. | No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
scatter | Either:
|
The scatter data. For example, a list of pairs in the form: [(x1,y1),(x2,y2),(x3,y3),...] .
|
Yes |
series | string | The data series label. | Yes |
title | string | The plot title. | Yes |
comment | string | Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None .
|
No |
labels | list of strings | Label text per data point in the scatter diagram. The default value is None .
|
No |
mode | string | Type of 2D scatter diagram. The values are:
|
No |
xaxis | string | x-axis title. The default value is None .
|
No |
yaxis | string | y-axis title. The default value is None .
|
No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
scatter | Either:
|
The scatter data. For example, a list of series in the form:
[[(x1,y1,z1),(x2,y2,z2),...],[(x3,y3,z3),(x4,y4,z4),...],[(x5,y5,z5),(x6,y6,z6),...]]
|
Yes |
series | string | The data series label. | Yes |
title | string | The plot title. | Yes |
comment | string | Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None .
|
No |
fill | boolean | Indicates whether to fill the area under the scatter diagram. The values are:
|
No |
labels | list of strings | Label text per data point in the scatter. The default value is None .
|
No |
mode | string | Type of 3D scatter diagram. The values are:
lines .
|
No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
matrix | ndarray | A heat-map matrix. | Yes |
series | string | The data series label. | Yes |
title | string | The plot title. | Yes |
xlabels | list of strings | Label per column of the matrix. The default value is None .
|
No |
ylabels | list of strings | Label per row of the matrix. The default value is None .
|
No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number (x-axis). | Yes |
matrix | ndarray | A heat-map matrix. | Yes |
series | string | The data series label. | Yes |
title | string | The plot title. | Yes |
camera | tuple | The position of the camera as (x, y, x), if applicable. | No |
comment | string | Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is None .
|
No |
xlabels | list of strings | Label per column of the matrix. The default value is None .
|
No |
xaxis | string | x-axis title. | No |
ylabels | list of strings | Label per row of the matrix. The default value is None .
|
No |
yaxis | string | y-axis title. | No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
iteration | integer | The iteration number. | Yes |
series | string | The label of the series. | Yes |
title | string | The title of the image. | Yes |
delete_after_upload | boolean | Indicates whether to delete the local copy of the file after uploading it. The values are:
|
No |
matrix | ndarray | A 3D numpy.ndarray object containing image data (RGB). If path is not specified, then matrix is required. The default values is None .
|
No |
max_image_history | integer | The maximum number of images to store per metric / variant combination. For an unlimited number of images to store, specify a negative number.
The default value which is set in the global configuration is 5 .
|
No |
local_path | string | The path of the image file. If matrix is not specified, then path is required. The default values is None .
|
No |
learning_rate
, batch_size
, display_step
, model_path
, n_hidden_1
, and n_hidden_2
:
```python
# Create a dictionary of parameters
parameters_dict = { 'learning_rate': 0.001, 'batch_size': 100, 'display_step': 1,
'model_path': "/tmp/model.ckpt", 'n_hidden_1': 256, 'n_hidden_2': 256 }
# Connect the dictionary to your ClearML Task
parameters_dict = Task.current_task().connect(parameters_dict)
```
### Specifying Environment Variables to Track
By setting the `CLEARML_LOG_ENVIRONMENT` environment variable, make **ClearML** log either:
* All environment variables
export CLEARML_LOG_ENVIRONMENT="*"
* Specific environment variables
For example, log `PWD` and `PYTHONPATH`
export CLEARML_LOG_ENVIRONMENT="PWD,PYTHONPATH"
* No environment variables
export CLEARML_LOG_ENVIRONMENT=
## Logging Messages
Use the methods in this section to log various types of messages. The method name describes the type of message.
### Debugging Messages
**Method**:
```python
def debug(self, msg, *args, **kwargs)
```
First [get the current logger](#get-the-current-logger) and then use it (see an [example script](https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py)) with the following method.
**Arguments**:
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
level | string | The log level. The values are:
|
No |
omit_console | boolean | Indicates whether to send the message to the log only. The values are:
|
No |
Parameter | Type | Description | Mandatory |
---|---|---|---|
msg | string | The text to log. | Yes |
level | string | The log level. The values are:
|
No |
print_console | boolean | Indicates whether to log to the console, in addition to the log. The values are:
|
No |
~/clearml.conf
).
**Method**:
```python
def set_default_upload_destination(self, uri)
```
**Arguments**:
Parameter | Type | Description | Mandatory |
---|---|---|---|
uri | string | The destination storage location, for example s3://bucket/directory/ or file:///tmp/debug/ .
|
Yes |