mirror of https://github.com/clearml/clearml synced 2025-01-31 09:07:00 +00:00

allegroai 00e66b5d63 Add missing x/y/z axis titles to various logger report

Changed report_surface arguments

2019-10-27 00:32:05 +03:00

35 KiB

Raw Blame History

TRAINS Explicit Logging

Using the TRAINS Logger module and other TRAINS features, you can explicitly log any of the following:

Report graphs and images
Track hyper-parameters and OS environment variables
- Logging experiment parameter dictionaries
- Specifying environment variables to track
Message logging
- Reporting text without formatting

Additionally, the TRAINS Logger module provides methods that allow you to do the following:

Get the current logger
Overrride the TRAINS configuration file with a default upload destination for images and files

Graphs and Images

Scalar Metrics

Use to report scalar metrics by iteration as a line plot.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_scalar(self, title, series, value, iteration)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
value	float	The scalar metric data value (y-axis).	Yes

Histograms

Use to report any data by iteration as a histogram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_histogram(self, title, series, values, iteration, labels=None, xlabels=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
values	Either: list of float numpy array	The histogram data values (y-axis).	Yes
labels	list of strings	Labels for each bar group in the histogram. The default value is `None`.	No
xlabels	list of strings	Labels for each bucket in the histogram. Each label in the `xlabels` list corresponds to a value in the `values` list (or numpy array). The default value is `None`.	No

Line Plots

Use to report any data by iteration as a single or multiple line plot.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_line_plot(self, title, series, iteration, xaxis, yaxis, mode='lines', reverse_xaxis=False, comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
series	LineSeriesInfo	One (single line plot) or more (multiple line plot) series of data.	Yes
title	string	The plot title.	Yes
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is `None`.	No
mode	string	Type of line plot. The values are: `lines` - lines connecting data points (default) `markers` - markers for each data point `lines+markers` - lines and markers	No
reverse_xaxis	boolean	Indicates whether to display the x-axis values in ascending or descending order. The values are: `True` - descending order `False` - ascending order (default)	No
xaxis	string	x-axis title.	No
yaxis	string	y-axis title.	No

2D Scatter Diagrams

Use to report any vector data as a 2D scatter diagram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_scatter2d(self, title, series, scatter, iteration, xaxis=None, yaxis=None, labels=None, mode='lines', comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
scatter	Either: list of (pairs of x, y) ndarray	The scatter data. For example, a list of pairs in the form: `[(x1,y1),(x2,y2),(x3,y3),...]`.	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is `None`.	No
labels	list of strings	Label text per data point in the scatter diagram. The default value is `None`.	No
mode	string	Type of 2D scatter diagram. The values are: `lines` - lines connecting data points (default) `markers` - markers for each data point `lines+markers` - lines and markers	No
xaxis	string	x-axis title. The default value is `None`.	No
yaxis	string	y-axis title. The default value is `None`.	No

3D Scatter Diagrams

Use to report any array data as a 3D scatter diagram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_scatter3d(self, title, series, scatter, iteration, labels=None, mode='markers', fill=False, comment=None)

Argument:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
scatter	Either: list of (pairs of x, y, z) ndarray list of series (x1,y1,z1)...	The scatter data. For example, a list of series in the form: `[[(x1,y1,z1),(x2,y2,z2),...],[(x3,y3,z3),(x4,y4,z4),...],[(x5,y5,z5),(x6,y6,z6),...]]`	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is `None`.	No
fill	boolean	Indicates whether to fill the area under the scatter diagram. The values are: `True` - fill `False` - do not fill (default)	No
labels	list of strings	Label text per data point in the scatter. The default value is `None`.	No
mode	string	Type of 3D scatter diagram. The values are: `lines` `markers` `lines+markers` The default values is `lines`.	No

Confusion Matrices

Use to report a heat-map matrix as a confusion matrix. You can also plot a heat-map as a surface diagram.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_confusion_matrix(self, title, series, matrix, iteration, xlabels=None, ylabels=None, comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
matrix	ndarray	A heat-map matrix.	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
xlabels	list of strings	Label per column of the matrix. The default value is `None`.	No
ylabels	list of strings	Label per row of the matrix. The default value is `None`.	No

Surface Diagrams

Use to plot a heat-map matrix as a surface diagram. You can also plot a heat-map as a confusion matrix.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_surface(self, title, series, matrix, iteration, xlabels=None, ylabels=None, xaxis=None, yaxis=None, camera=None, comment=None)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number (x-axis).	Yes
matrix	ndarray	A heat-map matrix.	Yes
series	string	The data series label.	Yes
title	string	The plot title.	Yes
camera	tuple	The position of the camera as (x, y, x), if applicable.	No
comment	string	Any text (e.g., subtitle or other comment) which displays under the plot title. The default value is `None`.	No
xlabels	list of strings	Label per column of the matrix. The default value is `None`.	No
xaxis	string	x-axis title.	No
ylabels	list of strings	Label per row of the matrix. The default value is `None`.	No
yaxis	string	y-axis title.	No

Images

Use to report an image and upload its contents to the bucket specified in the TRAINS configuration file, or a a default upload destination, if you set a default.

First get the current logger and then use it (see an example script) with the following method.

Method:

def report_image(self, title, series, iteration, local_path=None, matrix=None, max_image_history=None, delete_after_upload=False)

Arguments:

Parameter	Type	Description	Mandatory
iteration	integer	The iteration number.	Yes
series	string	The label of the series.	Yes
title	string	The title of the image.	Yes
delete_after_upload	boolean	Indicates whether to delete the local copy of the file after uploading it. The values are: `True` - delete `False` - do not delete (default)	No
matrix	ndarray	A 3D numpy.ndarray object containing image data (RGB). If `path` is not specified, then `matrix` is required. The default values is `None`.	No
max_image_history	integer	The maximum number of images to store per metric / variant combination. For an unlimited number of images to store, specify a negative number. The default value which is set in the global configuration is `5`.	No
local_path	string	The path of the image file. If `matrix` is not specified, then `path` is required. The default values is `None`.	No

Hyper-parameters and Environment Variables

Logging Experiment Parameter Dictionaries

In order for TRAINS to log a dictionary of parameters, use the Task.connect method.

For example, to log the hyper-parameters learning_rate, batch_size, display_step, model_path, n_hidden_1, and n_hidden_2:

# Create a dictionary of parameters
parameters_dict = { 'learning_rate': 0.001, 'batch_size': 100, 'display_step': 1, 
    'model_path': "/tmp/model.ckpt", 'n_hidden_1': 256, 'n_hidden_2': 256 }
    
# Connect the dictionary to your TRAINS Task    
parameters_dict = Task.current_task().connect(parameters_dict)

Specifying Environment Variables to Track

By setting the TRAINS_LOG_ENVIRONMENT environment variable, make TRAINS log either:

All environment variables
```
  export TRAINS_LOG_ENVIRONMENT="*"
```
Specific environment variables

For example, log PWD and PYTHONPATH
```
  export TRAINS_LOG_ENVIRONMENT="PWD,PYTHONPATH"
```
No environment variables
```
  export TRAINS_LOG_ENVIRONMENT=
```

Logging Messages

Use the methods in this section to log various types of messages. The method name describes the type of message.

Debugging Messages

Method:

def debug(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Informational Messages

Method:

def info(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Warnings

Method:

def warn(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

General Errors

Method:

def error(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Critical Errors

Method:

def critical(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Fatal Errors

Method:

def fatal(self, msg, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes

Console and Logger Messages

Method:

def console(self, msg, level=logging.INFO, omit_console=False, *args, **kwargs)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes
level	string	The log level. The values are: `logging.DEBUG` `logging.INFO` `logging.WARNING` `logging.ERROR` `logging.FATAL` `logging.CRITICAL`	No
omit_console	boolean	Indicates whether to send the message to the log only. The values are: `True` - send the message to the log only `False` - send the message to the console and the log	No

Reporting Text Without Formatting

Method:

def report_text(self, msg, level=logging.INFO, print_console=False, *args, **_)

First get the current logger and then use it (see an example script) with the following method.

Arguments:

Parameter	Type	Description	Mandatory
msg	string	The text to log.	Yes
level	string	The log level. The values are: `logging.DEBUG` `logging.INFO` `logging.WARNING` `logging.ERROR` `logging.FATAL` `logging.CRITICAL`	No
print_console	boolean	Indicates whether to log to the console, in addition to the log. The values are: `True` - print to the console and log `False` - print to the log, only (default)	No

Logger Object and Storage Methods

Get the Current Logger

Use to return a reference to the current logger object.

Method:

def current_logger(cls)

Arguments:

None.

Set Default Upload Destination

Use to specify the default destination storage location used for uploading images. Images are uploaded and a link to the image is reported.

Credentials for the storage location are in the global configuration file (for example, on Linux, ~/trains.conf).

Method:

def set_default_upload_destination(self, uri)

Arguments:

Parameter	Type	Description	Mandatory
uri	string	The destination storage location, for example `s3://bucket/directory/` or `file:///tmp/debug/`.	Yes

35 KiB Raw Blame History

TRAINS Explicit Logging

Graphs and Images

Scalar Metrics

Histograms

Line Plots

2D Scatter Diagrams

3D Scatter Diagrams

Confusion Matrices

Surface Diagrams

Images

Hyper-parameters and Environment Variables

Logging Experiment Parameter Dictionaries

Specifying Environment Variables to Track

Logging Messages

Debugging Messages

Informational Messages

Warnings

General Errors

Critical Errors

Fatal Errors

Console and Logger Messages

Reporting Text Without Formatting

Logger Object and Storage Methods

Get the Current Logger

Set Default Upload Destination

35 KiB

Raw Blame History