---
title: Explicit Reporting Tutorial
---
In this tutorial, learn how to extend ClearML automagical capturing of inputs and outputs with explicit reporting.
In this example, we will add the following to the [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
example script from ClearML's GitHub repo:
* Setting an output destination for model checkpoints (snapshots).
* Explicitly logging a scalar, other (non-scalar) data, and logging text.
* Registering an artifact, which is uploaded to **ClearML Server**, and ClearML logs changes to it.
* Uploading an artifact, which is uploaded, but changes to it are not logged.
## Prerequisites
* The [clearml](https://github.com/allegroai/clearml) repository is cloned.
* The `clearml` package is installed.
## Before Starting
Make a copy of [`pytorch_mnist.py`](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
in order to add explicit reporting to it.
```bash
cp pytorch_mnist.py pytorch_mnist_tutorial.py
```
## Step 1: Setting an Output Destination for Model Checkpoints
Specify a default output location, which is where model checkpoints (snapshots) and artifacts will be stored when the
experiment runs. Some possible destinations include:
* Local destination
* Shared folder
* Cloud storage:
* S3 EC2
* Google Cloud Storage
* Azure Storage.
Specify the output location in the `output_uri` parameter of the [Task.init](../../references/sdk/task.md#taskinit) method.
In this tutorial, we specify a local folder destination.
In `pytorch_mnist_tutorial.py`, change the code from:
```python
task = Task.init(project_name='examples', task_name='pytorch mnist train')
```
to:
```python
model_snapshots_path = '/mnt/clearml'
if not os.path.exists(model_snapshots_path):
os.makedirs(model_snapshots_path)
task = Task.init(project_name='examples',
task_name='extending automagical ClearML example',
output_uri=model_snapshots_path)
```
When the script runs, ClearML creates the following directory structure:
+ - <output destination name>
| +-- <project name>
| +-- <task name>.<Task Id>
| +-- models
| +-- artifacts
and puts the model checkpoints (snapshots) and artifacts in that folder.
For example, if the Task ID is `9ed78536b91a44fbb3cc7a006128c1b0`, then the directory structure will be:
+ - model_snapshots
| +-- examples
| +-- extending automagical ClearML example.9ed78536b91a44fbb3cc7a006128c1b0
| +-- models
| +-- artifacts
## Step 2: Logger Class Reporting Methods
In addition to ClearML automagical logging, the `clearml` Python
package contains methods for explicit reporting of plots, log text, media, and tables. These methods include:
* [Logger.report_histogram](../../references/sdk/logger.md#report_histogram)
* [Logger.report_confusion_matrix](../../references/sdk/logger.md#report_confusion_matrix)
* [Logger.report_line_plot](../../references/sdk/logger.md#report_line_plot)
* [Logger.report_scatter2d](../../references/sdk/logger.md#report_scatter2d)
* [Logger.report_scatter3d](../../references/sdk/logger.md#report_scatter3d)
* [Logger.report_surface](../../references/sdk/logger.md#report_surface)
(surface diagrams)
* [Logger.report_image](../../references/sdk/logger.md#report_image) - Report an image and upload its contents.
* [Logger.report_table](../../references/sdk/logger.md#report_table) - Report a table as a Pandas DataFrame, CSV file,
or URL for a CSV file.
* [Logger.report_media](../../references/sdk/logger.md#report_media) - Report media including images, audio, and video.
* [Logger.get_default_upload_destination](../../references/sdk/logger.md#get_default_upload_destination) - Retrieve the destination that is set for uploaded media.
### Get a Logger
First, create a logger for the Task using the [Task.get_logger](../../references/sdk/task.md#get_logger)
method.
```python
logger = task.get_logger
```
### Plot Scalar Metrics
Add scalar metrics using the [Logger.report_scalar](../../references/sdk/logger.md#report_scalar)
method to report loss metrics.
```python
def train(args, model, device, train_loader, optimizer, epoch):
save_loss = []
model.train()
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
optimizer.zero_grad()
output = model(data)
loss = F.nll_loss(output, target)
loss.backward()
save_loss.append(loss)
optimizer.step()
if batch_idx % args.log_interval == 0:
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
epoch, batch_idx * len(data), len(train_loader.dataset),
100. * batch_idx / len(train_loader), loss.item()))
# Add manual scalar reporting for loss metrics
logger.report_scalar(title='Scalar example {} - epoch'.format(epoch),
series='Loss', value=loss.item(), iteration=batch_idx)
```
### Plot Other (Not Scalar) Data
The script contains a function named `test`, which determines loss and correct for the trained model. We add a histogram
and confusion matrix to log them.
```python
def test(args, model, device, test_loader):
save_test_loss = []
save_correct = []
model.eval()
test_loss = 0
correct = 0
with torch.no_grad():
for data, target in test_loader:
data, target = data.to(device), target.to(device)
output = model(data)
# sum up batch loss
test_loss += F.nll_loss(output, target, reduction='sum').item()
# get the index of the max log-probability
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(target.view_as(pred)).sum().item()
save_test_loss.append(test_loss)
save_correct.append(correct)
test_loss /= len(test_loader.dataset)
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
test_loss, correct, len(test_loader.dataset),
100. * correct / len(test_loader.dataset)))
logger.report_histogram(
title='Histogram example',
series='correct',
iteration=1,
values=save_correct,
xaxis='Test',
yaxis='Correct'
)
# Manually report test loss and correct as a confusion matrix
matrix = np.array([save_test_loss, save_correct])
logger.report_confusion_matrix(
title='Confusion matrix example',
series='Test loss / correct',
matrix=matrix,
iteration=1
)
```
### Log Text
Extend ClearML by explicitly logging text, including errors, warnings, and debugging statements. We use the [Logger.report_text](../../references/sdk/logger.md#report_text)
method and its argument `level` to report a debugging message.
```python
logger.report_text(
'The default output destination for model snapshots and artifacts is: {}'.format(
model_snapshots_path
),
level=logging.DEBUG
)
```
## Step 3: Registering Artifacts
Registering an artifact uploads it to **ClearML Server**, and if it changes, the change is logged in **ClearML Server**.
Currently, ClearML supports Pandas DataFrames as registered artifacts.
### Register the Artifact
In the tutorial script, `test` function, we can assign the test loss and correct data to a Pandas DataFrame object and register
that Pandas DataFrame using the [Task.register_artifact](../../references/sdk/task.md#register_artifact) method.
```python
# Create the Pandas DataFrame
test_loss_correct = {
'test lost': save_test_loss,
'correct': save_correct
}
df = pd.DataFrame(test_loss_correct, columns=['test lost','correct'])
# Register the test loss and correct as a Pandas DataFrame artifact
task.register_artifact(
'Test_Loss_Correct',
df,
metadata={
'metadata string': 'apple',
'metadata int': 100,
'metadata dict': {'dict string': 'pear', 'dict int': 200}
}
)
```
### Reference the Registered Artifact
Once an artifact is registered, it can be referenced and utilized in the Python experiment script.
In the tutorial script, we add [Task.current_task](../../references/sdk/task.md#taskcurrent_task) and
[Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
methods to take a sample.
```python
# Once the artifact is registered, we can get it and work with it. Here, we sample it.
sample = Task.current_task().get_registered_artifacts()['Test_Loss_Correct'].sample(
frac=0.5,
replace=True,
random_state=1
)
```
## Step 4: Uploading Artifacts
Artifact can be uploaded to the **ClearML Server**, but changes are not logged.
Supported artifacts include:
* Pandas DataFrames
* Files of any type, including image files
* Folders - stored as ZIP files
* Images - stored as PNG files
* Dictionaries - stored as JSONs
* Numpy arrays - stored as NPZ files
In the tutorial script, we upload the loss data as an artifact using the [Task.upload_artifact](../../references/sdk/task.md#upload_artifact)
method with metadata specified in the `metadata` parameter.
```python
# Upload test loss as an artifact. Here, the artifact is numpy array
task.upload_artifact(
'Predictions',
artifact_object=np.array(save_test_loss),
metadata={
'metadata string': 'banana',
'metadata integer': 300,
'metadata dictionary': {'dict string': 'orange', 'dict int': 400}
}
)
```
## Additional Information
After extending the Python experiment script, run it and view the results in the **ClearML Web UI**.
```bash
python pytorch_mnist_tutorial.py
```
**To view the experiment results, do the following:**
1. In the **ClearML Web UI**, on the Projects page, click the examples project.
1. In the experiments table, click the **Extending automagical ClearML example** experiment.
1. In the **ARTIFACTS** tab, **DATA AUDIT** section, click **Test_Loss_Correct**. The registered Pandas DataFrame appears,
including the file path, size, hash, metadata, and a preview.
1. In the **OTHER** section, click **Loss**. The uploaded numpy array appears, including its related information.
1. Click the **RESULTS** tab.
1. Click the **CONSOLE** sub-tab, and see the debugging message showing the Pandas DataFrame sample.
1. Click the **SCALARS** sub-tab, and see the scalar plots for epoch logging loss.
1. Click the **PLOTS** sub-tab, and see the confusion matrix and histogram.
## Next Steps
* See the [User Interface](../../webapp/webapp_overview.md) section to learn about its features.
* See the [ClearML Python Package Reference](../../clearml_sdk.md) to learn about
all the available classes and methods.