ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Small edits (#136)

Browse Source
This commit is contained in:
pollfly
2021-12-22 10:54:04 +02:00
committed by GitHub
parent 5bc60cfac6
commit eae9708461
10 changed files with 165 additions and 137 deletions
Download Patch File Download Diff File Expand all files Collapse all files

242
docs/guides/reporting/clearml_logging_example.md
View File

@@ -1,9 +1,9 @@
---
title: Explicit Reporting - Jupyter Notebook
title: Using Logger - Jupyter Notebook
---
The [jupyter_logging_example.ipynb](https://github.com/allegroai/clearml/blob/master/examples/reporting/jupyter_logging_example.ipynb)
script demonstrates the integration of **ClearML** explicit reporting running in a Jupyter Notebook. All **ClearML**
script demonstrates the integration of ClearML's explicit reporting module, `Logger`, in a Jupyter Notebook. All ClearML
explicit reporting works with Jupyter Notebook.
This example includes several types of explicit reporting, including:
@@ -20,19 +20,23 @@ In the ``clearml`` GitHub repository, this example includes a clickable icon to
To reports scalars, call the [Logger.report_scalar](../../references/sdk/logger.md#report_scalar)
method. The scalar plots appear in the **web UI** in **RESULTS** **>** **SCALARS**.
# report two scalar series on two different graphs
for i in range(10):
logger.report_scalar("graph A", "series A", iteration=i, value=1./(i+1))
logger.report_scalar("graph B", "series B", iteration=i, value=10./(i+1))
```python
# report two scalar series on two different graphs
for i in range(10):
logger.report_scalar("graph A", "series A", iteration=i, value=1./(i+1))
logger.report_scalar("graph B", "series B", iteration=i, value=10./(i+1))
```
![image](../../img/colab_explicit_reporting_01.png)
![Separate scalar plots](../../img/colab_explicit_reporting_01.png)
# report two scalar series on the same graph
for i in range(10):
logger.report_scalar("unified graph", "series A", iteration=i, value=1./(i+1))
logger.report_scalar("unified graph", "series B", iteration=i, value=10./(i+1))
```python
# report two scalar series on the same graph
for i in range(10):
logger.report_scalar("unified graph", "series A", iteration=i, value=1./(i+1))
logger.report_scalar("unified graph", "series B", iteration=i, value=10./(i+1))
```
![image](../../img/colab_explicit_reporting_02.png)
![Unified scalar plots](../../img/colab_explicit_reporting_02.png)
## Plots
@@ -43,114 +47,126 @@ Plots appear in **RESULTS** **>** **PLOTS**.
Report 2D scatter plots by calling the [Logger.report_scatter2d](../../references/sdk/logger.md#report_scatter2d) method.
Use the `mode` parameter to plot data points as markers, or both lines and markers.
scatter2d = np.hstack(
(np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1)))
)
# report 2d scatter plot with markers
logger.report_scatter2d(
"example_scatter",
"series_lines+markers",
iteration=iteration,
scatter=scatter2d,
xaxis="title x",
yaxis="title y",
mode='lines+markers'
)
```python
scatter2d = np.hstack(
(np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1)))
)
# report 2d scatter plot with markers
logger.report_scatter2d(
"example_scatter",
"series_lines+markers",
iteration=iteration,
scatter=scatter2d,
xaxis="title x",
yaxis="title y",
mode='lines+markers'
)
```
![image](../../img/colab_explicit_reporting_04.png)
![2d scatter plot](../../img/colab_explicit_reporting_04.png)
### 3D Plots
To plot a series as a 3-dimensional scatter plot, use the [Logger.report_scatter3d](../../references/sdk/logger.md#report_scatter3d) method.
# report 3d scatter plot
scatter3d = np.random.randint(10, size=(10, 3))
logger.report_scatter3d(
"example_scatter_3d",
"series_xyz",
iteration=iteration,
scatter=scatter3d,
xaxis="title x",
yaxis="title y",
zaxis="title z",
)
```python
# report 3d scatter plot
scatter3d = np.random.randint(10, size=(10, 3))
logger.report_scatter3d(
"example_scatter_3d",
"series_xyz",
iteration=iteration,
scatter=scatter3d,
xaxis="title x",
yaxis="title y",
zaxis="title z",
)
```
![image](../../img/colab_explicit_reporting_05.png)
![3d scatter plot](../../img/colab_explicit_reporting_05.png)
To plot a series as a surface plot, use the [Logger.report_surface](../../references/sdk/logger.md#report_surface)
method.
# report 3d surface
surface = np.random.randint(10, size=(10, 10))
logger.report_surface(
"example_surface",
"series1",
iteration=iteration,
matrix=surface,
xaxis="title X",
yaxis="title Y",
zaxis="title Z",
)
```python
# report 3d surface
surface = np.random.randint(10, size=(10, 10))
logger.report_surface(
"example_surface",
"series1",
iteration=iteration,
matrix=surface,
xaxis="title X",
yaxis="title Y",
zaxis="title Z",
)
```
![image](../../img/colab_explicit_reporting_06.png)
![3d surface plot](../../img/colab_explicit_reporting_06.png)
### Confusion Matrices
Report confusion matrices by calling the [Logger.report_matrix](../../references/sdk/logger.md#report_matrix)
method.
# report confusion matrix
confusion = np.random.randint(10, size=(10, 10))
logger.report_matrix(
"example_confusion",
"ignored",
iteration=iteration,
matrix=confusion,
xaxis="title X",
yaxis="title Y",
)
```python
# report confusion matrix
confusion = np.random.randint(10, size=(10, 10))
logger.report_matrix(
"example_confusion",
"ignored",
iteration=iteration,
matrix=confusion,
xaxis="title X",
yaxis="title Y",
)
```
![image](../../img/colab_explicit_reporting_03.png)
![Confusion matrix](../../img/colab_explicit_reporting_03.png)
### Histograms
Report histograms by calling the [Logger.report_histogram](../../references/sdk/logger.md#report_histogram)
method. To report more than one series on the same plot, use the same `title` argument.
# report a single histogram
histogram = np.random.randint(10, size=10)
logger.report_histogram(
"single_histogram",
"random histogram",
iteration=iteration,
values=histogram,
xaxis="title x",
yaxis="title y",
)
```python
# report a single histogram
histogram = np.random.randint(10, size=10)
logger.report_histogram(
"single_histogram",
"random histogram",
iteration=iteration,
values=histogram,
xaxis="title x",
yaxis="title y",
)
```
![image](../../img/colab_explicit_reporting_12.png)
![Histogram](../../img/colab_explicit_reporting_12.png)
# report a two histograms on the same plot
histogram1 = np.random.randint(13, size=10)
histogram2 = histogram * 0.75
logger.report_histogram(
"two_histogram",
"series 1",
iteration=iteration,
values=histogram1,
xaxis="title x",
yaxis="title y",
)
logger.report_histogram(
"two_histogram",
"series 2",
iteration=iteration,
values=histogram2,
xaxis="title x",
yaxis="title y",
)
```python
# report a two histograms on the same plot
histogram1 = np.random.randint(13, size=10)
histogram2 = histogram * 0.75
logger.report_histogram(
"two_histogram",
"series 1",
iteration=iteration,
values=histogram1,
xaxis="title x",
yaxis="title y",
)
logger.report_histogram(
"two_histogram",
"series 2",
iteration=iteration,
values=histogram2,
xaxis="title x",
yaxis="title y",
)
```
![image](../../img/colab_explicit_reporting_07.png)
![Two histograms in one plot](../../img/colab_explicit_reporting_07.png)
## Media
@@ -162,39 +178,51 @@ method.
For example, to download an image:
image_local_copy = StorageManager.get_local_copy(
remote_url="https://pytorch.org/tutorials/_static/img/neural-style/picasso.jpg",
name="picasso.jpg"
)
```python
image_local_copy = StorageManager.get_local_copy(
remote_url="https://pytorch.org/tutorials/_static/img/neural-style/picasso.jpg",
name="picasso.jpg"
)
```
### Audio
logger.report_media('audio', 'pink panther', iteration=1, local_path=audio_local_copy)
```python
logger.report_media('audio', 'pink panther', iteration=1, local_path=audio_local_copy)
```
![image](../../img/colab_explicit_reporting_08.png)
![Audio sample](../../img/colab_explicit_reporting_08.png)
### HTML
logger.report_media("html", "url_html", iteration=1, url="https://allegro.ai/docs/index.html")
```python
logger.report_media("html", "url_html", iteration=1, url="https://allegro.ai/docs/index.html")
```
![image](../../img/colab_explicit_reporting_09.png)
![HTML sample](../../img/colab_explicit_reporting_09.png)
### Images
logger.report_image("image", "image from url", iteration=100, local_path=image_local_copy)
```python
logger.report_image("image", "image from url", iteration=100, local_path=image_local_copy)
```
![image](../../img/colab_explicit_reporting_10.png)
![Image sample](../../img/colab_explicit_reporting_10.png)
### Video
logger.report_media('video', 'big bunny', iteration=1, local_path=video_local_copy)
```python
logger.report_media('video', 'big bunny', iteration=1, local_path=video_local_copy)
```
![image](../../img/colab_explicit_reporting_11.png)
![Video sample](../../img/colab_explicit_reporting_11.png)
## Text
Report text messages by calling the [Logger.report_text](../../references/sdk/logger.md#report_text).
logger.report_text("hello, this is plain text")
![image](../../img/colab_explicit_reporting_13.png)
```python
logger.report_text("hello, this is plain text")
```
![Text report to console](../../img/colab_explicit_reporting_13.png)

25
docs/guides/reporting/explicit_reporting.md
View File

@@ -1,15 +1,15 @@
---
title: Explicit Reporting
title: Explicit Reporting Tutorial
---
In this tutorial, learn how to extend **ClearML** automagical capturing of inputs and outputs with explicit reporting.
In this tutorial, learn how to extend ClearML automagical capturing of inputs and outputs with explicit reporting.
In this example, we will add the following to the [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
example script from ClearML's GitHub repo:
* Setting an output destination for model checkpoints (snapshots).
* Explicitly logging a scalar, other (non-scalar) data, and logging text.
* Registering an artifact, which is uploaded to **ClearML Server**, and **ClearML** logs changes to it.
* Registering an artifact, which is uploaded to **ClearML Server**, and ClearML logs changes to it.
* Uploading an artifact, which is uploaded, but changes to it are not logged.
## Prerequisites
@@ -19,10 +19,9 @@ example script from ClearML's GitHub repo:
## Before Starting
Make a copy of `pytorch_mnist.py` in order to add explicit reporting to it.
Make a copy of [`pytorch_mnist.py`](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
in order to add explicit reporting to it.
* In the local **ClearML** repository, `example` directory.
```bash
cp pytorch_mnist.py pytorch_mnist_tutorial.py
```
@@ -59,7 +58,7 @@ task = Task.init(project_name='examples',
output_uri=model_snapshots_path)
```
When the script runs, **ClearML** creates the following directory structure:
When the script runs, ClearML creates the following directory structure:
+ - <output destination name>
| +-- <project name>
@@ -79,7 +78,7 @@ For example, if the Task ID is `9ed78536b91a44fbb3cc7a006128c1b0`, then the dire
## Step 2: Logger Class Reporting Methods
In addition to **ClearML** automagical logging, the **ClearML** Python
In addition to ClearML automagical logging, the `clearml` Python
package contains methods for explicit reporting of plots, log text, media, and tables. These methods include:
* [Logger.report_histogram](../../references/sdk/logger.md#report_histogram)
@@ -99,6 +98,7 @@ package contains methods for explicit reporting of plots, log text, media, and t
First, create a logger for the Task using the [Task.get_logger](../../references/sdk/task.md#get_logger)
method.
```python
logger = task.get_logger
```
@@ -187,7 +187,7 @@ def test(args, model, device, test_loader):
### Log Text
Extend **ClearML** by explicitly logging text, including errors, warnings, and debugging statements. We use the [Logger.report_text](../../references/sdk/logger.md#report_text)
Extend ClearML by explicitly logging text, including errors, warnings, and debugging statements. We use the [Logger.report_text](../../references/sdk/logger.md#report_text)
method and its argument `level` to report a debugging message.
```python
@@ -203,7 +203,7 @@ logger.report_text(
## Step 3: Registering Artifacts
Registering an artifact uploads it to **ClearML Server**, and if it changes, the change is logged in **ClearML Server**.
Currently, **ClearML** supports Pandas DataFrames as registered artifacts.
Currently, ClearML supports Pandas DataFrames as registered artifacts.
### Register the Artifact
@@ -245,7 +245,6 @@ sample = Task.current_task().get_registered_artifacts()['Test_Loss_Correct'].sam
replace=True,
random_state=1
)
```
## Step 4: Uploading Artifacts
@@ -280,7 +279,9 @@ task.upload_artifact(
After extending the Python experiment script, run it and view the results in the **ClearML Web UI**.
python pytorch_mnist_tutorial.py
```bash
python pytorch_mnist_tutorial.py
```
**To view the experiment results, do the following:**