mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-03 18:53:37 +00:00
Add abseil example (#157)
This commit is contained in:
parent
b8fc589a4b
commit
15d28865c3
71
docs/guides/frameworks/pytorch/pytorch_abseil.md
Normal file
71
docs/guides/frameworks/pytorch/pytorch_abseil.md
Normal file
@ -0,0 +1,71 @@
|
|||||||
|
---
|
||||||
|
title: PyTorch Abseil
|
||||||
|
---
|
||||||
|
|
||||||
|
The [pytorch_abseil.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_abseil.py)
|
||||||
|
example demonstrates the integration of ClearML into code that uses PyTorch and [`absl.flags`](https://abseil.io/docs/python/guides/flags).
|
||||||
|
|
||||||
|
The example script does the following:
|
||||||
|
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
|
||||||
|
dataset
|
||||||
|
* Creates an experiment named `pytorch mnist train with abseil`, which is associated with the `examples` project
|
||||||
|
* ClearML automatically logs the absl.flags, and the models (and their snapshots) created by PyTorch
|
||||||
|
* Additional metrics are logged by calling the [Logger.report_scalar](../../../references/sdk/logger.md#report_scalar)
|
||||||
|
method
|
||||||
|
|
||||||
|
## Scalars
|
||||||
|
|
||||||
|
In the example script's `train` function, the following code explicitly reports scalars to **ClearML**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
Logger.current_logger().report_scalar(
|
||||||
|
"train",
|
||||||
|
"loss",
|
||||||
|
iteration=(epoch * len(train_loader) + batch_idx),
|
||||||
|
value=loss.item()
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
In the `test` method, the code explicitly reports `loss` and `accuracy` scalars.
|
||||||
|
|
||||||
|
```python
|
||||||
|
Logger.current_logger().report_scalar(
|
||||||
|
"test", "loss", iteration=epoch, value=test_loss
|
||||||
|
)
|
||||||
|
Logger.current_logger().report_scalar(
|
||||||
|
"test",
|
||||||
|
"accuracy",
|
||||||
|
iteration=epoch,
|
||||||
|
value=(correct / len(test_loader.dataset))
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
These scalars can be visualized in plots, which appear in the [ClearML web UI](../../../webapp/webapp_overview.md), in
|
||||||
|
the experiment's page **>** **RESULTS** **>** **SCALARS**.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
## Hyperparameters
|
||||||
|
|
||||||
|
ClearML automatically logs command line options defined with abseil flags. They appear in **CONFIGURATIONS** **>**
|
||||||
|
**HYPER PARAMETERS** **>** **TF_DEFINE**.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
## Console
|
||||||
|
|
||||||
|
Text printed to the console for training progress, as well as all other console output, appear in **RESULTS** **>** **CONSOLE**.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
## Artifacts
|
||||||
|
|
||||||
|
Models created by the experiment appear in the experiment's **ARTIFACTS** tab. ClearML automatically logs and tracks
|
||||||
|
models and any snapshots created using PyTorch.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
Clicking on the model name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can view
|
||||||
|
the model's details and access the model.
|
||||||
|
|
||||||
|

|
BIN
docs/img/examples_pytorch_abseil_models.png
Normal file
BIN
docs/img/examples_pytorch_abseil_models.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 23 KiB |
BIN
docs/img/examples_pytorch_abseil_models_2.png
Normal file
BIN
docs/img/examples_pytorch_abseil_models_2.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 76 KiB |
BIN
docs/img/examples_pytorch_abseil_params.png
Normal file
BIN
docs/img/examples_pytorch_abseil_params.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 62 KiB |
@ -77,9 +77,10 @@ module.exports = {
|
|||||||
'guides/frameworks/matplotlib/matplotlib_example',
|
'guides/frameworks/matplotlib/matplotlib_example',
|
||||||
'guides/frameworks/megengine/megengine_mnist',
|
'guides/frameworks/megengine/megengine_mnist',
|
||||||
{'PyTorch':
|
{'PyTorch':
|
||||||
['guides/frameworks/pytorch/pytorch_distributed_example', 'guides/frameworks/pytorch/pytorch_matplotlib',
|
[
|
||||||
'guides/frameworks/pytorch/pytorch_mnist', 'guides/frameworks/pytorch/pytorch_tensorboard', 'guides/frameworks/pytorch/pytorch_tensorboardx',
|
'guides/frameworks/pytorch/pytorch_distributed_example', 'guides/frameworks/pytorch/pytorch_matplotlib',
|
||||||
'guides/frameworks/pytorch/tensorboard_toy_pytorch',
|
'guides/frameworks/pytorch/pytorch_mnist', 'guides/frameworks/pytorch/pytorch_tensorboard','guides/frameworks/pytorch/tensorboard_toy_pytorch',
|
||||||
|
'guides/frameworks/pytorch/pytorch_tensorboardx', 'guides/frameworks/pytorch/pytorch_abseil',
|
||||||
{'PyTorch Notebooks': [
|
{'PyTorch Notebooks': [
|
||||||
{'Audio': ['guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K', 'guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example']},
|
{'Audio': ['guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K', 'guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example']},
|
||||||
{'Image': ['guides/frameworks/pytorch/notebooks/image/hyperparameter_search', 'guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10']},
|
{'Image': ['guides/frameworks/pytorch/notebooks/image/hyperparameter_search', 'guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10']},
|
||||||
|
Loading…
Reference in New Issue
Block a user