ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-05-29 17:50:11 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Small edits (#177)

Browse Source
This commit is contained in:
pollfly 2022-01-30 12:54:40 +02:00 committed by GitHub
parent 1667d6b64e
commit 020cf9a819
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 35 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/clearml_agent.md
View File

@ -128,7 +128,7 @@ Install ClearML Agent as a system Python package and not in a Python virtual env
* Mac - `$HOME/clearml.conf`
* Windows - `\User\<username>\clearml.conf`
1. Optionally, configure **ClearML** options for **ClearML Agent** (default docker, package manager, etc.). See the [ClearML Configuration Reference](configs/clearml_conf.md).
1. Optionally, configure ClearML options for **ClearML Agent** (default docker, package manager, etc.). See the [ClearML Configuration Reference](configs/clearml_conf.md).
:::note
The ClearML Enterprise server provides a [configuration vault](webapp/webapp_profile.md#configuration-vault), the contents
@ -142,7 +142,7 @@ In case a `clearml.conf` file already exists, add a few ClearML Agent specific c
**Adding ClearML Agent to a ClearML configuration file:**
1. Open the **ClearML** configuration file for editing. Depending upon the operating system, it is:
1. Open the ClearML configuration file for editing. Depending upon the operating system, it is:
* Linux - `~/clearml.conf`
* Mac - `$HOME/clearml.conf`
* Windows - `\User\<username>\clearml.conf`
@ -384,7 +384,7 @@ A single agent can listen to multiple queues. The priority is set by their order
```bash
clearml-agent daemon --detached --queue high_q low_q --gpus 0
```
This ensures the agent first tries to pull a Task from the `hiqh_q` queue, and only if it is empty, the agent will try to pull
This ensures the agent first tries to pull a Task from the `high_q` queue, and only if it is empty, the agent will try to pull
from the `low_q` queue.
To make sure an agent pulls from all queues equally, add the `--order-fairness` flag.
@ -392,7 +392,7 @@ To make sure an agent pulls from all queues equally, add the `--order-fairness`
clearml-agent daemon --detached --queue group_a group_b --order-fairness --gpus 0
```
It will make sure the agent will pull from the `group_a` queue, then from `group_b`, then back to `group_a`, etc. This ensures
that `group A` or `group_b` will not be able to starve one another of resources.
that `group_a` or `group_b` will not be able to starve one another of resources.
### Explicit Task Execution
@ -468,7 +468,7 @@ When executing the ClearML Agent in Docker mode, it will:
ClearML Agent uses the provided default Docker container, which can be overridden from the UI.
All ClearML Agent flags (Such as `--gpus` and `--foreground`) are applicable to Docker mode as well.
All ClearML Agent flags (such as `--gpus` and `--foreground`) are applicable to Docker mode as well.
To execute ClearML Agent in Docker mode, run:
```bash

8
docs/deploying_clearml/clearml_server_aws_ec2_ami.md
View File

@ -14,15 +14,15 @@ and in the AWS community AMI catalog.
* [ClearML Server community AMIs](#clearml-server-aws-community-amis) - Configured by default without authentication to allow quick access and onboarding.
After deploying either type of AMI, configure the **ClearML Server** instance to provide the authentication scheme that
After deploying either type of AMI, configure the ClearML Server instance to provide the authentication scheme that
best matches the workflow.
For information about upgrading a **ClearML Server** in an AWS instance, see [here](upgrade_server_aws_ec2_ami.md).
For information about upgrading a ClearML Server in an AWS instance, see [here](upgrade_server_aws_ec2_ami.md).
:::important
If **ClearML Server** is being reinstalled, we recommend clearing browser cookies for **ClearML Server**. For example,
If ClearML Server is being reinstalled, we recommend clearing browser cookies for ClearML Server. For example,
for Firefox, go to Developer Tools > Storage > Cookies, and for Chrome, go to Developer Tools > Application > Cookies,
and delete all cookies under the **ClearML Server** URL.
and delete all cookies under the ClearML Server URL.
:::
## Launching

3
docs/deploying_clearml/clearml_server_linux_mac.md
View File

@ -23,6 +23,7 @@ and delete all cookies under the ClearML Server URL.
## Prerequisites
For Linux users only:
* Linux distribution must support Docker. For more information, see this [explanation](https://docs.docker.com/engine/install/) in the Docker documentation.
@ -123,7 +124,7 @@ instructions in the [Security](clearml_server_security.md) page.
sudo curl https://raw.githubusercontent.com/allegroai/clearml-server/master/docker/docker-compose.yml -o /opt/clearml/docker-compose.yml
1. For Linux only, configure the **ClearML Agent Services**. If `CLEARML_HOST_IP` is not provided, then ClearML Agent Services uses the external public address of the **ClearML Server**. If `CLEARML_AGENT_GIT_USER` / `CLEARML_AGENT_GIT_PASS` are not provided, then **ClearML Agent Services** can't access any private repositories for running service tasks.
1. For Linux only, configure the **ClearML Agent Services**. If `CLEARML_HOST_IP` is not provided, then ClearML Agent Services uses the external public address of the ClearML Server. If `CLEARML_AGENT_GIT_USER` / `CLEARML_AGENT_GIT_PASS` are not provided, then ClearML Agent Services can't access any private repositories for running service tasks.
export CLEARML_HOST_IP=server_host_ip_here
export CLEARML_AGENT_GIT_USER=git_username_here

2
docs/faq.md
View File

@ -717,7 +717,7 @@ Yes! You can run ClearML in Jupyter Notebooks using either of the following:
1. Use the [Task.set_credentials](references/sdk/task.md#taskset_credentials)
method to specify the host, port, access key and secret key (see step 1).
```python
# Set your credentials using the trains apiserver URI and port, access_key, and secret_key.
# Set your credentials using the clearml apiserver URI and port, access_key, and secret_key.
Task.set_credentials(host='http://localhost:8008',key='<access_key>', secret='<secret_key>')
```

2
docs/guides/frameworks/pytorch/pytorch_distributed_example.md
View File

@ -46,7 +46,7 @@ The single scalar plot for loss appears in **RESULTS** **>** **SCALARS**.
## Hyperparameters
**ClearML** automatically logs the command line options defined using `argparse`.
ClearML automatically logs the command line options defined using `argparse`.
A parameter dictionary is logged by connecting it to the Task using a call to the [Task.connect](../../../references/sdk/task.md#connect)
method.

9
docs/guides/frameworks/pytorch/pytorch_matplotlib.md
View File

@ -6,10 +6,11 @@ The [pytorch_matplotlib.py](https://github.com/allegroai/clearml/blob/master/exa
example demonstrates the integration of **ClearML** into code that uses PyTorch and Matplotlib.
The example does the following:
1. The script calls Matplotlib methods to show images, each with a different title.
1. **ClearML** automatically logs the images as debug samples.
1. When the script runs, it creates an experiment named `pytorch with matplotlib example`, which is associated with the
`examples` project.
* Creates an experiment named `pytorch with matplotlib example`, in the `examples` project.
* The script calls Matplotlib methods to show images, each with a different title.
* ClearML automatically logs the images as debug samples.
## Debug Samples
The images shown in the example script's `imshow` function appear according to metric in **RESULTS** **>** **DEBUG SAMPLES**.

2
docs/guides/frameworks/pytorch/pytorch_tensorboardx.md
View File

@ -8,7 +8,7 @@ example demonstrates the integration of ClearML into code that uses PyTorch and
The example does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
dataset.
* Creates an experiment named `pytorch with tensorboardX`, which is associated with the `examples` project.
* Creates an experiment named `pytorch with tensorboardX` in the `examples` project.
* ClearML automatically captures scalars and text logged using the TensorBoardX `SummaryWriter` object, and
the model created by PyTorch.

4
docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md
View File

@ -3,7 +3,7 @@ title: PyTorch TensorBoard Toy
---
The [tensorboard_toy_pytorch.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/tensorboard_toy_pytorch.py)
example demonstrates the integration of **ClearML** into code, which creates a TensorBoard `SummaryWriter` object to log
example demonstrates the integration of ClearML into code, which creates a TensorBoard `SummaryWriter` object to log
debug sample images. When the script runs, it creates an experiment named `pytorch tensorboard toy example`, which is
associated with the `examples` project.
@ -16,6 +16,6 @@ The debug sample images appear according to metric, in the experiment page in th
## Hyperparameters
**ClearML** automatically logs TensorFlow Definitions. They appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **TF_DEFINE**.
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **TF_DEFINE**.
![image](../../../img/examples_tensorboard_toy_pytorch_00.png)

2
docs/guides/frameworks/scikit-learn/sklearn_joblib_example.md
View File

@ -5,7 +5,7 @@ title: scikit-learn with Joblib
The [sklearn_joblib_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py)
demonstrates the integration of ClearML into code that uses `scikit-learn` and `joblib` to store a model and model snapshots,
and `matplotlib` to create a scatter diagram. When the script runs, it creates an experiment named
`scikit-learn joblib examplescikit-learn joblib example`, which is associated with the `examples` project.
`scikit-learn joblib example`, which is associated with the `examples` project.
## Plots

6
docs/guides/frameworks/scikit-learn/sklearn_matplotlib_example.md
View File

@ -3,13 +3,13 @@ title: scikit-learn with Matplotlib
---
The [sklearn_matplotlib_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_matplotlib_example.py)
script demonstrates the integration of **ClearML** into code that uses `scikit-learn` and `matplotlib`.
script demonstrates the integration of ClearML into code that uses `scikit-learn` and `matplotlib`.
The example does the following:
* Uses `scikit-learn` to determine cross-validated training and test scores.
* Uses `matplotlib` to plot the learning curves.
* Through ClearML, automatically logs the scatter diagrams for the learning curves.
* Creates an experiment named `scikit-learn matplotlib example` which is associated with the `examples` project.
* ClearML automatically logs the scatter diagrams for the learning curves.
* Creates an experiment named `scikit-learn matplotlib example` in the `examples` project.
## Plots

8
docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md
View File

@ -3,16 +3,16 @@ title: TensorBoard PR Curve
---
The [tensorboard_pr_curve.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_pr_curve.py)
example demonstrates the integration of **ClearML** into code that uses TensorFlow and TensorBoard.
example demonstrates the integration of ClearML into code that uses TensorFlow and TensorBoard.
The example script does the following:
* Creates an experiment named `tensorboard pr_curve` in the `examples` project.
* Creates three classes, R, G, and B, and generates colors within the RGB space from normal distributions. The true
label of each random color is associated with the normal distribution that generated it.
* Computes the probability that each color belongs to the class, using three other normal distributions.
* Generate PR curves using those probabilities.
* Creates a summary per class using [tensorboard.plugins.pr_curve.summary](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/pr_curve/summary.py),
* Automatically logs the TensorBoard output, TensorFlow Definitions, and output to the console, using **ClearML**.
* When the script runs, Creates an experiment named `tensorboard pr_curve`, which is associated with the `examples` project.
* ClearML automatically captures TensorBoard output, TensorFlow Definitions, and output to the console
## Plots
@ -27,7 +27,7 @@ In the **ClearML Web UI**, the PR Curve summaries appear in the experiment's pag
## Hyperparameters
**ClearML** automatically logs TensorFlow Definitions. They appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **TF_DEFINE**.
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **TF_DEFINE**.
![image](../../../img/examples_tensorboard_pr_curve_04.png)

17
docs/guides/frameworks/tensorflow/tensorboard_toy.md
View File

@ -3,22 +3,15 @@ title: TensorBoard Toy
---
The [tensorboard_toy.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_toy.py)
example demonstrates **ClearML**'s automatic logging of TensorBoard scalars, histograms, images, and text, as well as
example demonstrates ClearML's automatic logging of TensorBoard scalars, histograms, images, and text, as well as
all other console output and TensorFlow Definitions.
The script uses `tf.summary.create_file_writer` with the following:
* `tf.summary.histogram`
* `tf.summary.scalar`
* `tf.summary.text`
* `tf.summary.image`
When the script runs, it creates an experiment named `tensorboard toy example`, which is associated with the `examples`
When the script runs, it creates an experiment named `tensorboard toy example` in the `examples`
project.
## Scalars
The `tf.summary.scalar` output appears in the experiment's page in the **ClearML web UI** under **RESULTS** **>**
The `tf.summary.scalar` output appears in the ClearML web UI, in the experiment's **RESULTS** **>**
**SCALARS**. Resource utilization plots, which are titled **:monitor: machine**, also appear in the **SCALARS** tab.
![image](../../../img/examples_tensorboard_toy_03.png)
@ -31,13 +24,13 @@ The `tf.summary.histogram` output appears in **RESULTS** **>** **PLOTS**.
## Debug Samples
**ClearML** automatically tracks images and text output to TensorFlow. They appear in **RESULTS** **>** **DEBUG SAMPLES**.
ClearML automatically tracks images and text output to TensorFlow. They appear in **RESULTS** **>** **DEBUG SAMPLES**.
![image](../../../img/examples_tensorboard_toy_05.png)
## Hyperparameters
**ClearML** automatically logs TensorFlow Definitions. They appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>**
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>**
**TF_DEFINE**.
![image](../../../img/examples_tensorboard_toy_01.png)

9
docs/guides/frameworks/tensorflow/tensorflow_mnist.md
View File

@ -6,14 +6,7 @@ The [tensorflow_mnist.py](https://github.com/allegroai/clearml/blob/master/examp
example demonstrates the integration of ClearML into code that uses TensorFlow and Keras to train a neural network on
the Keras built-in [MNIST](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/mnist) handwritten digits dataset.
The script builds a TensorFlow Keras model, and trains and tests it with the following:
* Loss objective function - [tf.keras.metrics.SparseCategoricalCrossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/SparseCategoricalCrossentropy)
* Accuracy metric - [tf.keras.metrics.SparseCategoricalAccuracy](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/SparseCategoricalAccuracy)
* Model checkpointing - [tf.clearml.Checkpoint](https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint?hl=ca) and [tf.train.CheckpointManager](https://www.tensorflow.org/api_docs/python/tf/train/CheckpointManager?hl=ca)
When the script runs, it creates an experiment named `Tensorflow v2 mnist with summaries`, which is associated with the
`examples` project.
When the script runs, it creates an experiment named `Tensorflow v2 mnist with summaries` in the `examples` project.
## Scalars