ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-03-03 18:53:37 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Small edits (#891)

Browse Source
This commit is contained in:
pollfly 2024-08-05 10:12:18 +03:00 committed by GitHub
parent 5353bd3cf6
commit e2cc07c509
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 37 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/clearml_agent/clearml_agent_env_var.md
View File

@ -22,7 +22,7 @@ but can be overridden by command-line arguments.
|**CLEARML_CUDA_VERSION** | Sets the CUDA version to be used | |**CLEARML_CUDA_VERSION** | Sets the CUDA version to be used |
|**CLEARML_CUDNN_VERSION** | Sets the CUDNN version to be used | |**CLEARML_CUDNN_VERSION** | Sets the CUDNN version to be used |
|**CLEARML_CPU_ONLY** | Force CPU only mode | |**CLEARML_CPU_ONLY** | Force CPU only mode |
|**CLEARML_DOCKER_SKIP_GPUS_FLAG** | Skips the GPUs flag (support for docker V18 | |**CLEARML_DOCKER_SKIP_GPUS_FLAG** | Skips the GPUs flag (support for docker V18) |
|**CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV** | Hide Docker environment variables containing secrets when printing out the Docker command. When printed, the variable values will be replaced by `********`. See [`agent.hide_docker_command_env_vars`](../configs/clearml_conf.md#hide_docker) | |**CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV** | Hide Docker environment variables containing secrets when printing out the Docker command. When printed, the variable values will be replaced by `********`. See [`agent.hide_docker_command_env_vars`](../configs/clearml_conf.md#hide_docker) |
|**CLEARML_AGENT_DISABLE_SSH_MOUNT** | Disables the auto `.ssh` mount into the docker | |**CLEARML_AGENT_DISABLE_SSH_MOUNT** | Disables the auto `.ssh` mount into the docker |
|**CLEARML_AGENT_FORCE_CODE_DIR**| Allows overriding the remote execution code directory to bypass repository cloning and use a repo already available where the remote agent is running. | |**CLEARML_AGENT_FORCE_CODE_DIR**| Allows overriding the remote execution code directory to bypass repository cloning and use a repo already available where the remote agent is running. |

2
docs/clearml_data/data_management_examples/data_man_cifar_classification.md
View File

@ -10,7 +10,7 @@ class to ingest the data.
### Downloading the Data ### Downloading the Data
Before registering the CIFAR dataset with `clearml-data`, you need to obtain a local copy of it. Before registering the CIFAR dataset with `clearml-data`, you need to obtain a local copy of it.
Execute this python script to download the data Execute this python script to download the data:
```python ```python
from clearml import StorageManager from clearml import StorageManager

4
docs/configs/clearml_conf.md
View File

@ -128,7 +128,7 @@ Use with care! This might introduce security risks by allowing access to keys/se
--- ---
**`docker_args_extra_precedes_task`** (*bool*) **`agent.docker_args_extra_precedes_task`** (*bool*)
* Allow the arguments specified in `agent.extra_docker_arguments` to override task level docker arguments, in the case that * Allow the arguments specified in `agent.extra_docker_arguments` to override task level docker arguments, in the case that
the same argument is passed in both. If set to `False`, a task's docker arguments will override the `extra_docker_arguments`. the same argument is passed in both. If set to `False`, a task's docker arguments will override the `extra_docker_arguments`.
@ -1039,6 +1039,8 @@ and limitations on bucket naming.
--- ---
<a id="log_env_var"/>
**`sdk.development.log_os_environments`** (*[string]*) **`sdk.development.log_os_environments`** (*[string]*)
* Log specific environment variables. OS environments are listed in the UI, under an experiment's * Log specific environment variables. OS environments are listed in the UI, under an experiment's

2
docs/faq.md
View File

@ -855,7 +855,7 @@ export CLEARML_API_HOST="http://localhost:8008"
#### How can I track OS environment variables with experiments? <a id="track-env-vars"></a> #### How can I track OS environment variables with experiments? <a id="track-env-vars"></a>
You can set environment variables to track in an experiment by specifying them in the `sdk.development.log_os_environments` You can set environment variables to track in an experiment by specifying them in the `sdk.development.log_os_environments`
field of the [`clearml.conf`](configs/clearml_conf.md) file: field of the [`clearml.conf`](configs/clearml_conf.md#log_env_var) file:
```editorconfig ```editorconfig
log_os_environments: ["AWS_*", "CUDA_VERSION"] log_os_environments: ["AWS_*", "CUDA_VERSION"]

2
docs/fundamentals/agents_and_queues.md
View File

@ -1,5 +1,5 @@
--- ---
title: Workers & Queues title: Workers and Queues
--- ---
Two major components of MLOps/LLMOps are experiment reproducibility, and the ability to scale work to multiple machines. ClearML workers, Two major components of MLOps/LLMOps are experiment reproducibility, and the ability to scale work to multiple machines. ClearML workers,

2
docs/fundamentals/hyperparameters.md
View File

@ -61,7 +61,7 @@ You can also specify environment variables using the `CLEARML_LOG_ENVIRONMENT` e
export CLEARML_LOG_ENVIRONMENT=PWD,PYTHONPATH export CLEARML_LOG_ENVIRONMENT=PWD,PYTHONPATH
``` ```
* No environment variables * No environment variables:
``` ```
export CLEARML_LOG_ENVIRONMENT= export CLEARML_LOG_ENVIRONMENT=
``` ```

1
docs/guides/frameworks/megengine/megengine_mnist.md
View File

@ -50,5 +50,6 @@ The model info panel contains the model details, including:
## Console ## Console
All console output during the script's execution appears in the experiment's **CONSOLE** page. All console output during the script's execution appears in the experiment's **CONSOLE** page.
![Console tab](../../../img/examples_megengine_console.png) ![Console tab](../../../img/examples_megengine_console.png)

2
docs/guides/frameworks/pytorch/pytorch_mnist.md
View File

@ -9,7 +9,7 @@ The example script does the following:
* Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist) * Trains a simple deep neural network on the PyTorch built-in [MNIST](https://pytorch.org/vision/stable/datasets.html#mnist)
dataset. dataset.
* Creates an experiment named `pytorch mnist train` in the `examples` project. * Creates an experiment named `pytorch mnist train` in the `examples` project.
* ClearML automatically logs `argparse` command line options, and models (and their snapshots) created by PyTorch * ClearML automatically logs `argparse` command line options, and models (and their snapshots) created by PyTorch.
* Additional metrics are logged by calling [`Logger.report_scalar()`](../../../references/sdk/logger.md#report_scalar). * Additional metrics are logged by calling [`Logger.report_scalar()`](../../../references/sdk/logger.md#report_scalar).
## Scalars ## Scalars

4
docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md
View File

@ -11,8 +11,8 @@ The example script does the following:
label of each random color is associated with the normal distribution that generated it. label of each random color is associated with the normal distribution that generated it.
* Computes the probability that each color belongs to the class, using three other normal distributions. * Computes the probability that each color belongs to the class, using three other normal distributions.
* Generate PR curves using those probabilities. * Generate PR curves using those probabilities.
* Creates a summary per class using [tensorboard.plugins.pr_curve.summary](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/pr_curve/summary.py), * Creates a summary per class using [tensorboard.plugins.pr_curve.summary](https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/pr_curve/summary.py).
* ClearML automatically captures TensorBoard output, TensorFlow Definitions, and output to the console * ClearML automatically captures TensorBoard output, TensorFlow Definitions, and output to the console.
## Plots ## Plots

2
docs/guides/ide/remote_jupyter_tutorial.md
View File

@ -85,7 +85,7 @@ Jupyter Lab URL: http://localhost:8878/?token=ff7e5e8b9e5493a01b1a72530d18181320
VSCode server available at http://localhost:8898/ VSCode server available at http://localhost:8898/
``` ```
Click on the JupyterLab link, which will open the remote session Click on the JupyterLab link, which will open the remote session.
Now, let's execute some code in the remote session! Now, let's execute some code in the remote session!

16
docs/guides/reporting/media_reporting.md
View File

@ -3,15 +3,16 @@ title: Media Reporting
--- ---
The [media_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/media_reporting.py) example The [media_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/media_reporting.py) example
demonstrates reporting (uploading) images, audio, and video. Use the [Logger.report_media](../../references/sdk/logger.md#report_media) demonstrates reporting (uploading) images, audio, and video. Use [`Logger.report_media()`](../../references/sdk/logger.md#report_media)
method to upload from: to upload from:
* Local path * Local path
* BytesIO stream * BytesIO stream
* URL of media already uploaded to some storage * URL of media already uploaded to some storage
ClearML uploads media to the bucket specified in the ClearML configuration file or ClearML can be configured for image storage, see [Logger.set_default_upload_destination](../../references/sdk/logger.md#set_default_upload_destination) ClearML uploads media to the bucket specified in the ClearML configuration file. You can configure ClearML for image
(storage for [artifacts](../../clearml_sdk/task_sdk.md#setting-upload-destination) is different). Set credentials for storage in the ClearML storage using [`Logger.set_default_upload_destination()`](../../references/sdk/logger.md#set_default_upload_destination)
[configuration file](../../configs/clearml_conf.md). (note that [artifact storage](../../clearml_sdk/task_sdk.md#setting-upload-destination) is handled differently).
Set the storage credentials in the [clearml.conf file](../../configs/clearml_conf.md#sdk-section).
ClearML reports media in the **ClearML Web UI** **>** experiment details **>** **DEBUG SAMPLES** ClearML reports media in the **ClearML Web UI** **>** experiment details **>** **DEBUG SAMPLES**
tab. tab.
@ -21,8 +22,7 @@ project.
## Reporting (Uploading) Media from a Source by URL ## Reporting (Uploading) Media from a Source by URL
Report by calling the [Logger.report_media](../../references/sdk/logger.md#report_media) Report by using the `url` parameter of [`Logger.report_media()`](../../references/sdk/logger.md#report_media):
method using the `url` parameter.
```python ```python
# report video, an already uploaded video media (url) # report video, an already uploaded video media (url)
@ -45,7 +45,7 @@ The reported audio can be viewed in the **DEBUG SAMPLES** tab. Click a thumbnail
## Reporting (Uploading) Media from a Local File ## Reporting (Uploading) Media from a Local File
Use the `local_path` parameter. Report by using the `local_path` parameter of [`Logger.report_media()`](../../references/sdk/logger.md#report_media):
```python ```python
# report audio, report local media audio file # report audio, report local media audio file

7
docs/guides/reporting/text_reporting.md
View File

@ -3,14 +3,15 @@ title: Text Reporting
--- ---
The [text_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py) script The [text_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py) script
demonstrates reporting explicit text, by calling the [Logger.report_text](../../references/sdk/logger.md#report_text) demonstrates reporting explicit text by calling [`Logger.report_text()`](../../references/sdk/logger.md#report_text).
method.
ClearML reports these tables in the **ClearML Web UI**, experiment details, **CONSOLE** tab. ClearML reports the text in the **ClearML Web UI**, in the experiment's **CONSOLE** tab.
When the script runs, it creates an experiment named `text reporting` in the `examples` project. When the script runs, it creates an experiment named `text reporting` in the `examples` project.
```python
# report text # report text
Logger.current_logger().report_text("hello, this is plain text") Logger.current_logger().report_text("hello, this is plain text")
```
![image](../../img/examples_reporting_text.png) ![image](../../img/examples_reporting_text.png)

14
docs/guides/ui/tuning_exp.md
View File

@ -16,7 +16,9 @@ example script.
In the `examples/frameworks/pytorch` directory, run the experiment script: In the `examples/frameworks/pytorch` directory, run the experiment script:
```commandline
python pytorch_mnist.py python pytorch_mnist.py
```
## Step 2: Clone the Experiment ## Step 2: Clone the Experiment
@ -42,15 +44,12 @@ To demonstrate tuning, change two hyperparameter values.
## Step 4: Run a Worker Daemon Listening to a Queue ## Step 4: Run a Worker Daemon Listening to a Queue
To execute the cloned experiment, use a worker that can run a worker daemon listening to a queue. To execute the cloned experiment, use a [ClearML Agent](../../fundamentals/agents_and_queues.md).
:::note Run the agent on the local development machine:
For more information about workers, worker daemons, and queues, see [Agents and queues](../../fundamentals/agents_and_queues.md).
:::
Run the worker daemon on the local development machine.
1. Open a terminal session. 1. Open a terminal session.
1. Run the following `clearml-agent` command which runs a worker daemon listening to the `default` queue: 1. Run the following `clearml-agent` command which runs a worker daemon listening to the `default` queue:
``` ```
clearml-agent daemon --queue default clearml-agent daemon --queue default
``` ```
@ -119,5 +118,4 @@ To compare the original and tuned experiments:
## Next Steps ## Next Steps
* For more information about editing experiments, see [modify experiments](../../webapp/webapp_exp_tuning.md#modifying-experiments) * For more information about editing experiments, see [modifying experiments](../../webapp/webapp_exp_tuning.md#modifying-experiments).
in the User Interface section.

2
docs/webapp/webapp_exp_track_visual.md
View File

@ -132,7 +132,7 @@ parameter's line, and the type, description, and default value appear, if they w
#### Environment Variables #### Environment Variables
If environment variables were listed in the `CLEARML_LOG_ENVIRONMENT` environment variable or the `sdk.development.log_os_environments` If environment variables were listed in the `CLEARML_LOG_ENVIRONMENT` environment variable or the [`sdk.development.log_os_environments`](../configs/clearml_conf.md#log_env_var)
field of the `clearml.conf` file, the **Environment** group displays the listed environment variables (see [this FAQ](../faq.md#track-env-vars)). field of the `clearml.conf` file, the **Environment** group displays the listed environment variables (see [this FAQ](../faq.md#track-env-vars)).
:::note :::note

4
docs/webapp/webapp_exp_tuning.md
View File

@ -127,7 +127,7 @@ Add, change, or delete hyperparameters, which are organized in the **ClearML Web
* **General** - Parameter dictionaries (from code, connected to the Task by calling [`Task.connect()`](../references/sdk/task.md#connect)). * **General** - Parameter dictionaries (from code, connected to the Task by calling [`Task.connect()`](../references/sdk/task.md#connect)).
* Environment variables - Tracked if variables were listed in the `CLEARML_LOG_ENVIRONMENT` environment variable * Environment variables - Tracked if variables were listed in the `CLEARML_LOG_ENVIRONMENT` environment variable
or the `sdk.development.log_os_environments` field of the `clearml.conf` file (see this [FAQ](../faq.md#track-env-vars)). or the [`sdk.development.log_os_environments`](../configs/clearml_conf.md#log_env_var) field of the `clearml.conf` file (see this [FAQ](../faq.md#track-env-vars)).
* Custom named parameter groups (see the `name` parameter in [`Task.connect`](../references/sdk/task.md#connect)). * Custom named parameter groups (see the `name` parameter in [`Task.connect`](../references/sdk/task.md#connect)).
@ -178,7 +178,7 @@ model in the **MODELS** tab.
**To edit a model's configuration or label enumeration:** **To edit a model's configuration or label enumeration:**
1. Click the model name hyperlink. The model details appear in the **MODELS** tab. 1. Click the model name hyperlink. The model details appear in the **MODELS** tab.
1. Edit the model configuration or label enumeration. 1. Edit the model configuration or label enumeration:
* Model configuration - In the **NETWORK** tab **>** Hover and click **EDIT**. **>** CLick **EDIT** or **CLEAR** (to * Model configuration - In the **NETWORK** tab **>** Hover and click **EDIT**. **>** CLick **EDIT** or **CLEAR** (to
remove the configuration). remove the configuration).