ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-03-19 19:49:03 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/allegroai/clearml-docs

Browse Source
This commit is contained in:
revital 2025-02-06 15:38:50 +02:00
commit c41e83a61a
14 changed files with 50 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/guides/ide/google_colab.md
View File

@ -30,7 +30,7 @@ and running, users can send Tasks to be executed on Google Colab's hardware.
This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.
1. Create new credentials. Go to your [**Settings**](https://app.clear.ml/settings/workspace-configuration) page > **WORKSPACE** section.
Under **App Credentials**, click **+ Create new credentials**, and copy the information that pops up.
Under **API Credentials**, click **+ Create new credentials**, and copy the information that pops up.
1. Set the credentials. In the third cell, enter your own credentials:
```python

34
docs/hyperdatasets/dataviews.md
View File

@ -71,40 +71,6 @@ ROI label mapping (label translation) applies to the new model. For example, app
Define class labels for the new model and assign integers to each in order to maintain data conformity across multiple
codebases and datasets. It is important to set enumeration values for all labels of importance.
## Data Augmentation
On-the-fly data augmentation is applied to SingleFrames, transforming images without creating new data. Apply data augmentation
in steps, where each step is composed of a method, an operation, and a strength as follows:
* **Affine** augmentation method - Transform an image's geometric shape to another position on a 2-dimensional plane.
Use any of the following operations:
* Rotate
* Reflect-horiz - Flip images horizontally
* Reflect-vert - Flip images vertically
* Scale
* Shear - Skew
* No operation - Randomly select SingleFrames that are not transformed (skipped). If the experiment runs again, and
the random seed in [iteration control](#iteration-control) is unchanged, the same SingleFrames are not augmented.
* **Pixel** augmentation method - Transform images by modifying pixel values while retaining shape and perspective.
Use any of the following operations:
* Blur - Gaussian smoothing
* Noise - ClearML Enterprise's own noise augmentation consisting of:
* **high** noise - like snow on analog televisions with a weak TV signal
* **low** noise - like a low resolution image magnified in localized areas on the image
* Recolor - using an internal RGB lookup-table
* No operation - Randomly select SingleFrames that are not transformed (skipped). If the experiment runs again, and
the random seed in [iteration control](#iteration-control) is unchanged, the same SingleFrames are not augmented.
* Strength - A number applied to adjust the degree of transformation. The recommended strengths are the following:
* 0.0 - No effect
* 0.5 - Low (weak)
* 1.0 - Medium (recommended)
* 2.0 - High (strong)
## Iteration Control
The input data **iteration control** settings determine the order, number, timing, and reproducibility of the Dataview iterating

25
docs/hyperdatasets/webapp/webapp_datasets_frames.md
View File

@ -159,11 +159,32 @@ To draw a new annotation:
A new annotation is created.
1. In the newly created annotation, select or type-in a label(s). Click the circle in the label name to select a
different label color.
1. In the newly created annotation, select or type-in the labels for this annotation.
You can use the **Default ROI Label(s)** list to automatically set labels to all new annotations.
:::info Annotation color
Each annotation label is automatically assigned a color based on its value. The annotation color is automatically
calculated based on the colors of its labels.
Click the color circle in the label name to manually set the label's color.
<div class="max-w-75">
![Set label color](../../img/annotation_label_color.png)
</div>
Click the color circle in the annotation header to manually set the annotation’s color and its opacity.
<div class="max-w-75">
![Set annotation color and opacity](../../img/annotation_label_opacity.png)
</div>
:::
#### Copying Frame Objects
You can copy existing annotations, and paste them to any frame of your choice:
1. Click the annotation or bounded area in the frame.

13
docs/hyperdatasets/webapp/webapp_datasets_versioning.md
View File

@ -8,16 +8,19 @@ deleting [Dataset versions](../dataset.md#dataset-versioning).
![Dataset versions page](../../img/hyperdatasets/web-app/dataset_versions.png)
## Dataset Version History
The WebApp (UI) presents your dataset version structure in list view <img src="/docs/latest/icons/ico-list-view.svg" alt="List view" className="icon size-md space-sm" />
or tree view <img src="/docs/latest/icons/ico-tree-view.svg" alt="Tree view" className="icon size-md space-sm" />.
The WebApp (UI) presents your dataset version structure in tree view <img src="/docs/latest/icons/ico-tree-view.svg" alt="Tree view" className="icon size-md space-sm" />
or list view <img src="/docs/latest/icons/ico-list-view.svg" alt="List view" className="icon size-md space-sm" />.
The tree view shows the lineage of the dataset's versions.
![Versions tree view](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
The list view lists the dataset's versions chronologically by last update time.
![Versions list view](../../img/hyperdatasets/web-app/dataset_simple_adv_01.png)
The tree view shows the lineage of the dataset's versions.
![Versions tree view](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
Click <img src="/docs/latest/icons/ico-sort.svg" alt="Sort order" className="icon size-md space-sm" /> to order the
dataset versions in ascending or descending order based on their last update time.
Use the search bar to find specific versions. You can query by version name, version description, or version ID. The search returns
all versions that match the query.

2
docs/hyperdatasets/webapp/webapp_dataviews.md
View File

@ -78,7 +78,7 @@ Access these actions with the context menu in any of the following ways:
| ClearML Action | Description |
|---|---|
| Details | View Dataview details, including input datasets, label mapping, augmentation operations, and iteration control. Can also be accessed by double-clicking a Dataview in the Dataviews table. |
| Details | View Dataview details, including input datasets, label mapping, and iteration control. Can also be accessed by double-clicking a Dataview in the Dataviews table. |
| Archive | Move Dataview to the Dataview's archive. |
| Restore | Action available in the archive. Restore a Dataview to the active Dataviews table. |
| Delete | Action available in the archive. Permanently delete a Dataview. |

19
docs/hyperdatasets/webapp/webapp_exp_modifying.md
View File

@ -12,7 +12,6 @@ enables modifying [Dataviews](webapp_dataviews.md), including:
* [Frame filtering](#filtering-frames)
* [Label mapping](#mapping-labels-label-translation)
* [Class label enumeration](#label-enumeration)
* [Data augmentation](#data-augmentation)
* [Input frame iteration controls](#iteration-controls)
## Selecting Dataviews
@ -22,7 +21,7 @@ enables modifying [Dataviews](webapp_dataviews.md), including:
* Create a new Dataview
* Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation),
and set label enumeration, data augmentation, and iteration controls.
and set label enumeration and iteration controls.
* Select a different Dataview already associated with the experiment.
@ -118,22 +117,6 @@ Modify the label enumeration assigned to output models.
1. Click **SAVE**.
## Data Augmentation
Modify the on-the-fly data augmentation applied to frame input from the select Hyper-Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
**To modify data augmentation**
1. In the **AUGMENTATION** section, click **EDIT**.
* Add (**+**) or edit an augmentation step - Select a **METHOD**, **OPERATION**, and **STRENGTH**.
* Remove (<img src="/docs/latest/icons/ico-trash.svg" alt="Trash" className="icon size-md space-sm" />) an augmentation step.
1. Click **SAVE**.
## Iteration Controls
Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames

8
docs/hyperdatasets/webapp/webapp_exp_track_visual.md
View File

@ -14,7 +14,6 @@ In an experiment's page, go to the **DATAVIEWS** tab to view all the experiment'
* Input data [selection](#input) and [filtering](#filtering)
* ROI [mapping](#mapping) (label translation)
* [Label enumeration](#label-enumeration)
* On-the-fly [data augmentation](#augmentation)
* [Iteration controls](#iteration-control)
![Dataview tab](../../img/hyperdatasets/web-app/dataview_tab.png)
@ -52,13 +51,6 @@ For detailed information, see [Mapping ROI labels](../dataviews.md#mapping-roi-l
Assign label enumeration in the **LABELS ENUMERATION** area.
### Augmentation
On-the-fly data augmentation applied to SingleFrames, which does not create new data. Apply data Augmentation in steps,
where each step is composed of a method, an operation, and a strength.
For detailed information, see [Data augmentation](../dataviews.md#data-augmentation).
### Iteration Control
The input data iteration control settings determine the order, number, timing, and reproducibility of the Dataview iterating

BIN
docs/img/annotation_label_color.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
docs/img/annotation_label_opacity.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

2
docs/webapp/settings/webapp_settings_overview.md
View File

@ -25,5 +25,5 @@ The Settings page consists of the following sections:
* [Resource Configuration](webapp_settings_resource_configs.md) (ClearML Enterprise Server) - Define the available resources and the way in which they
will be allocated to different workloads
* [Billing & Usage](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current billing details and usage information
* [Storage Credentials](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to
* [Storage Cleanup](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to
enable ClearML to delete artifacts stored in cloud storage when tasks and models are deleted

6
docs/webapp/settings/webapp_settings_profile.md
View File

@ -63,7 +63,7 @@ to switch to.
![Workspace configuration page](../../img/settings_workspace_configuration.png#light-mode-only)
![Workspace configuration page](../../img/settings_workspace_configuration_dark.png#dark-mode-only)
### ClearML App Credentials
### ClearML API Credentials
Generate ClearML credentials, made up of an access and secret key pair, and insert them into your [configuration file](../../configs/clearml_conf.md)
or Jupyter Notebook to grant the ClearML SDK and the ClearML Agent API access to the server.
@ -74,7 +74,7 @@ You can create credentials for any workspace that you are a member of.
1. In **WORKSPACE**, expand the desired workspace's panel (self-deployed ClearML Server users have one workspace)
1. In **App Credentials**, click **+ Create new credentials**
1. In **API Credentials**, click **+ Create new credentials**
1. In the dialog that pops up, you can input a label for the new credentials
@ -114,7 +114,7 @@ Generate tokens providing API access to the AI Application Gateway endpoints:
### Changing Your Workspace Name
To change the name of your own workspace, click **Edit workspace name** <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" />
(under App credentials) **>** modify the name **>** click <img src="/docs/latest/icons/ico-save.svg" alt="Check Mark" className="icon size-md" />.
(under API credentials) **>** modify the name **>** click <img src="/docs/latest/icons/ico-save.svg" alt="Check Mark" className="icon size-md" />.
### Adding Users to Your Workspace

6
docs/webapp/settings/webapp_settings_storage_credentials.md
View File

@ -1,5 +1,5 @@
---
title: Storage Credentials
title: Storage Cleanup
---
To enable ClearML to delete task artifacts stored in cloud storage when a task is deleted, configure access credentials for your storage provider:
@ -7,8 +7,8 @@ To enable ClearML to delete task artifacts stored in cloud storage when a task i
* [AWS S3 Storage](#aws-s3-storage)
* [Azure](#azure)
![Storage Credentials page](../../img/webapp_settings_storage_credentials.png#light-mode-only)
![Storage Credentials page](../../img/webapp_settings_storage_credentials_dark.png#dark-mode-only)
![Storage Cleanup page](../../img/webapp_settings_storage_credentials.png#light-mode-only)
![Storage Cleanup page](../../img/webapp_settings_storage_credentials_dark.png#dark-mode-only)
## Google Cloud Storage
Set up credentials for Google Cloud buckets:

8
docs/webapp/settings/webapp_settings_users.md
View File

@ -19,6 +19,9 @@ Each row of the table includes:
* If the user's invitation is pending, the date the user was added
* [User groups](#user-groups) (ClearML Enterprise feature)
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
to find a specific user by their name, email, or user ID.
![Users table](../../img/settings_user_table.png#light-mode-only)
![Users table](../../img/settings_user_table_dark.png#dark-mode-only)
@ -68,6 +71,9 @@ Each row of the table includes:
* Credentials - Number of credentials currently available to the account
* Last active time
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
to find a specific service account by its name or ID.
Hover over a service account in the table to **Edit** or **Delete** it.
![Service accounts](../../img/settings_service_accounts.png#light-mode-only)
@ -163,6 +169,8 @@ The system includes three pre-configured groups that can't be removed:
* `Queue admins` - Can create / delete / rename queues
The user group table lists all the active user groups. Each row includes a group's name, description, member list, and ID.
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
to find a specific user group by its name, description, or ID.
![User Management Page](../../img/settings_user_management_table.png#light-mode-only)
![User Management Page](../../img/settings_user_management_table_dark.png#dark-mode-only)

2
docs/webapp/webapp_overview.md
View File

@ -37,7 +37,7 @@ Click the profile menu button <img src="/docs/latest/icons/ico-me.svg" alt="Prof
in the top right corner of the web UI screen to access the following:
* **Settings** - Navigate to ClearML's user [Settings](settings/webapp_settings_profile.md) page:
* Set [WebApp preferences](settings/webapp_settings_profile.md)
* Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-credentials)
* Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-api-credentials)
* Configure [cloud storage access credentials](settings/webapp_settings_profile.md#browser-cloud-storage-access) for the ClearML Web UI
* ClearML Hosted service specific options
* Manage [users and workspaces](settings/webapp_settings_users.md)