This commit is contained in:
revital 2025-02-06 15:38:50 +02:00
commit c41e83a61a
14 changed files with 50 additions and 77 deletions

View File

@ -30,7 +30,7 @@ and running, users can send Tasks to be executed on Google Colab's hardware.
This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen. This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.
1. Create new credentials. Go to your [**Settings**](https://app.clear.ml/settings/workspace-configuration) page > **WORKSPACE** section. 1. Create new credentials. Go to your [**Settings**](https://app.clear.ml/settings/workspace-configuration) page > **WORKSPACE** section.
Under **App Credentials**, click **+ Create new credentials**, and copy the information that pops up. Under **API Credentials**, click **+ Create new credentials**, and copy the information that pops up.
1. Set the credentials. In the third cell, enter your own credentials: 1. Set the credentials. In the third cell, enter your own credentials:
```python ```python

View File

@ -71,40 +71,6 @@ ROI label mapping (label translation) applies to the new model. For example, app
Define class labels for the new model and assign integers to each in order to maintain data conformity across multiple Define class labels for the new model and assign integers to each in order to maintain data conformity across multiple
codebases and datasets. It is important to set enumeration values for all labels of importance. codebases and datasets. It is important to set enumeration values for all labels of importance.
## Data Augmentation
On-the-fly data augmentation is applied to SingleFrames, transforming images without creating new data. Apply data augmentation
in steps, where each step is composed of a method, an operation, and a strength as follows:
* **Affine** augmentation method - Transform an image's geometric shape to another position on a 2-dimensional plane.
Use any of the following operations:
* Rotate
* Reflect-horiz - Flip images horizontally
* Reflect-vert - Flip images vertically
* Scale
* Shear - Skew
* No operation - Randomly select SingleFrames that are not transformed (skipped). If the experiment runs again, and
the random seed in [iteration control](#iteration-control) is unchanged, the same SingleFrames are not augmented.
* **Pixel** augmentation method - Transform images by modifying pixel values while retaining shape and perspective.
Use any of the following operations:
* Blur - Gaussian smoothing
* Noise - ClearML Enterprise's own noise augmentation consisting of:
* **high** noise - like snow on analog televisions with a weak TV signal
* **low** noise - like a low resolution image magnified in localized areas on the image
* Recolor - using an internal RGB lookup-table
* No operation - Randomly select SingleFrames that are not transformed (skipped). If the experiment runs again, and
the random seed in [iteration control](#iteration-control) is unchanged, the same SingleFrames are not augmented.
* Strength - A number applied to adjust the degree of transformation. The recommended strengths are the following:
* 0.0 - No effect
* 0.5 - Low (weak)
* 1.0 - Medium (recommended)
* 2.0 - High (strong)
## Iteration Control ## Iteration Control
The input data **iteration control** settings determine the order, number, timing, and reproducibility of the Dataview iterating The input data **iteration control** settings determine the order, number, timing, and reproducibility of the Dataview iterating

View File

@ -159,11 +159,32 @@ To draw a new annotation:
A new annotation is created. A new annotation is created.
1. In the newly created annotation, select or type-in a label(s). Click the circle in the label name to select a 1. In the newly created annotation, select or type-in the labels for this annotation.
different label color.
You can use the **Default ROI Label(s)** list to automatically set labels to all new annotations. You can use the **Default ROI Label(s)** list to automatically set labels to all new annotations.
:::info Annotation color
Each annotation label is automatically assigned a color based on its value. The annotation color is automatically
calculated based on the colors of its labels.
Click the color circle in the label name to manually set the label's color.
<div class="max-w-75">
![Set label color](../../img/annotation_label_color.png)
</div>
Click the color circle in the annotation header to manually set the annotations color and its opacity.
<div class="max-w-75">
![Set annotation color and opacity](../../img/annotation_label_opacity.png)
</div>
:::
#### Copying Frame Objects #### Copying Frame Objects
You can copy existing annotations, and paste them to any frame of your choice: You can copy existing annotations, and paste them to any frame of your choice:
1. Click the annotation or bounded area in the frame. 1. Click the annotation or bounded area in the frame.

View File

@ -8,16 +8,19 @@ deleting [Dataset versions](../dataset.md#dataset-versioning).
![Dataset versions page](../../img/hyperdatasets/web-app/dataset_versions.png) ![Dataset versions page](../../img/hyperdatasets/web-app/dataset_versions.png)
## Dataset Version History ## Dataset Version History
The WebApp (UI) presents your dataset version structure in list view <img src="/docs/latest/icons/ico-list-view.svg" alt="List view" className="icon size-md space-sm" /> The WebApp (UI) presents your dataset version structure in tree view <img src="/docs/latest/icons/ico-tree-view.svg" alt="Tree view" className="icon size-md space-sm" />
or tree view <img src="/docs/latest/icons/ico-tree-view.svg" alt="Tree view" className="icon size-md space-sm" />. or list view <img src="/docs/latest/icons/ico-list-view.svg" alt="List view" className="icon size-md space-sm" />.
The tree view shows the lineage of the dataset's versions.
![Versions tree view](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
The list view lists the dataset's versions chronologically by last update time. The list view lists the dataset's versions chronologically by last update time.
![Versions list view](../../img/hyperdatasets/web-app/dataset_simple_adv_01.png) ![Versions list view](../../img/hyperdatasets/web-app/dataset_simple_adv_01.png)
The tree view shows the lineage of the dataset's versions. Click <img src="/docs/latest/icons/ico-sort.svg" alt="Sort order" className="icon size-md space-sm" /> to order the
dataset versions in ascending or descending order based on their last update time.
![Versions tree view](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
Use the search bar to find specific versions. You can query by version name, version description, or version ID. The search returns Use the search bar to find specific versions. You can query by version name, version description, or version ID. The search returns
all versions that match the query. all versions that match the query.

View File

@ -78,7 +78,7 @@ Access these actions with the context menu in any of the following ways:
| ClearML Action | Description | | ClearML Action | Description |
|---|---| |---|---|
| Details | View Dataview details, including input datasets, label mapping, augmentation operations, and iteration control. Can also be accessed by double-clicking a Dataview in the Dataviews table. | | Details | View Dataview details, including input datasets, label mapping, and iteration control. Can also be accessed by double-clicking a Dataview in the Dataviews table. |
| Archive | Move Dataview to the Dataview's archive. | | Archive | Move Dataview to the Dataview's archive. |
| Restore | Action available in the archive. Restore a Dataview to the active Dataviews table. | | Restore | Action available in the archive. Restore a Dataview to the active Dataviews table. |
| Delete | Action available in the archive. Permanently delete a Dataview. | | Delete | Action available in the archive. Permanently delete a Dataview. |

View File

@ -12,7 +12,6 @@ enables modifying [Dataviews](webapp_dataviews.md), including:
* [Frame filtering](#filtering-frames) * [Frame filtering](#filtering-frames)
* [Label mapping](#mapping-labels-label-translation) * [Label mapping](#mapping-labels-label-translation)
* [Class label enumeration](#label-enumeration) * [Class label enumeration](#label-enumeration)
* [Data augmentation](#data-augmentation)
* [Input frame iteration controls](#iteration-controls) * [Input frame iteration controls](#iteration-controls)
## Selecting Dataviews ## Selecting Dataviews
@ -22,7 +21,7 @@ enables modifying [Dataviews](webapp_dataviews.md), including:
* Create a new Dataview * Create a new Dataview
* Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation), * Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation),
and set label enumeration, data augmentation, and iteration controls. and set label enumeration and iteration controls.
* Select a different Dataview already associated with the experiment. * Select a different Dataview already associated with the experiment.
@ -118,22 +117,6 @@ Modify the label enumeration assigned to output models.
1. Click **SAVE**. 1. Click **SAVE**.
## Data Augmentation
Modify the on-the-fly data augmentation applied to frame input from the select Hyper-Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
**To modify data augmentation**
1. In the **AUGMENTATION** section, click **EDIT**.
* Add (**+**) or edit an augmentation step - Select a **METHOD**, **OPERATION**, and **STRENGTH**.
* Remove (<img src="/docs/latest/icons/ico-trash.svg" alt="Trash" className="icon size-md space-sm" />) an augmentation step.
1. Click **SAVE**.
## Iteration Controls ## Iteration Controls
Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames

View File

@ -14,7 +14,6 @@ In an experiment's page, go to the **DATAVIEWS** tab to view all the experiment'
* Input data [selection](#input) and [filtering](#filtering) * Input data [selection](#input) and [filtering](#filtering)
* ROI [mapping](#mapping) (label translation) * ROI [mapping](#mapping) (label translation)
* [Label enumeration](#label-enumeration) * [Label enumeration](#label-enumeration)
* On-the-fly [data augmentation](#augmentation)
* [Iteration controls](#iteration-control) * [Iteration controls](#iteration-control)
![Dataview tab](../../img/hyperdatasets/web-app/dataview_tab.png) ![Dataview tab](../../img/hyperdatasets/web-app/dataview_tab.png)
@ -52,13 +51,6 @@ For detailed information, see [Mapping ROI labels](../dataviews.md#mapping-roi-l
Assign label enumeration in the **LABELS ENUMERATION** area. Assign label enumeration in the **LABELS ENUMERATION** area.
### Augmentation
On-the-fly data augmentation applied to SingleFrames, which does not create new data. Apply data Augmentation in steps,
where each step is composed of a method, an operation, and a strength.
For detailed information, see [Data augmentation](../dataviews.md#data-augmentation).
### Iteration Control ### Iteration Control
The input data iteration control settings determine the order, number, timing, and reproducibility of the Dataview iterating The input data iteration control settings determine the order, number, timing, and reproducibility of the Dataview iterating

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

View File

@ -25,5 +25,5 @@ The Settings page consists of the following sections:
* [Resource Configuration](webapp_settings_resource_configs.md) (ClearML Enterprise Server) - Define the available resources and the way in which they * [Resource Configuration](webapp_settings_resource_configs.md) (ClearML Enterprise Server) - Define the available resources and the way in which they
will be allocated to different workloads will be allocated to different workloads
* [Billing & Usage](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current billing details and usage information * [Billing & Usage](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current billing details and usage information
* [Storage Credentials](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to * [Storage Cleanup](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to
enable ClearML to delete artifacts stored in cloud storage when tasks and models are deleted enable ClearML to delete artifacts stored in cloud storage when tasks and models are deleted

View File

@ -63,7 +63,7 @@ to switch to.
![Workspace configuration page](../../img/settings_workspace_configuration.png#light-mode-only) ![Workspace configuration page](../../img/settings_workspace_configuration.png#light-mode-only)
![Workspace configuration page](../../img/settings_workspace_configuration_dark.png#dark-mode-only) ![Workspace configuration page](../../img/settings_workspace_configuration_dark.png#dark-mode-only)
### ClearML App Credentials ### ClearML API Credentials
Generate ClearML credentials, made up of an access and secret key pair, and insert them into your [configuration file](../../configs/clearml_conf.md) Generate ClearML credentials, made up of an access and secret key pair, and insert them into your [configuration file](../../configs/clearml_conf.md)
or Jupyter Notebook to grant the ClearML SDK and the ClearML Agent API access to the server. or Jupyter Notebook to grant the ClearML SDK and the ClearML Agent API access to the server.
@ -74,7 +74,7 @@ You can create credentials for any workspace that you are a member of.
1. In **WORKSPACE**, expand the desired workspace's panel (self-deployed ClearML Server users have one workspace) 1. In **WORKSPACE**, expand the desired workspace's panel (self-deployed ClearML Server users have one workspace)
1. In **App Credentials**, click **+ Create new credentials** 1. In **API Credentials**, click **+ Create new credentials**
1. In the dialog that pops up, you can input a label for the new credentials 1. In the dialog that pops up, you can input a label for the new credentials
@ -114,7 +114,7 @@ Generate tokens providing API access to the AI Application Gateway endpoints:
### Changing Your Workspace Name ### Changing Your Workspace Name
To change the name of your own workspace, click **Edit workspace name** <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" /> To change the name of your own workspace, click **Edit workspace name** <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" />
(under App credentials) **>** modify the name **>** click <img src="/docs/latest/icons/ico-save.svg" alt="Check Mark" className="icon size-md" />. (under API credentials) **>** modify the name **>** click <img src="/docs/latest/icons/ico-save.svg" alt="Check Mark" className="icon size-md" />.
### Adding Users to Your Workspace ### Adding Users to Your Workspace

View File

@ -1,5 +1,5 @@
--- ---
title: Storage Credentials title: Storage Cleanup
--- ---
To enable ClearML to delete task artifacts stored in cloud storage when a task is deleted, configure access credentials for your storage provider: To enable ClearML to delete task artifacts stored in cloud storage when a task is deleted, configure access credentials for your storage provider:
@ -7,8 +7,8 @@ To enable ClearML to delete task artifacts stored in cloud storage when a task i
* [AWS S3 Storage](#aws-s3-storage) * [AWS S3 Storage](#aws-s3-storage)
* [Azure](#azure) * [Azure](#azure)
![Storage Credentials page](../../img/webapp_settings_storage_credentials.png#light-mode-only) ![Storage Cleanup page](../../img/webapp_settings_storage_credentials.png#light-mode-only)
![Storage Credentials page](../../img/webapp_settings_storage_credentials_dark.png#dark-mode-only) ![Storage Cleanup page](../../img/webapp_settings_storage_credentials_dark.png#dark-mode-only)
## Google Cloud Storage ## Google Cloud Storage
Set up credentials for Google Cloud buckets: Set up credentials for Google Cloud buckets:

View File

@ -19,6 +19,9 @@ Each row of the table includes:
* If the user's invitation is pending, the date the user was added * If the user's invitation is pending, the date the user was added
* [User groups](#user-groups) (ClearML Enterprise feature) * [User groups](#user-groups) (ClearML Enterprise feature)
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
to find a specific user by their name, email, or user ID.
![Users table](../../img/settings_user_table.png#light-mode-only) ![Users table](../../img/settings_user_table.png#light-mode-only)
![Users table](../../img/settings_user_table_dark.png#dark-mode-only) ![Users table](../../img/settings_user_table_dark.png#dark-mode-only)
@ -68,6 +71,9 @@ Each row of the table includes:
* Credentials - Number of credentials currently available to the account * Credentials - Number of credentials currently available to the account
* Last active time * Last active time
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
to find a specific service account by its name or ID.
Hover over a service account in the table to **Edit** or **Delete** it. Hover over a service account in the table to **Edit** or **Delete** it.
![Service accounts](../../img/settings_service_accounts.png#light-mode-only) ![Service accounts](../../img/settings_service_accounts.png#light-mode-only)
@ -163,6 +169,8 @@ The system includes three pre-configured groups that can't be removed:
* `Queue admins` - Can create / delete / rename queues * `Queue admins` - Can create / delete / rename queues
The user group table lists all the active user groups. Each row includes a group's name, description, member list, and ID. The user group table lists all the active user groups. Each row includes a group's name, description, member list, and ID.
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
to find a specific user group by its name, description, or ID.
![User Management Page](../../img/settings_user_management_table.png#light-mode-only) ![User Management Page](../../img/settings_user_management_table.png#light-mode-only)
![User Management Page](../../img/settings_user_management_table_dark.png#dark-mode-only) ![User Management Page](../../img/settings_user_management_table_dark.png#dark-mode-only)

View File

@ -37,7 +37,7 @@ Click the profile menu button <img src="/docs/latest/icons/ico-me.svg" alt="Prof
in the top right corner of the web UI screen to access the following: in the top right corner of the web UI screen to access the following:
* **Settings** - Navigate to ClearML's user [Settings](settings/webapp_settings_profile.md) page: * **Settings** - Navigate to ClearML's user [Settings](settings/webapp_settings_profile.md) page:
* Set [WebApp preferences](settings/webapp_settings_profile.md) * Set [WebApp preferences](settings/webapp_settings_profile.md)
* Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-credentials) * Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-api-credentials)
* Configure [cloud storage access credentials](settings/webapp_settings_profile.md#browser-cloud-storage-access) for the ClearML Web UI * Configure [cloud storage access credentials](settings/webapp_settings_profile.md#browser-cloud-storage-access) for the ClearML Web UI
* ClearML Hosted service specific options * ClearML Hosted service specific options
* Manage [users and workspaces](settings/webapp_settings_users.md) * Manage [users and workspaces](settings/webapp_settings_users.md)