mirror of
				https://github.com/clearml/clearml-docs
				synced 2025-06-26 18:17:44 +00:00 
			
		
		
		
	This commit is contained in:
		
							parent
							
								
									18d3209059
								
							
						
					
					
						commit
						30805e474d
					
				@ -30,7 +30,7 @@ and running, users can send Tasks to be executed on Google Colab's hardware.
 | 
			
		||||
   This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.
 | 
			
		||||
   
 | 
			
		||||
1. Create new credentials. Go to your [**Settings**](https://app.clear.ml/settings/workspace-configuration) page > **WORKSPACE** section. 
 | 
			
		||||
   Under **App Credentials**, click **+ Create new credentials**, and copy the information that pops up. 
 | 
			
		||||
   Under **API Credentials**, click **+ Create new credentials**, and copy the information that pops up. 
 | 
			
		||||
   
 | 
			
		||||
1. Set the credentials. In the third cell, enter your own credentials:
 | 
			
		||||
   ```python
 | 
			
		||||
 | 
			
		||||
@ -71,40 +71,6 @@ ROI label mapping (label translation) applies to the new model. For example, app
 | 
			
		||||
Define class labels for the new model and assign integers to each in order to maintain data conformity across multiple 
 | 
			
		||||
codebases and datasets. It is important to set enumeration values for all labels of importance. 
 | 
			
		||||
 | 
			
		||||
## Data Augmentation
 | 
			
		||||
 | 
			
		||||
On-the-fly data augmentation is applied to SingleFrames, transforming images without creating new data. Apply data augmentation 
 | 
			
		||||
in steps, where each step is composed of a method, an operation, and a strength as follows: 
 | 
			
		||||
 | 
			
		||||
* **Affine** augmentation method - Transform an image's geometric shape to another position on a 2-dimensional plane. 
 | 
			
		||||
  Use any of the following operations:
 | 
			
		||||
 | 
			
		||||
    * Rotate
 | 
			
		||||
    * Reflect-horiz - Flip images horizontally
 | 
			
		||||
    * Reflect-vert - Flip images vertically
 | 
			
		||||
    * Scale
 | 
			
		||||
    * Shear - Skew
 | 
			
		||||
    * No operation - Randomly select SingleFrames that are not transformed (skipped). If the experiment runs again, and 
 | 
			
		||||
      the random seed in [iteration control](#iteration-control) is unchanged, the same SingleFrames are not augmented.
 | 
			
		||||
    
 | 
			
		||||
* **Pixel** augmentation method - Transform images by modifying pixel values while retaining shape and perspective.  
 | 
			
		||||
  Use any of the following operations:
 | 
			
		||||
 | 
			
		||||
    * Blur - Gaussian smoothing 
 | 
			
		||||
    * Noise - ClearML Enterprise's own noise augmentation consisting of: 
 | 
			
		||||
      * **high** noise - like snow on analog televisions with a weak TV signal 
 | 
			
		||||
      * **low** noise - like a low resolution image magnified in localized areas on the image
 | 
			
		||||
    * Recolor - using an internal RGB lookup-table
 | 
			
		||||
    * No operation - Randomly select SingleFrames that are not transformed (skipped). If the experiment runs again, and 
 | 
			
		||||
      the random seed in [iteration control](#iteration-control) is unchanged, the same SingleFrames are not augmented.
 | 
			
		||||
    
 | 
			
		||||
* Strength - A number applied to adjust the degree of transformation. The recommended strengths are the following:
 | 
			
		||||
 | 
			
		||||
    * 0.0 - No effect
 | 
			
		||||
    * 0.5 - Low (weak)
 | 
			
		||||
    * 1.0 - Medium (recommended)
 | 
			
		||||
    * 2.0 - High (strong)
 | 
			
		||||
 | 
			
		||||
## Iteration Control
 | 
			
		||||
 | 
			
		||||
The input data **iteration control** settings determine the order, number, timing, and reproducibility of the Dataview iterating 
 | 
			
		||||
 | 
			
		||||
@ -159,11 +159,32 @@ To draw a new annotation:
 | 
			
		||||
 | 
			
		||||
    A new annotation is created. 
 | 
			
		||||
 | 
			
		||||
1. In the newly created annotation, select or type-in a label(s). Click the circle in the label name to select a 
 | 
			
		||||
   different label color. 
 | 
			
		||||
1. In the newly created annotation, select or type-in the labels for this annotation. 
 | 
			
		||||
 | 
			
		||||
You can use the **Default ROI Label(s)** list to automatically set labels to all new annotations. 
 | 
			
		||||
 | 
			
		||||
:::info Annotation color
 | 
			
		||||
Each annotation label is automatically assigned a color based on its value. The annotation color is automatically 
 | 
			
		||||
calculated based on the colors of its labels.
 | 
			
		||||
 | 
			
		||||
Click the color circle in the label name to manually set the label's color.
 | 
			
		||||
 | 
			
		||||
<div class="max-w-75">
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
</div>
 | 
			
		||||
 | 
			
		||||
Click the color circle in the annotation header to manually set the annotation’s color and its opacity.  
 | 
			
		||||
 | 
			
		||||
<div class="max-w-75">
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
</div>
 | 
			
		||||
:::
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
#### Copying Frame Objects 
 | 
			
		||||
You can copy existing annotations, and paste them to any frame of your choice:
 | 
			
		||||
1. Click the annotation or bounded area in the frame.
 | 
			
		||||
 | 
			
		||||
@ -8,16 +8,19 @@ deleting [Dataset versions](../dataset.md#dataset-versioning).
 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
## Dataset Version History
 | 
			
		||||
The WebApp (UI) presents your dataset version structure in list view <img src="/docs/latest/icons/ico-list-view.svg" alt="List view" className="icon size-md space-sm" /> 
 | 
			
		||||
or tree view <img src="/docs/latest/icons/ico-tree-view.svg" alt="Tree view" className="icon size-md space-sm" />. 
 | 
			
		||||
The WebApp (UI) presents your dataset version structure in tree view <img src="/docs/latest/icons/ico-tree-view.svg" alt="Tree view" className="icon size-md space-sm" />
 | 
			
		||||
or list view <img src="/docs/latest/icons/ico-list-view.svg" alt="List view" className="icon size-md space-sm" />. 
 | 
			
		||||
 | 
			
		||||
The tree view shows the lineage of the dataset's versions.
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
The list view lists the dataset's versions chronologically by last update time.
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
The tree view shows the lineage of the dataset's versions.
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
Click <img src="/docs/latest/icons/ico-sort.svg" alt="Sort order" className="icon size-md space-sm" /> to order the 
 | 
			
		||||
dataset versions in ascending or descending order based on their last update time. 
 | 
			
		||||
 | 
			
		||||
Use the search bar to find specific versions. You can query by version name, version description, or version ID. The search returns 
 | 
			
		||||
all versions that match the query.
 | 
			
		||||
 | 
			
		||||
@ -78,7 +78,7 @@ Access these actions with the context menu in any of the following ways:
 | 
			
		||||
 | 
			
		||||
| ClearML Action | Description |
 | 
			
		||||
|---|---|
 | 
			
		||||
| Details | View Dataview details, including input datasets, label mapping, augmentation operations, and iteration control. Can also be accessed by double-clicking a Dataview in the Dataviews table. |
 | 
			
		||||
| Details | View Dataview details, including input datasets, label mapping, and iteration control. Can also be accessed by double-clicking a Dataview in the Dataviews table. |
 | 
			
		||||
| Archive | Move Dataview to the Dataview's archive. | 
 | 
			
		||||
| Restore | Action available in the archive. Restore a Dataview to the active Dataviews table. |
 | 
			
		||||
| Delete | Action available in the archive. Permanently delete a Dataview. |
 | 
			
		||||
 | 
			
		||||
@ -12,7 +12,6 @@ enables modifying [Dataviews](webapp_dataviews.md), including:
 | 
			
		||||
* [Frame filtering](#filtering-frames)
 | 
			
		||||
* [Label mapping](#mapping-labels-label-translation)
 | 
			
		||||
* [Class label enumeration](#label-enumeration)
 | 
			
		||||
* [Data augmentation](#data-augmentation)
 | 
			
		||||
* [Input frame iteration controls](#iteration-controls)
 | 
			
		||||
 | 
			
		||||
## Selecting Dataviews
 | 
			
		||||
@ -22,7 +21,7 @@ enables modifying [Dataviews](webapp_dataviews.md), including:
 | 
			
		||||
* Create a new Dataview  
 | 
			
		||||
    
 | 
			
		||||
    * Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation), 
 | 
			
		||||
      and set label enumeration, data augmentation, and iteration controls.
 | 
			
		||||
      and set label enumeration and iteration controls.
 | 
			
		||||
      
 | 
			
		||||
* Select a different Dataview already associated with the experiment. 
 | 
			
		||||
    
 | 
			
		||||
@ -118,22 +117,6 @@ Modify the label enumeration assigned to output models.
 | 
			
		||||
 | 
			
		||||
1. Click **SAVE**.
 | 
			
		||||
 | 
			
		||||
## Data Augmentation
 | 
			
		||||
 | 
			
		||||
Modify the on-the-fly data augmentation applied to frame input from the select Hyper-Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
 | 
			
		||||
 
 | 
			
		||||
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
 | 
			
		||||
 | 
			
		||||
**To modify data augmentation**
 | 
			
		||||
 | 
			
		||||
1. In the **AUGMENTATION** section, click **EDIT**.
 | 
			
		||||
 | 
			
		||||
    * Add (**+**) or edit an augmentation step - Select a **METHOD**, **OPERATION**, and **STRENGTH**.
 | 
			
		||||
        
 | 
			
		||||
    * Remove (<img src="/docs/latest/icons/ico-trash.svg" alt="Trash" className="icon size-md space-sm" />)  an augmentation step.
 | 
			
		||||
    
 | 
			
		||||
1. Click **SAVE**.    
 | 
			
		||||
    
 | 
			
		||||
## Iteration Controls
 | 
			
		||||
 | 
			
		||||
Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames 
 | 
			
		||||
 | 
			
		||||
@ -14,7 +14,6 @@ In an experiment's page, go to the **DATAVIEWS** tab to view all the experiment'
 | 
			
		||||
* Input data [selection](#input) and [filtering](#filtering)
 | 
			
		||||
* ROI [mapping](#mapping) (label translation)
 | 
			
		||||
* [Label enumeration](#label-enumeration)
 | 
			
		||||
* On-the-fly [data augmentation](#augmentation)
 | 
			
		||||
* [Iteration controls](#iteration-control)
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
@ -52,13 +51,6 @@ For detailed information, see [Mapping ROI labels](../dataviews.md#mapping-roi-l
 | 
			
		||||
 | 
			
		||||
Assign label enumeration in the **LABELS ENUMERATION** area.
 | 
			
		||||
 | 
			
		||||
### Augmentation
 | 
			
		||||
 | 
			
		||||
On-the-fly data augmentation applied to SingleFrames, which does not create new data. Apply data Augmentation in steps, 
 | 
			
		||||
where each step is composed of a method, an operation, and a strength.
 | 
			
		||||
 
 | 
			
		||||
For detailed information, see [Data augmentation](../dataviews.md#data-augmentation).
 | 
			
		||||
 | 
			
		||||
### Iteration Control
 | 
			
		||||
 | 
			
		||||
The input data iteration control settings determine the order, number, timing, and reproducibility of the Dataview iterating 
 | 
			
		||||
 | 
			
		||||
							
								
								
									
										
											BIN
										
									
								
								docs/img/annotation_label_color.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										
											BIN
										
									
								
								docs/img/annotation_label_color.png
									
									
									
									
									
										Normal file
									
								
							
										
											Binary file not shown.
										
									
								
							| 
		 After Width: | Height: | Size: 13 KiB  | 
							
								
								
									
										
											BIN
										
									
								
								docs/img/annotation_label_opacity.png
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										
											BIN
										
									
								
								docs/img/annotation_label_opacity.png
									
									
									
									
									
										Normal file
									
								
							
										
											Binary file not shown.
										
									
								
							| 
		 After Width: | Height: | Size: 13 KiB  | 
@ -25,5 +25,5 @@ The Settings page consists of the following sections:
 | 
			
		||||
  * [Resource Configuration](webapp_settings_resource_configs.md) (ClearML Enterprise Server) - Define the available resources and the way in which they 
 | 
			
		||||
  will be allocated to different workloads 
 | 
			
		||||
  * [Billing & Usage](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current billing details and usage information 
 | 
			
		||||
  * [Storage Credentials](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to 
 | 
			
		||||
  * [Storage Cleanup](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to 
 | 
			
		||||
  enable ClearML to delete artifacts stored in cloud storage when tasks and models are deleted
 | 
			
		||||
@ -63,7 +63,7 @@ to switch to.
 | 
			
		||||

 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
### ClearML App Credentials
 | 
			
		||||
### ClearML API Credentials
 | 
			
		||||
 | 
			
		||||
Generate ClearML credentials, made up of an access and secret key pair, and insert them into your [configuration file](../../configs/clearml_conf.md) 
 | 
			
		||||
or Jupyter Notebook to grant the ClearML SDK and the ClearML Agent API access to the server. 
 | 
			
		||||
@ -74,7 +74,7 @@ You can create credentials for any workspace that you are a member of.
 | 
			
		||||
 | 
			
		||||
1. In **WORKSPACE**, expand the desired workspace's panel (self-deployed ClearML Server users have one workspace)
 | 
			
		||||
 | 
			
		||||
1. In **App Credentials**, click **+ Create new credentials**
 | 
			
		||||
1. In **API Credentials**, click **+ Create new credentials**
 | 
			
		||||
 | 
			
		||||
1. In the dialog that pops up, you can input a label for the new credentials 
 | 
			
		||||
 | 
			
		||||
@ -114,7 +114,7 @@ Generate tokens providing API access to the AI Application Gateway endpoints:
 | 
			
		||||
 | 
			
		||||
### Changing Your Workspace Name
 | 
			
		||||
To change the name of your own workspace, click **Edit workspace name**  <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" /> 
 | 
			
		||||
(under App credentials) **>** modify the name **>** click <img src="/docs/latest/icons/ico-save.svg" alt="Check Mark" className="icon size-md" />. 
 | 
			
		||||
(under API credentials) **>** modify the name **>** click <img src="/docs/latest/icons/ico-save.svg" alt="Check Mark" className="icon size-md" />. 
 | 
			
		||||
 | 
			
		||||
### Adding Users to Your Workspace
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
@ -1,5 +1,5 @@
 | 
			
		||||
---
 | 
			
		||||
title: Storage Credentials 
 | 
			
		||||
title: Storage Cleanup 
 | 
			
		||||
---
 | 
			
		||||
 | 
			
		||||
To enable ClearML to delete task artifacts stored in cloud storage when a task is deleted, configure access credentials for your storage provider:
 | 
			
		||||
@ -7,8 +7,8 @@ To enable ClearML to delete task artifacts stored in cloud storage when a task i
 | 
			
		||||
* [AWS S3 Storage](#aws-s3-storage)
 | 
			
		||||
* [Azure](#azure)
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||

 | 
			
		||||

 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
## Google Cloud Storage
 | 
			
		||||
Set up credentials for Google Cloud buckets: 
 | 
			
		||||
 | 
			
		||||
@ -19,6 +19,9 @@ Each row of the table includes:
 | 
			
		||||
* If the user's invitation is pending, the date the user was added
 | 
			
		||||
* [User groups](#user-groups) (ClearML Enterprise feature)
 | 
			
		||||
 | 
			
		||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" />
 | 
			
		||||
to find a specific user by their name, email, or user ID. 
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
@ -68,6 +71,9 @@ Each row of the table includes:
 | 
			
		||||
* Credentials - Number of credentials currently available to the account
 | 
			
		||||
* Last active time
 | 
			
		||||
 | 
			
		||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" /> 
 | 
			
		||||
to find a specific service account by its name or ID.
 | 
			
		||||
 | 
			
		||||
Hover over a service account in the table to **Edit** or **Delete** it.
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||
@ -163,6 +169,8 @@ The system includes three pre-configured groups that can't be removed:
 | 
			
		||||
* `Queue admins` - Can create / delete / rename queues
 | 
			
		||||
 | 
			
		||||
The user group table lists all the active user groups. Each row includes a group's name, description, member list, and ID. 
 | 
			
		||||
Use the search bar <img src="/docs/latest/icons/ico-search.svg" alt="Magnifying glass" className="icon size-md space-md" /> 
 | 
			
		||||
to find a specific user group by its name, description, or ID.
 | 
			
		||||
 | 
			
		||||

 | 
			
		||||

 | 
			
		||||
 | 
			
		||||
@ -37,7 +37,7 @@ Click the profile menu button <img src="/docs/latest/icons/ico-me.svg" alt="Prof
 | 
			
		||||
in the top right corner of the web UI screen to access the following:
 | 
			
		||||
* **Settings** - Navigate to ClearML's user [Settings](settings/webapp_settings_profile.md) page:
 | 
			
		||||
  * Set [WebApp preferences](settings/webapp_settings_profile.md)
 | 
			
		||||
  * Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-credentials)
 | 
			
		||||
  * Manage [workspace API credentials](settings/webapp_settings_profile.md#clearml-api-credentials)
 | 
			
		||||
  * Configure [cloud storage access credentials](settings/webapp_settings_profile.md#browser-cloud-storage-access) for the ClearML Web UI
 | 
			
		||||
  * ClearML Hosted service specific options
 | 
			
		||||
    * Manage [users and workspaces](settings/webapp_settings_users.md)
 | 
			
		||||
 | 
			
		||||
		Loading…
	
		Reference in New Issue
	
	Block a user