ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add Hyper-Datasets

Browse Source
This commit is contained in:
allegroai
2021-06-21 01:00:16 +03:00
parent 31b3b52cac
commit 81bcabcb10
76 changed files with 3319 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files

162
docs/hyperdatasets/webapp/webapp_annotator.md Normal file
View File

@@ -0,0 +1,162 @@
---
title: Annotation Tasks
---
Use the Annotations page to access and manage annotation Tasks.
Use annotation tasks to efficiently organize the annotation of frames in Dataset versions and manage the work of annotators
(see [Annotating Images and Videos](#annotating-images-and-video)).
## Managing Annotation Tasks
### Creating Annotation Tasks
![image](../../img/hyperdatasets/annotation_task_01.png)
**To create an annotation task:**
1. On the Annotator page, click **+ ADD NEW ANNOTATION**.
1. Enter a name for your new annotation task.
1. Choose a Dataset version to annotate. If the selected Dataset version's status is *Published*, then creating this
annotation task also creates a child version of the selected version. The new child version's status is *Draft*, and
its name is the same as the annotation task.
1. Set the filters for the frames this annotation task presents to the annotator.
* In the **SET FILTERS** list, choose either:
* **All Frames** - Include all frames in this task.
* **Empty Frames** - Include only frames without any annotations in this task.
* **By Label** - Include only frames with specific labels, and optionally filter these frames by confidence level and
the number of instances. You can also click <img src="/static/icons/ico-code.svg" className="icon size-md space-sm" /> and then add a Lucene query for this ROI label filter.
1. Choose the iteration parameters specifying how frames in this version are presented to the annotator.
1. In **ITERATION**, in the **ORDER** list, choose either:
* **Sequential** - Frames are sorted by the frame top-level `context_id` (primary sort key) and `timestamp` (secondary sort key) metadata key values, and returned by the iterator in the sorted order.
* **Random** - Frames are randomly returned using the value of the `random_seed` argument. The random seed is maintained with the experiments. Therefore, the random order is reproducible if the experiment is rerun.
1. In **REPETITION**, choose either **Use Each Frame Once** or **Limit Frames**. If you select **Limit Frames**, then in **Use Max. Frames**, type the number of frames to annotate.
1. If iterating randomly, in **RANDOM SEED** type your seed or leave blank, and the ClearML Enterprise platform generates a seed for you.
1. If annotating video, then in **CLIP LENGTH (FOR VIDEO)**, type of the number of sequential frames per iteration to annotate.
1. Click **Create**.
### Completing annotation tasks
To mark an annotation task as **Completed**:
* In the annotation task card, click <img src="/static/icons/ico-bars-menu.svg" className="icon size-md space-sm" /> (menu) **>** **Complete** **>** **CONFIRM**.
### Deleting annotation tasks
To delete an annotation task:
* In the annotation task card, click <img src="/static/icons/ico-bars-menu.svg" className="icon size-md space-sm" /> (menu) **>** **Delete** **>** **CONFIRM**.
### Filtering annotation tasks
There are two option for filtering annotation tasks:
* Active / Completed Filter - Toggle to show annotation tasks that are either **Active** or **Completed**
* Dataset Filter - Use to view only the annotation tasks for a specific Dataset.
### Sorting annotation tasks
Sort the annotation tasks by either using **RECENT** or **NAME** from the drop-down menu on the top left of the page.
### Viewing annotation task information
To View the Dataset version, filters, and iteration information:
* In the annotation task card, click <img src="/static/icons/ico-bars-menu.svg" className="icon size-md space-sm" /> (menu) **>** **Info**
## Annotating Images and Video
Annotate images and video by labeling regions of interest in Dataset version frames. The frames presented for annotation
depend upon the settings in the annotation task (see [Creating Annotation Tasks](#creating-annotation-tasks)).
### Annotating Frames
**To annotate frames:**
1. On the Annotator page, click the annotation task card, or click <img src="/static/icons/ico-bars-menu.svg" className="icon size-md space-sm" /> (menu)
and then click **Annotate**.
1. See instructions below about annotating frames.
#### Add FrameGroup objects
1. Select an annotation mode and add the bounded area to the frame image.
* Rectangle mode - Click <img src="/static/icons/ico-rectangle-icon-purple.svg" className="icon size-md space-sm" /> and then click the image, drag and release.
* Polygon mode - Click <img src="/static/icons/ico-polygon-icon-purple.svg" className="icon size-md space-sm" /> and then click the image for the first vertex,
move to another vertex and click, continue until closing the last vertex.
* Key points mode - Click <img src="/static/icons/ico-keypoint-icon-purple.svg" className="icon size-md space-sm" /> and then click each key point.
1. In the new label area, choose or enter a label.
1. Optionally, add metadata.
1. Optionally, lock the annotation.
#### Add frame labels
1. In **FRAME LABEL**, click **+ Add new**.
1. In the new label area, choose or enter a label.
1. Optionally, add metadata.
1. Optionally, lock the annotation.
#### Copy / paste an annotations
1. Click the annotation or bounded area in the image or video clip.
1. Optionally, navigate to a different frame.
1. Click **PASTE**. The new annotation appears in the same location as the one you copied.
1. Optionally, to paste the same annotation, again, click **PASTE**.
#### Copy / paste all annotations
1. Click **COPY ALL**.
1. Optionally, navigate to a different frame.
1. Click **PASTE**.
#### Move annotations
* Move a bounded area by clicking on it and dragging.
#### Resize annotations
* Resize a bounded area by clicking on a vertex and dragging.
#### Delete annotations
1. Click the annotation or bounded area in the image or video clip.
1. Press **DELETE** or in the annotation, click **>X**.
#### Add labels
* Click in the annotation and choose a label from the label list, or type a new label.
#### Modify labels
* In the annotation label textbox, choose a label from the list or type a new label.
#### Delete labels
* In the annotation, in the label area, click the label's **X**.
#### Modify annotation metadata
* In the label, click edit and then in the popup modify the metadata dictionary (in JSON format).
#### Modify annotation color
* Modify the color of an area by clicking the circle in the label name and select a new color.
#### Lock / unlock annotations
* Click the lock.
#### Modify frame metadata
* Expand the **FRAME METADATA** area, click edit, and then in the popup modify the metadata dictionary (in JSON format).

44
docs/hyperdatasets/webapp/webapp_datasets.md Normal file
View File

@@ -0,0 +1,44 @@
---
title: Datasets Page
---
The Datasets page offers the following functionalities:
* Managing the ClearML Enterprise **Datasets** and **versions**, which connect raw data to the ClearML Enterprise platform
* Using ClearML Enterprise's Git-like Dataset versioning features
* Managing SingleFrames and FrameGroups.
![image](../../img/hyperdatasets/datasets_01.png)
## Dataset cards
Dataset cards show summary information about versions, frames, and labels in a Dataset, and the elapsed time since the Dataset was last update and the user doing the update. Dataset cards allow you to open a specific Dataset to perform Dataset versioning and frames management.
* Dataset name
* Elapsed time since the last update. Hover over elapsed time and view date of last update.
* User updating the Dataset
* The number of versions in the Dataset
* The total number of frames in all versions of the Dataset. If an asterisk (\*) appears next to **FRAMES**, then you can hover it and see the name of the version whose frames were last updated appears.
* The percentage of frames annotated in all versions of the Dataset. If an asterisk (\*) appears next to **ANNOTATED**, then you can hover it and see the name of the version whose frames were last annotated appears.
* If the Dataset version's status is *Published*, then the top labels in the Dataset, color coded (colors are editable) appear. If the Dataset version is Draft, then no labels appear.
:::note
To change the label color coding, hover over a label color, click thr hand pointer, and then select a new color.
:::
## Creating new Datasets
Create a new Dataset which will contain one version named `Current`. The new version will not contain any frames.
* Click **+ NEW DATASET** **>** Enter a name and optionally a description **>** **CREATE DATASET**.
## Sort Datasets
* In **RECENT**, choose either:
* **RECENT** - Most recently update of the Datasets.
* **NAME** - Alphabetically sort by Dataset name.

310
docs/hyperdatasets/webapp/webapp_datasets_frames.md Normal file
View File

@@ -0,0 +1,310 @@
---
title: Working with Frames
---
View and edit SingleFrames in the Dataset page. After selecting a Dataset version, the **Version Browser** shows a sample
of frames and enables viewing SingleFrames and FramesGroups, and edit SingleFrames, in the [frame viewer](#frame-viewer).
Before opening the frame viewer, you can filter the frames by applying [simple](#simple-frame-filtering) or [advanced](#advanced-frame-filtering)
filtering logic.
![image](../../img/hyperdatasets/frames_01.png)
## Frame viewer
Frame viewer allows you to view and edit annotations which can be FrameGroup objects (Regions of Interest) and FrameGroup
labels applied to the entire frame not a region of the frame, the frame details (see [frames](../frames.md)),
frame metadata, the raw data source URI, as well as providing navigation and viewing tools.
![image](../../img/hyperdatasets/web-app/dataset_example_frame_editor.png)
### Frame viewer controls
Use frame viewer controls to navigate between frames in a Dataset Version, and control frame changes and viewing.
|Control Icon|Actions|
|-----|------|
|<img src="/static/icons/ico-skip-backward.svg" className="icon size-md space-sm" />|Jump backwards (CTRL + Left). Jumps backwards by five frames.|
|<img src="/static/icons/ico-skip-previous.svg" className="icon size-md space-sm" />|Go to the previous frame containing a non-filtered annotation. The filter is the minimum confidence level setting. If the confidence level filter is set to zero, any frame containing annotations matches the filter.|
|<img src="/static/icons/ico-arrow-left.svg" className="icon size-md space-sm" />|Go to the previous frame (Left Arrow).|
|<img src="/static/icons/ico-arrow-right.svg" className="icon size-md space-sm" />|Go to the next frame (Right Arrow).|
|<img src="/static/icons/ico-skip-next.svg" className="icon size-md space-sm" />|Go to the next frame containing a non-filtered annotation (same filter as <img src="/static/icons/ico-skip-previous.svg" className="icon size-md space-sm" />).|
|<img src="/static/icons/ico-skip-forward.svg" className="icon size-md space-sm" />|Jump forwards (CTRL + Right). Jumps 5 frames forwards.|
|<img src="/static/icons/ico-revert.svg" className="icon size-md space-sm" />|Reload the frame.|
|<img src="/static/icons/ico-undo.svg" className="icon size-md space-sm" />|Undo changes.|
|<img src="/static/icons/ico-redo.svg" className="icon size-md space-sm" />|Redo changes.|
|<img src="/static/icons/ico-reset_1.svg" className="icon size-md space-sm" />|Autofit|
|<img src="/static/icons/ico-zoom-in.svg" className="icon size-md space-sm" />|Zoom in|
|<img src="/static/icons/ico-zoom-out.svg" className="icon size-md space-sm" />|Zoom out|
|Percentage textbox|Zoom percentage|
### Viewing and editing frames
**To view / edit a frame in the frame editor**
1. Locate your frame by applying a [simple frame filter](#simple) or [advanced frame filter](#advanced), and clicking <span class="tr_gui">LOAD MORE</span>, if required.
1. Click the frame thumbnail. The frame editor appears.
1. Do any of the following:
* View frame details, including:
* Frame file path
* Dimensions of the image or video
* Frame details
* Frame metadata
* Annotations
* Frame objects - Labeled Regions of Interest, with confidence levels and custom metadata per frame object.
* Frame labels - Labels applied to the entire frame, not a region in the frame.
* Optionally, filter annotations by confidence level using the <span class="tr_gui">Minimum confidence</span> slider.
* Add, change, and delete [annotations](#annotations) and [frame metadata](#frame-metadata).
:::important
To save frames changes at any time, click **SAVE** (below the annotation list area).
:::
### Viewing FrameGroups
Viewing and editing frames in a FrameGroup is similar to viewing and editing SingleFrames.
Click the FrameGroup in the Dataset. In the frame viewer, select SingleFrame to view / modify from
a dropdown list in the **Current Source** section.
![image](../../img/hyperdatasets/framegroup_01.png)
## Filtering frames
### Simple frame filtering
Simple frame filtering applies one annotation object (ROI) label and returns frames containing at least one annotation
with that label.
**To apply a simple frame filter:**
* In the **Version Browser**, choose a label on the label list.
For example:
* Before filtering, the **Version Browser** in the image below contains seven frames.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_01.png)
</div>
</details>
<br/>
* A simple label filter for `person` shows three frames with each containing at least one ROI labeled `person`.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_02.png)
</div>
</details>
### Advanced frame filtering
Advanced frame filtering applies sophisticated filtering logic, which is composed of as many frame filters as needed,
where each frame filter can be a combination of ROI, frame, and source rules.
* ROI rules use include and exclude logic to match frames by ROI label; an ROI label can match frames containing at least
one annotation object (ROI) with all labels in the rule.
* Frame rules and source rules use Lucene queries with AND, OR, and NOT logic. Frame rules apply to frame metadata.
* Source rules apply to frame source information.
**To apply advanced filters:**
1. In the **Version Browser**, click **Switch to advanced filters**.
1. In a **FRAME FILTER**, create one of the following rules:
* ROI rule
* Choose **Include** or **Exclude**, select ROI labels, and optionally set the confidence level range.
* To switch from the ROI dropdown list to a Lucene query mode, click <img src="/static/icons/ico-edit.svg" className="icon size-md space-sm" />.
* Frame rule - Enter a Lucene query using frame metadata fields in the format `meta.<key>:<value>`.
* Source rule - Enter a Lucene query using frame metadata fields in the format `sources.<key>:<value>`.
### Examples
#### ROI rules
* Create one ROI rule for <code>person</code> shows the same three frames as the simple frame filter (above).
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_03.png)
</div>
</details>
<br/>
* In the ROI rule, add a second label. Add `partially_occluded`. Only frames containing at least one ROI labeled as both <code>person</code> and <code>partially_occluded</code> match the filter.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_04.png)
</div>
</details>
<br/>
By opening a frame in the frame viewer, you can see an ROI labeled with both.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_05.png)
</div>
</details>
<br/>
#### Frame rules
Filter by metadata using Lucene queries.
* Add a frame rule to filter by the metadata key <code>dangerous</code> for the value of <code>no</code>.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_08.png)
</div>
</details>
<br/>
By opening a frame in the frame viewer, you can see the metadata.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_09.png)
</div>
</details>
<br/>
#### Source rules
Filter by sources using Lucene queries.
* Add a source rule to filter for sources URIs with a wildcards.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">View a screenshot</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/frame_filtering_10.png)
</div>
</details>
<br/>
Use Lucene queries in ROI label filters and frame rules.
## Annotations
### Frame objects (Regions of Interest)
You can add annotations by drawing new bounding areas, and copying existing annotations in the same or other frames.
**To draw a bounding area for a new annotation:**
1. Optionally, select a default label in the Default ROI Label(s) list. New annotations are automatically assigned this label.
1. Click one of the following modes and create a bounding area in the frame:
* <img src="/static/icons/ico-rectangle-icon-purple.svg" className="icon size-md space-sm" /> - Rectangle mode: Drag a
rectangle onto the frame.
* <img src="/static/icons/ico-ellipse-icon-purple.svg" className="icon size-md space-sm" /> - Ellipse mode: Drag an ellipse
onto the frame.
* <img src="/static/icons/ico-polygon-icon-purple.svg" className="icon size-md space-sm" /> - Polygon mode: Click the polygon
vertices onto the frame.
* <img src="/static/icons/ico-keypoint-icon-purple.svg" className="icon size-md space-sm" /> - Key points mode: Click each
keypoint onto the frame. After the clicking the last keypoint, click the first again to close the bounding area.
A new annotation is created.
1. In the newly created annotation, select or type a label.
1. Optionally, add metadata. This is metadata for the annotation, not the entire frame.
1. Optionally, lock the annotation.
1. If you move to another frame, the frame editor automatically saves changes. Otherwise, if you exit the frame editor,
you are prompted to save.
**To copy an annotation:**
1. Click the annotation or bounded area in the image or video clip.
1. Optionally, navigate to a different frame.
1. Click **PASTE**. The new annotation appears in the same location as the one you copied.
1. Optionally, to paste the same annotation, click **PASTE** again in the desired frame.
**To copy all annotations:**
1. Click **COPY ALL**.
1. Optionally, navigate to a different frame.
1. Click **PASTE**.
### Frame labels
**To add frame labels:**
1. Expand the **FRAME LABELS** area.
1. Click **+ Add new**.
1. Enter a label.
1. Optionally, click <img src="/static/icons/ico-edit.svg" className="icon size-md space-sm" />.
### Annotation management
**To move annotations:**
* Move a bounded area by clicking on it and dragging.
**To resize annotations:**
* Resize a bounded area by clicking on a vertex and dragging.
**To modify annotation metadata:**
* In the label, click edit and then in the popup modify the metadata dictionary (in JSON format).
**To modify annotation colors:**
* Modify the color of an area by clicking the circle in the label name and select a new color.
**To lock annotations:**
* All annotations - Above the annotations, click the lock / unlock.
* A specific annotation - In an annotation, click the lock / unlock.
**To delete annotations:**
1. Click the annotation or bounded area in the image or video clip.
1. Press the **DELETE** key, or in the annotation, click **X**.
**To add, change, or delete labels to annotations labels:**
* Add - Click in the annotation and choose a label from the label list, or type a new label.
* Change - In the annotation label textbox, choose a label from the list or type a new label.
* Delete - In the annotation, in the label area, click the label's **X**.
## Frame metadata
**To edit frame metadata:**
* Expand the **FRAME METADATA** area, click edit, and then in the popup modify the metadata dictionary (in JSON format).

113
docs/hyperdatasets/webapp/webapp_datasets_versioning.md Normal file
View File

@@ -0,0 +1,113 @@
---
title: Dataset Versioning
---
Use the Dataset versioning WebApp (UI) features for viewing, creating, modifying, and
deleting Dataset versions.
From the Datasets page, click on one of the Datasets in order to see and work with its versions.
### Viewing snapshots
View snapshots in the simple version structure using either:
* The simple view, a table of snapshots.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Simple view (snapshot table)</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/web-app/dataset_simple_adv_01.png)
</div>
</details>
<br/>
* The advanced view, a tree of versions. The tree contains one version whose status is <i>Draft</i>, and snapshots appear in
chronological order, with oldest at the top, and the most recent at the bottom.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Advanced view (version tree)</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
</div>
</details>
### Creating snapshots
To create a snapshot, you must be in the simple (version table) view.
**To create a snapshot, do the following:**
1. If you are in the advanced view, click **Switch to Simple View** (In certain situations, this may not be possible,
see [Dataset Versioning](../dataset.md#dataset-versioning))
1. If the **DATASET HISTORY** section is not opened, click it.
1. If a snapshot is currently selected, click **RETURN TO CURRENT VERSION**.
1. Click **+ CREATE SNAPSHOT**.
1. Enter a version name, and optionally a description.
1. Click **CREATE**.
:::note
The WebApp (UI) does not currently support the automatic naming of snapshots with timestamps appended. You must provide a snapshot name.
:::
### Creating versions
To create a version, you must be in the advanced (version tree) view.
**To create a child version, do the following:**
1. If you are in the simple view, click **Switch to Advanced View**.
1. Click the (parent) version from which to create a child (inherit all frames).
1. Click **+ CREATE NEW VERSION**.
1. Enter a version name, and optionally a description.
1. Click **CREATE**.
### Publishing versions
Publish (make read-only) any Dataset version whose status is *Draft*. If the Dataset is in the simple structure,
and you publish the current version, then only the advanced view is available,
and you cannot create snapshots.
**To publish a version, do the following:**
1. If you are in the simple view, click **Switch to Advanced View**.
1. Click the version to publish.
1. Click **PUBLISH**.
1. Click **PUBLISH** again to confirm.
### Exporting frames
Frame exports downloaded filtered frames as a JSON file.
**To export frames, do the following:**
* In the Thumbnails area, click **EXPORT FRAMES**. The frames JSON file downloads.
### Modifying version names
**To modify a Dataset version name, do the following:**
* At the top right of the Dataset page, hover over the Dataset version name, click <img src="/static/icons/ico-edit.svg" className="icon size-md space-sm" /> , edit the name, and then click <img src="/static/icons/ico-save.svg" className="icon size-md space-sm" /> (check).
### Modifying version descriptions
**To modify a version description, do the following:**
* Expand the **INFO** area, hover over the **Description**, click <img src="/static/icons/ico-edit.svg" className="icon size-md space-sm" />,
edit the name, and then click <img src="/static/icons/ico-save.svg" className="icon size-md space-sm" /> (check).
### Deleting versions
You can delete versions whose status is *Draft*.
**To delete the current version, do the following:**
* If you are in the simple view, click **Switch to Advanced View**.
* Click the version to delete.
* Click **DELETE**.
* Click **DELETE** again to confirm.

95
docs/hyperdatasets/webapp/webapp_dataviews.md Normal file
View File

@@ -0,0 +1,95 @@
---
title: Dataviews Table
---
[Dataviews](../dataviews.mda) appear in the same Project as the experiment that stored the Dataview in the **ClearML Enterprise** platform,
as well as the **DATAVIEWS** tab in the **All Projects** page.
The **Dataviews table** is a [customizable](#customizing-the-dataviews-table) list of Dataviews associated with a project.
Use it to [view, create, and edit Dataviews](#viewing-adding-and-editing-dataviews) in the info panel. Dataview tables
can be filtered by name or name fragments and / or ID, by using the search bar.
![image](../../img/hyperdatasets/dataviews_table_01.png)
The Dataviews table columns in their default order are below. Dynamically order the columns by dragging a column heading
to a new position.
* **DATAVIEW** - Dataview name.
* **USER** - User creating the Dataview.
* **CREATED** - Elapsed time since the Dataview was created.
* **DESCRIPTION**
## Customizing the Dataviews table
The Dataviews table can be customized. Changes are persistent (cached in the browser), and represented in the URL.
Save customized settings in a browser bookmark, and share the URL with teammates.
Customize any combination of the following:
* Dynamic column ordering - Drag a column title to a different position.
* Filter by user
* Sort columns - By experiment name and / or elapsed time since creation.
* Column autofit - In the column heading, double click a resizer (column separator).
## Viewing, adding, and editing Dataviews
**To view, add, or edit a Dataview:**
1. Do one of the following:
* Create a new Dataview - Click **+ NEW DATAVIEW**.
* View or edit a Dataview - In the Dataview table, click the Dataview.
1. To edit sections of the Dataview, follow the steps on the "Modifying Dataviews" page for the following:
1. [Selecting Dataset versions](webapp_exp_modifying.md#selecting-dataset-versions)
1. [Filtering frames](webapp_exp_modifying.md#filtering-frames)
1. [Mapping labels (label translation)](webapp_exp_modifying.md#mapping-labels-label-translation) (if appropriate for
the data and experiment)
1. [Label enumeration](webapp_exp_modifying.md#label-enumeration)
1. [Data augmentation](webapp_exp_modifying.md#data-augmentation) (if appropriate for the data
and experiment)
1. [Iteration controls](webapp_exp_modifying.md#iteration-controls)
## Cloning Dataviews
Create an exact editable copy of a Dataview. For example, when tuning an experiment, clone a Dataview to apply the same
frame filters to different Dataset versions.
**To clone a Dataview:**
1. Do one of the following:
* In the Dataview table, right click a Dataview and then click **Clone**.
* If the info panel is opened, click <img src="/docs/img/svg/bars-menu.svg" alt="Menu" className="icon size-lg space-sm" />
(menu) and then click **Clone**.
1. Select a project or accept the current project, enter a name, and optionally enter a description
1. Click **CLONE**.
## Archiving Dataviews
Archive Dataviews to more easily manage current work. Archived Dataviews do not appear on the active Dataviews table.
They only appear in the archive. After archiving, the Dataview can be restored from the archive later.
**To archive a Dataview:**
* In the Dataview table:
* Archive one Dataview - Right click the Dataview **>** **Archive**.
* Archive multiple Dataviews - Select the Dataview checkboxes **>** In the footer menu that appears at the bottom of
the page, click **ARCHIVE**.
* In the Dataview info panel - Click <img src="/docs/img/svg/bars-menu.svg" alt="Menu" className="icon size-lg space-sm" />
(menu) **>** **ARCHIVE**.
**To restore a Dataview:**
1. Go to the Dataview table of the archived Dataview or of the **All Projects** page
1. Click **OPEN ARCHIVE**
1. Do any of the following:
* In the Dataview table:
* Restore one Dataview - Right click the Dataview **>** **Restore**.
* Restore multiple Dataviews - Select the Dataview checkboxes **>** **Restore**.
* In the info panel, restore one Dataview - Click <img src="/docs/img/svg/bars-menu.svg" alt="Menu" className="icon size-lg space-sm" />
(menu) **>** **Restore**.

37
docs/hyperdatasets/webapp/webapp_exp_comparing.md Normal file
View File

@@ -0,0 +1,37 @@
---
title: Comparing Dataviews
---
In addition to [**ClearML**'s comparison features](../../webapp/webapp_exp_comparing.md), the **ClearML Enterprise** WebApp
provides a deep comparison of input data selection criteria of experiment Dataviews, enabling to easily locate, visualize, and analyze differences.
## Selecting experiments
**To select experiments to compare:**
1. In the experiment's table, select the checkbox of each experiment to compare, or select the top checkbox for all experiments.
After selecting the second checkbox, the bottom bar appears.
1. In the bottom bar, click **COMPARE**. The comparison page appears, showing a column for each experiment and differences with
a highlighted background color. The experiment on the left is the base experiment. Other experiments compare to the base experiment.
## Dataviews (input data)
**To locate the input data differences:**
1. Click the **DETAILS** tab **>** Expand the **DATAVIEWS** section, or, in the header, click <img src="/static/icons/ico-previous-diff.svg" alt="Previous diff" className="icon size-md" />
(Previous diff) or <img src="/static/icons/ico-next-diff.svg" className="icon size-md space-sm" /> (Next diff).
1. Expand any of the following sections:
* **Augmentation** - On-the-fly data augmentation.
* **Filtering**
* Frame inclusion and exclusion rules based on ROI labels
* Frame metadata
* Frame sources
* Number of instances of a rule matching ROIs in each frame
* Confidence levels.
* **Iteration** - Iteration controls.
* **Labels Enumeration** - Class label enumeration.
* **Mapping** - ROI label translation.
* **View**
![image](../../img/hyperdatasets/web-app/compare_dataviews.png)

168
docs/hyperdatasets/webapp/webapp_exp_modifying.md Normal file
View File

@@ -0,0 +1,168 @@
---
title: Modifying Dataviews
---
An experiment that has been executed can be [cloned](../../webapp/webapp_exp_reproducing.md), then the cloned experiment's
execution details can be modified, and the modified experiment can be executed.
In addition to all the [**ClearML** tuning capabilities](../../webapp/webapp_exp_tuning.md), the **ClearML Enterprise WebApp** (UI)
enables modifying Dataviews, including:
* [Selected Dataview](#selected-dataview)
* [Dataset versions](#selecting-dataset-versions)
* [Frame filtering](#filtering-frames)
* [Label mapping](#mapping-labels-label-translation)
* [Class label enumeration](#label-enumeration)
* [Data augmentation](#data-augmentation)
* [Input frame iteration controls](#iteration-controls)
The selection and control of input data can be modified in *Draft* experiments that are not [development experiments](../task.md#development-experiments).
Do this by modifying the Dataview used by the experiment. The Dataview specifies the Dataset versions from which frames
are iterated and frame filters (see [Dataviews](webapp_dataviews.md).
**To choose a Dataview**, do any of the following:
* Create a new Dataview
* Click **+** and then follow the instructions below to select Dataset versions, filter frames, map labels (label translation),
and set label enumeration, data augmentation, and iteration controls.
* Select a different Dataview already associated with the experiment.
* In the **SELECTED DATAVIEW** list, choose a Dataview.
* Import a different Dataview associated with the same or another project.
* Click <img src="/static/icons/ico-import.svg" className="icon size-md space-sm" /> (**Import dataview**) and then
select **Import to current dataview** or **Import to aux dataview**.
:::note
After importing a Dataview, it can be renamed and / or removed.
:::
### Selecting Dataset versions
To input data from a different data source or different version of a data source, select a different Dataset version used
by the Dataview.
**To select Dataset versions for input data:**
1. In the **INPUT** area, click **EDIT**.
1. Do any of the following:
* Add a Dataset version - Input frames from another a version of another Dataset.
* Click **+**
* Select a Dataset and a Dataset version
* Remove a Dataset version - Do not input frames from a Dataset version.
Select frames from as many Dataset versions as are needed.
1. Click **SAVE**.
## Filtering frames
Filtering of SingleFrames iterated by a Dataview for input to the experiment is accomplished by frame filters.
For more detailed information, see [Filtering](../dataviews.md#filtering).
**To modify frame filtering:**
1. In the **FILTERING** area, click **EDIT**.
1. For each frame filter:
1. Select the Dataset version to which the frame filter applies.
1. Add, change, or remove any combination of the following rules:
* ROI rule - Include or exclude frames containing any single ROI with any combination of labels in the Dataset
version. Specify a range of the number of matching ROI (instances) per frame, and a range of confidence levels.
* Frame rule - Filter by frame metadata key-value pairs, or ROI labels.
* Source rule - Filter by frame `source` dictionary key-value pairs.
1. Optionally, debias input data by setting ratios for frames returned by the Dataview for each frame filter. These
ratios allow adjusting an imbalance in input data.
1. Click **SAVE**.
## Mapping labels (label translation)
Modify the ROI label mapping rules, which translate one or more input labels to another label for the output model. Labels
that are not mapped are ignored.
**To modify label mapping:**
1. In the **MAPPING** section, click **EDIT**
* Add (**+**) or edit a mapping:
1. Select the Dataset and version whose labels will be mapped.
1. Select one or more labels to map.
1. Select or enter the label to map to in the output model.
* Remove (<img src="/static/icons/ico-trash.svg" className="icon size-md space-sm" />) a mapping.
1. Click **SAVE**
## Label enumeration
Modify the label enumeration assigned to output models.
**To modify label enumeration:**
1. In the **LABELS ENUMERATION** section, click **EDIT**.
* Add (**+**) or edit an enumeration:
* Select a label and then enter an integer for it.
* Remove (<img src="/static/icons/ico-trash.svg" className="icon size-md space-sm" />) an enumeration.
1. Click **SAVE**.
## Data augmentation
Modify the on-the-fly data augmentation applied to frames input from the select Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
**To modify data augmentation**
1. In the **AUGMENTATION** section, click **EDIT**.
* Add (**+**) or edit an augmentation step - Select a **METHOD**, **OPERATION**, and **STRENGTH**.
* Remove (<img src="/static/icons/ico-trash.svg" className="icon size-md space-sm" />) an augmentation step.
1. Click **SAVE**.
## Iteration controls
Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames
for training.
For more detailed information, see [Iteration Control](../dataviews.md#iteration-control).
**To modify iteration controls:**
1. In the **ITERATION** sections, click **EDIT**.
1. Select the **ORDER** of the SingleFrames returned by the iteration, either:
* **Sequential** - Iterate SingleFrames in sorted order by context ID and timestamp.
* **Random** - Iterate SingleFrames randomly using the random seed you can set (see Random Seed below).
1. Select the frame **REPETITION** option, either:
* **Use Each Frame Once**
* **Limit Frames**
* **Infinite Iterations**
1. Select the **RANDOM SEED** - If the experiment is rerun and the seed remains unchanged, the frames iteration is the same.
1. For video, enter a **CLIP LENGTH** - For video data sources, in the number of sequential frames from a clip to iterate.
1. Click **SAVE**.

69
docs/hyperdatasets/webapp/webapp_exp_track_visual.md Normal file
View File

@@ -0,0 +1,69 @@
---
title: Viewing Experiments
---
While an experiment is running, and any time after it finishes, results are tracked and can be visualized in the ClearML
Enterprise WebApp (UI).
In addition to all of **ClearML**'s offerings, ClearML Enterprise keeps track of the Dataviews associated with an
experiment, which can be viewed and [modified](webapp_exp_modifying.md) in the WebApp.
## Viewing an experiment's Dataviews
In an experiment's page, go to the **DATAVIEWS** tab to view all the experiment's Dataview details, including:
* Input data [selection](#dataset-versions) and [filtering](#filtering)
* ROI [mapping](#mapping) (label translation)
* [Label enumeration](#label-enumeration)
* On-the-fly [data augmentation](#augmentation)
* [Iteration controls](#iteration-control)
![image](../../img/hyperdatasets/web-app/dataview_tab.png)
### Input
SingleFrames are iterated from the Dataset versions specified in the **INPUT** area, in the **SELECTED DATAVIEW** drop-down
menu.
### Filtering
The **FILTERING** section lists the SingleFrame filters iterated by a Dataview, applied to the experiment data.
Each frame filter is composed of:
* A Dataset version to input from
* ROI Rules for SingleFrames to include and / or exclude certain criteria.
* Weights for debiasing input data.
Combinations of frame filters can implement complex querying.
For more detailed information, see [Filtering](../dataviews.md#filtering).
### Mapping
ROI label mapping (label translation) applies to the new model. For example, use ROI label mapping to accomplish the following:
* Combine several labels under another more generic label.
* Consolidate disparate datasets containing different names for the ROI.
* Hide labeled objects from the training process.
For detailed information, see [Mapping ROI labels](../dataviews.md#mapping-roi-labels).
### Label enumeration
Assign label enumeration in the **LABELS ENUMERATION** area.
### Augmentation
On-the-fly data augmentation applied to SingleFrames, which does not create new data. Apply data Augmentation in steps,
where each step is composed of a method, an operation, and a strength.
For detailed information, see [Data augmentation](../dataviews.md#data-augmentation).
### Iteration control
The input data iteration control settings determine the order, number, timing, and reproducibility of the Dataview iterating
SingleFrames. Depending upon the combination of iteration control settings, all SingleFrames may not be iterated, and some may repeat.
For detailed information, see [Iteration control](../dataviews.md#iteration-control).