clearml-docs/docs/hyperdatasets/webapp/webapp_exp_modifying.md

166 lines
6.0 KiB
Markdown
Raw Normal View History

2021-06-20 22:00:16 +00:00
---
title: Modifying Dataviews
---
2021-06-21 18:13:17 +00:00
An experiment that has been executed can be [cloned](../../webapp/webapp_exp_reproducing.md), then the cloned experiment's
execution details can be modified, and the modified experiment can be executed.
2021-06-20 22:00:16 +00:00
In addition to all the [ClearML tuning capabilities](../../webapp/webapp_exp_tuning.md), the **ClearML Enterprise WebApp** (UI)
2021-12-02 17:53:37 +00:00
enables modifying [Dataviews](webapp_dataviews.md), including:
* [Selected Dataview](#selecting-dataviews)
2021-06-20 22:00:16 +00:00
* [Dataset versions](#selecting-dataset-versions)
* [Frame filtering](#filtering-frames)
* [Label mapping](#mapping-labels-label-translation)
* [Class label enumeration](#label-enumeration)
* [Data augmentation](#data-augmentation)
* [Input frame iteration controls](#iteration-controls)
2021-12-02 17:53:37 +00:00
## Selecting Dataviews
2021-06-20 22:00:16 +00:00
**To choose a Dataview**, do any of the following:
2021-06-21 18:13:17 +00:00
* Create a new Dataview
* Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation),
2021-06-20 22:00:16 +00:00
and set label enumeration, data augmentation, and iteration controls.
2021-06-21 18:13:17 +00:00
* Select a different Dataview already associated with the experiment.
* In the **SELECTED DATAVIEW** list, choose a Dataview.
* Import a different Dataview associated with the same or another project.
2021-12-02 17:53:37 +00:00
* Click <img src="/docs/latest/icons/ico-import.svg" alt="Import" className="icon size-md space-sm" /> (**Import dataview**) and then
select **Import to current dataview** or **Import as aux dataview**.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
:::note
2023-04-04 13:16:54 +00:00
After importing a Dataview, it can be renamed and/or removed.
2021-06-20 22:00:16 +00:00
:::
2021-09-09 10:17:46 +00:00
### Selecting Dataset Versions
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
To input data from a different data source or different version of a data source, select a different Dataset version used
2021-06-20 22:00:16 +00:00
by the Dataview.
**To select Dataset versions for input data:**
1. In the **INPUT** area, click **EDIT**.
1. Do any of the following:
2023-05-14 06:51:01 +00:00
* Add a Dataset version - Input frames from another version of another Dataset.
2021-06-21 18:13:17 +00:00
* Click **+**
* Select a Dataset and a Dataset version
2021-06-20 22:00:16 +00:00
* Remove a Dataset version - Do not input frames from a Dataset version.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
Select frames from as many Dataset versions as are needed.
1. Click **SAVE**.
2021-06-21 18:13:17 +00:00
2021-09-09 10:17:46 +00:00
## Filtering Frames
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
Filtering of SingleFrames iterated by a Dataview for input to the experiment is accomplished by frame filters.
2021-06-20 22:00:16 +00:00
For more detailed information, see [Filtering](../dataviews.md#filtering).
**To modify frame filtering:**
1. In the **FILTERING** area, click **EDIT**.
1. For each frame filter:
1. Select the Hyper-Dataset version to which the frame filter applies.
2021-06-20 22:00:16 +00:00
1. Add, change, or remove any combination of the following rules:
2021-06-21 18:13:17 +00:00
* ROI rule - Include or exclude frames containing any single ROI with any combination of labels in the Dataset
2021-06-20 22:00:16 +00:00
version. Specify a range of the number of matching ROI (instances) per frame, and a range of confidence levels.
* Frame rule - Filter by frame metadata key-value pairs, or ROI labels.
* Source rule - Filter by frame `source` dictionary key-value pairs.
2021-06-21 18:13:17 +00:00
1. Optionally, debias input data by setting ratios for frames returned by the Dataview for each frame filter. These
2021-06-20 22:00:16 +00:00
ratios allow adjusting an imbalance in input data.
1. Click **SAVE**.
2021-06-21 18:13:17 +00:00
2021-09-09 10:17:46 +00:00
## Mapping Labels (Label Translation)
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
Modify the ROI label mapping rules, which translate one or more input labels to another label for the output model. Labels
that are not mapped are ignored.
2021-06-20 22:00:16 +00:00
**To modify label mapping:**
1. In the **MAPPING** section, click **EDIT**
* Add (**+**) or edit a mapping:
2021-06-21 18:13:17 +00:00
1. Select the Hyper-Dataset and version whose labels will be mapped.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select one or more labels to map.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select or enter the label to map to in the output model.
2021-06-21 18:13:17 +00:00
2021-12-02 17:53:37 +00:00
* Remove (<img src="/docs/latest/icons/ico-trash.svg" alt="Trash" className="icon size-md space-sm" />) a mapping.
2021-06-20 22:00:16 +00:00
1. Click **SAVE**
2021-09-09 10:17:46 +00:00
## Label Enumeration
2021-06-20 22:00:16 +00:00
Modify the label enumeration assigned to output models.
**To modify label enumeration:**
1. In the **LABELS ENUMERATION** section, click **EDIT**.
* Add (**+**) or edit an enumeration:
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
* Select a label and then enter an integer for it.
2021-06-21 18:13:17 +00:00
2021-12-02 17:53:37 +00:00
* Remove (<img src="/docs/latest/icons/ico-trash.svg" alt="Trash" className="icon size-md space-sm" />) an enumeration.
2021-06-20 22:00:16 +00:00
1. Click **SAVE**.
2021-09-09 10:17:46 +00:00
## Data Augmentation
2021-06-20 22:00:16 +00:00
2021-12-02 17:53:37 +00:00
Modify the on-the-fly data augmentation applied to frame input from the select Hyper-Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
**To modify data augmentation**
1. In the **AUGMENTATION** section, click **EDIT**.
* Add (**+**) or edit an augmentation step - Select a **METHOD**, **OPERATION**, and **STRENGTH**.
2021-06-21 18:13:17 +00:00
2021-12-02 17:53:37 +00:00
* Remove (<img src="/docs/latest/icons/ico-trash.svg" alt="Trash" className="icon size-md space-sm" />) an augmentation step.
2021-06-21 18:13:17 +00:00
1. Click **SAVE**.
2021-09-09 10:17:46 +00:00
## Iteration Controls
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames
2021-06-20 22:00:16 +00:00
for training.
For more detailed information, see [Iteration Control](../dataviews.md#iteration-control).
**To modify iteration controls:**
1. In the **ITERATION** sections, click **EDIT**.
1. Select the **ORDER** of the SingleFrames returned by the iteration, either:
* **Sequential** - Iterate SingleFrames in sorted order by context ID and timestamp.
* **Random** - Iterate SingleFrames randomly using the random seed you can set (see Random Seed below).
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select the frame **REPETITION** option, either:
2021-06-21 18:13:17 +00:00
* **Use Each Frame Once**
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
* **Limit Frames**
2021-06-20 22:00:16 +00:00
* **Infinite Iterations**
2021-06-21 18:13:17 +00:00
2021-12-02 17:53:37 +00:00
1. Select the **RANDOM SEED** - If the experiment is rerun and the seed remains unchanged, the frame iteration is the same.
2021-06-20 22:00:16 +00:00
1. For video, enter a **CLIP LENGTH** - For video data sources, in the number of sequential frames from a clip to iterate.
1. Click **SAVE**.