clearml-docs/docs/hyperdatasets/webapp/webapp_exp_modifying.md

169 lines
6.3 KiB
Markdown
Raw Normal View History

2021-06-20 22:00:16 +00:00
---
title: Modifying Dataviews
---
2021-06-21 18:13:17 +00:00
An experiment that has been executed can be [cloned](../../webapp/webapp_exp_reproducing.md), then the cloned experiment's
execution details can be modified, and the modified experiment can be executed.
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
In addition to all the [**ClearML** tuning capabilities](../../webapp/webapp_exp_tuning.md), the **ClearML Enterprise WebApp** (UI)
enables modifying Dataviews, including:
2021-06-20 22:00:16 +00:00
* [Selected Dataview](#selected-dataview)
* [Dataset versions](#selecting-dataset-versions)
* [Frame filtering](#filtering-frames)
* [Label mapping](#mapping-labels-label-translation)
* [Class label enumeration](#label-enumeration)
* [Data augmentation](#data-augmentation)
* [Input frame iteration controls](#iteration-controls)
2021-06-21 18:13:17 +00:00
The selection and control of input data can be modified in *Draft* experiments that are not [development experiments](../task.md#development-experiments).
Do this by modifying the Dataview used by the experiment. The Dataview specifies the Dataset versions from which frames
are iterated and frame filters (see [Dataviews](webapp_dataviews.md).
2021-06-20 22:00:16 +00:00
**To choose a Dataview**, do any of the following:
2021-06-21 18:13:17 +00:00
* Create a new Dataview
* Click **+** and then follow the instructions below to select Dataset versions, filter frames, map labels (label translation),
2021-06-20 22:00:16 +00:00
and set label enumeration, data augmentation, and iteration controls.
2021-06-21 18:13:17 +00:00
* Select a different Dataview already associated with the experiment.
* In the **SELECTED DATAVIEW** list, choose a Dataview.
* Import a different Dataview associated with the same or another project.
* Click <img src="/docs/latest/icons/ico-import.svg" className="icon size-md space-sm" /> (**Import dataview**) and then
2021-06-20 22:00:16 +00:00
select **Import to current dataview** or **Import to aux dataview**.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
:::note
2021-06-21 18:13:17 +00:00
After importing a Dataview, it can be renamed and / or removed.
2021-06-20 22:00:16 +00:00
:::
2021-09-09 10:17:46 +00:00
### Selecting Dataset Versions
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
To input data from a different data source or different version of a data source, select a different Dataset version used
2021-06-20 22:00:16 +00:00
by the Dataview.
**To select Dataset versions for input data:**
1. In the **INPUT** area, click **EDIT**.
1. Do any of the following:
* Add a Dataset version - Input frames from another a version of another Dataset.
2021-06-21 18:13:17 +00:00
* Click **+**
* Select a Dataset and a Dataset version
2021-06-20 22:00:16 +00:00
* Remove a Dataset version - Do not input frames from a Dataset version.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
Select frames from as many Dataset versions as are needed.
1. Click **SAVE**.
2021-06-21 18:13:17 +00:00
2021-09-09 10:17:46 +00:00
## Filtering Frames
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
Filtering of SingleFrames iterated by a Dataview for input to the experiment is accomplished by frame filters.
2021-06-20 22:00:16 +00:00
For more detailed information, see [Filtering](../dataviews.md#filtering).
**To modify frame filtering:**
1. In the **FILTERING** area, click **EDIT**.
1. For each frame filter:
1. Select the Dataset version to which the frame filter applies.
1. Add, change, or remove any combination of the following rules:
2021-06-21 18:13:17 +00:00
* ROI rule - Include or exclude frames containing any single ROI with any combination of labels in the Dataset
2021-06-20 22:00:16 +00:00
version. Specify a range of the number of matching ROI (instances) per frame, and a range of confidence levels.
* Frame rule - Filter by frame metadata key-value pairs, or ROI labels.
* Source rule - Filter by frame `source` dictionary key-value pairs.
2021-06-21 18:13:17 +00:00
1. Optionally, debias input data by setting ratios for frames returned by the Dataview for each frame filter. These
2021-06-20 22:00:16 +00:00
ratios allow adjusting an imbalance in input data.
1. Click **SAVE**.
2021-06-21 18:13:17 +00:00
2021-09-09 10:17:46 +00:00
## Mapping Labels (Label Translation)
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
Modify the ROI label mapping rules, which translate one or more input labels to another label for the output model. Labels
that are not mapped are ignored.
2021-06-20 22:00:16 +00:00
**To modify label mapping:**
1. In the **MAPPING** section, click **EDIT**
* Add (**+**) or edit a mapping:
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select the Dataset and version whose labels will be mapped.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select one or more labels to map.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select or enter the label to map to in the output model.
2021-06-21 18:13:17 +00:00
2021-06-21 12:23:21 +00:00
* Remove (<img src="/docs/latest/icons/ico-trash.svg" className="icon size-md space-sm" />) a mapping.
2021-06-20 22:00:16 +00:00
1. Click **SAVE**
2021-09-09 10:17:46 +00:00
## Label Enumeration
2021-06-20 22:00:16 +00:00
Modify the label enumeration assigned to output models.
**To modify label enumeration:**
1. In the **LABELS ENUMERATION** section, click **EDIT**.
* Add (**+**) or edit an enumeration:
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
* Select a label and then enter an integer for it.
2021-06-21 18:13:17 +00:00
2021-06-21 12:23:21 +00:00
* Remove (<img src="/docs/latest/icons/ico-trash.svg" className="icon size-md space-sm" />) an enumeration.
2021-06-20 22:00:16 +00:00
1. Click **SAVE**.
2021-09-09 10:17:46 +00:00
## Data Augmentation
2021-06-20 22:00:16 +00:00
Modify the on-the-fly data augmentation applied to frames input from the select Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
**To modify data augmentation**
1. In the **AUGMENTATION** section, click **EDIT**.
* Add (**+**) or edit an augmentation step - Select a **METHOD**, **OPERATION**, and **STRENGTH**.
2021-06-21 18:13:17 +00:00
2021-06-21 12:23:21 +00:00
* Remove (<img src="/docs/latest/icons/ico-trash.svg" className="icon size-md space-sm" />) an augmentation step.
2021-06-21 18:13:17 +00:00
1. Click **SAVE**.
2021-09-09 10:17:46 +00:00
## Iteration Controls
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
Modify the frame iteration performed by the Dataview to control the order, number, timing, and reproducibility of frames
2021-06-20 22:00:16 +00:00
for training.
For more detailed information, see [Iteration Control](../dataviews.md#iteration-control).
**To modify iteration controls:**
1. In the **ITERATION** sections, click **EDIT**.
1. Select the **ORDER** of the SingleFrames returned by the iteration, either:
* **Sequential** - Iterate SingleFrames in sorted order by context ID and timestamp.
* **Random** - Iterate SingleFrames randomly using the random seed you can set (see Random Seed below).
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select the frame **REPETITION** option, either:
2021-06-21 18:13:17 +00:00
* **Use Each Frame Once**
2021-06-20 22:00:16 +00:00
2021-06-21 18:13:17 +00:00
* **Limit Frames**
2021-06-20 22:00:16 +00:00
* **Infinite Iterations**
2021-06-21 18:13:17 +00:00
2021-06-20 22:00:16 +00:00
1. Select the **RANDOM SEED** - If the experiment is rerun and the seed remains unchanged, the frames iteration is the same.
1. For video, enter a **CLIP LENGTH** - For video data sources, in the number of sequential frames from a clip to iterate.
1. Click **SAVE**.