Update docs (#153)

* add info for clear filter, float behavior, and tag exclusion

* add data examples to guides

* fix typo

* fix filtering wording, and float admonition title

* add abort all children action

* dataset metadata initial

* fix expand icon

* initial new exp comparison window

* initial scalar full screen

* fix typo

* edit dataset version metada

* edit exp comparison and add missing alts to icons

* add info about dataset-level datasets

* add info about full screen mode

* HOCON >>> JSON

* HOCON >>> JSON

* dataset card edits

* edit fullscreen scalar mode

* edit fullscreen scalar plots

* edit pipeline example based on code fixes

* full screen scalar edit

* add context navigation

* Add experiment selection info

* experiment selection edit

* custom ui plugin

* datasets and versioning

* fix link
This commit is contained in:
pollfly
2022-01-10 11:45:29 +02:00
committed by GitHub
parent 8339e4902f
commit eb91aaa361
14 changed files with 416 additions and 140 deletions

View File

@@ -9,25 +9,11 @@ and functionality for the following purposes:
* Integrating the powerful features of [Dataviews](dataviews.md) with an experiment
* [Annotating](webapp/webapp_datasets_frames.md#annotations) images and videos
Datasets consist of versions with SingleFrames and / or FrameGroups. Each Dataset can contain multiple versions, where
each version can have multiple children that inherit their parent's SingleFrames and / or FrameGroups. This inheritance
includes the frame metadata and data connecting the source data to the ClearML Enterprise platform, as well as the other
metadata and data.
Datasets consist of versions with SingleFrames and / or FrameGroups. Each Dataset can contain multiple versions, which
can have multiple children that inherit their parent's contents.
These parent-child version relationships can be represented as version trees with a root-level parent. A Dataset
can contain one or more trees.
Mask-labels can be defined globally, for a DatasetVersion, which will be applied to all masks in that version.
## Dataset Version State
Dataset versions can have either **Draft** or **Published** status.
A **Draft** version is editable, so frames can be added to and deleted and / or modified from the Dataset.
A **Published** version is read-only, which ensures reproducible experiments and preserves a version of a Dataset.
Child versions can only be created from *Published* versions. To create a child of a *Draft* Dataset version,
it must be published first.
Mask-labels can be defined globally, for a DatasetVersion. When defined this way, they will be applied to all masks in
that version.
## Example Datasets
@@ -123,19 +109,31 @@ Dataset.delete(dataset_name='MyDataset', delete_all_versions=True, force=True)
Dataset versioning refers to the group of ClearML Enterprise SDK and WebApp (UI) features for creating,
modifying, and deleting Dataset versions.
ClearML Enterprise supports simple and sophisticated Dataset versioning, including **simple version structures** and
**advanced version structures**.
ClearML Enterprise supports simple and advanced Dataset versioning paradigms. A **simple version structure** consists of
a single evolving version, with historic static snapshots. Continuously push your changes to your single dataset version,
and take a snapshot to record the content of your dataset at a specific point in time.
In a **simple version structure**, a parent can have one and only one child, and the last child in the Dataset versions
tree must be a *Draft*. This simple structure allows working with a single set of versions of a Dataset. Create children
and publish versions to preserve data history. Each version whose status is *Published* in a simple version structure is
referred to as a **snapshot**.
You can, alternatively, employ any **advanced structure**, where each version evolves in parallel, and you control which
versions are locked for further changes and which can be modified. See details [below](#dataset-version-structure).
In an **advanced version structure**, at least one parent has more than one child (this can include more than one parent
version at the root level), or the last child in the Dataset versions tree is *Published*.
## Dataset Version State
Dataset versions can have either *Draft* or *Published* state.
A *Draft* version is editable, so frames can be added to and deleted and / or modified.
A *Published* version is read-only, which ensures reproducible experiments and preserves the Dataset version contents.
Child versions can only be created from *Published* versions, as they inherit their predecessor version contents.
## Dataset Version Structure
To implement a simple version structure, where the dataset is ever evolving, with a linear set of historic snapshots,
a parent version can have one and only one child, with the last child in the Dataset versions tree in *Draft* state.
Different version structures, such as where at least one parent has more than one child, or the single last child in the
Dataset versions tree is *Published* are considered advanced version structures.
For details about programmatically implementing simple and advanced version structures, see [Creating Snapshots](#creating-snapshots)
and [Creating Child Versions](#creating-child-versions) respectively below.
Creating a version in a simple version structure may convert it to an advanced structure. This happens when creating
a Dataset version that yields a parent with two children, or when publishing the last child version.
## DatasetVersion Usage

View File

@@ -12,20 +12,42 @@ The Datasets page offers the following functionalities:
## Dataset Cards
Dataset cards show summary information about versions, frames, and labels in a Dataset, and the elapsed time since the Dataset was last update and the user doing the update. Dataset cards allow you to open a specific Dataset to perform Dataset versioning and frames management.
Dataset cards show summary information about versions, frames, and labels in a Dataset, and the elapsed time since the Dataset was last update and the user doing the update.
Dataset cards allow you to open a specific Dataset to perform Dataset versioning and frames management.
* Dataset name
* Elapsed time since the last update. Hover over elapsed time and view date of last update.
* User updating the Dataset
* If the dataset contains dataset-level metadata, the card displays the <img src="/docs/latest/icons/ico-status-completed.svg" alt="Check mark" className="icon size-md space-sm" />
`Metadata` indicator, which is also a shortcut to [edit the Dataset's metadata](#editing-dataset-level-metadata)
* The number of versions in the Dataset
* The total number of frames in all versions of the Dataset. If an asterisk (\*) appears next to **FRAMES**, then you can hover it and see the name of the version whose frames were last updated appears.
* The percentage of frames annotated in all versions of the Dataset. If an asterisk (\*) appears next to **ANNOTATED**, then you can hover it and see the name of the version whose frames were last annotated appears.
* If the Dataset version's status is *Published*, then the top labels in the Dataset, color coded (colors are editable) appear. If the Dataset version is Draft, then no labels appear.
* If the Dataset version's status is *Published*, then the Dataset's top labels appear (colors are editable). If the
Dataset version is *Draft*, then no labels appear.
:::note
:::note Change Label Color
To change the label color coding, hover over a label color, click thr hand pointer, and then select a new color.
:::
### Renaming a Dataset
1. Click <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Bars menu" className="icon size-md space-sm" />
1. Click **Rename** <img src="/docs/latest/icons/ico-edit.svg" alt="Edit pencil" className="icon size-md space-sm" />
1. Edit the name
1. Click <img src="/docs/latest/icons/ico-save.svg" alt="Check mark" className="icon size-md space-sm" />
### Editing Dataset-level Metadata
To edit the dataset-level metadata
1. Open the metadata edit window in one the following ways:
* Click <img src="/docs/latest/icons/ico-status-completed.svg" alt="Check mark" className="icon size-md space-sm" />
`Metadata`
* Click <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Bars menu" className="icon size-md space-sm" />, then click **Edit Metadata** <img src="/docs/latest/icons/ico-metadata.svg" alt="Edit metadata" className="icon size-md space-sm" />
1. Edit the section contents (JSON format)
1. Click **OK**
## Creating New Datasets
Create a new Dataset which will contain one version named `Current`. The new version will not contain any frames.

View File

@@ -37,7 +37,8 @@ Use frame viewer controls to navigate between frames in a Hyper-Dataset Version,
|<img src="/docs/latest/icons/ico-zoom-out.svg" alt="Zoom out icon" className="icon size-md space-sm" />|Zoom out| **-** or Ctrl + Mouse wheel |
|Percentage textbox|Zoom percentage| <img src="/docs/latest/icons/ico-optional-no.svg" alt="Not applicable" className="icon size-md center-md" /> |
#### Additional keyboard shortcuts
#### Additional Keyboard Shortcuts
**General Controls**
@@ -67,7 +68,7 @@ Use frame viewer controls to navigate between frames in a Hyper-Dataset Version,
**To view / edit a frame in the frame editor**
1. Locate your frame by applying a [simple frame filter](#simple-frame-filtering) or [advanced frame filter](#advanced-frame-filtering), and clicking <span class="tr_gui">LOAD MORE</span>, if required.
1. Locate your frame by applying a [simple frame filter](#simple-frame-filtering) or [advanced frame filter](#advanced-frame-filtering), and clicking **LOAD MORE**, if required.
1. Click the frame thumbnail. The frame editor appears.
1. Do any of the following:
* View frame details, including:
@@ -78,10 +79,10 @@ Use frame viewer controls to navigate between frames in a Hyper-Dataset Version,
* Annotations
* Frame objects - Labeled Regions of Interest, with confidence levels and custom metadata per frame object.
* Frame labels - Labels applied to the entire frame, not a region in the frame.
* Optionally, filter annotations by confidence level using the <span class="tr_gui">Minimum confidence</span> slider.
* Optionally, filter annotations by confidence level using the **Minimum confidence** slider.
* Add, change, and delete [annotations](#annotations) and [frame metadata](#frame-metadata).
:::important
:::important Saving Frame Changes
To save frames changes at any time, click **SAVE** (below the annotation list area).
:::
@@ -315,7 +316,7 @@ You can add annotations by drawing new bounding areas, and copying existing anno
1. Expand the **FRAME LABELS** area.
1. Click **+ Add new**.
1. Enter a label.
1. Optionally, click <img src="/docs/latest/icons/ico-edit.svg" className="icon size-md space-sm" />.
1. Optionally, click <img src="/docs/latest/icons/ico-edit.svg" alt="Edit pencil" className="icon size-md space-sm" />.
### Annotation Management

View File

@@ -1,66 +1,57 @@
---
title: Dataset Versioning
title: Dataset Versions
---
Use the Dataset versioning WebApp (UI) features for viewing, creating, modifying, and
deleting Dataset versions.
deleting [Dataset versions](../dataset.md#dataset-versioning).
From the Datasets page, click on one of the Datasets in order to see and work with its versions.
From the [Datasets page](webapp_datasets.md), click on one of the Datasets in order to see and work with its versions.
### Viewing Snapshots
View snapshots in the simple version structure using either:
* The simple view, a table of snapshots.
## Dataset Version History
The WebApp (UI) will present your dataset version structure in one of two ways, depending on the structure implemented.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Simple view (snapshot table)</summary>
<div className="cml-expansion-panel-content">
## Simple View
While your dataset maintains a simple (linear) [version structure](../dataset.md#dataset-version-structure), the WebApp
(UI) will present the version history as a table of historic snapshots:
![image](../../img/hyperdatasets/web-app/dataset_simple_adv_01.png)
</div>
</details>
<br/>
* The advanced view, a tree of versions. The tree contains one version whose status is <i>Draft</i>, and snapshots appear in
chronological order, with oldest at the top, and the most recent at the bottom.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Advanced view (version tree)</summary>
<div className="cml-expansion-panel-content">
![image](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
</div>
</details>
### Creating Snapshots
To create a snapshot, you must be in the simple (version table) view.
**To create a snapshot, do the following:**
1. If you are in the advanced view, click **Switch to Simple View** (In certain situations, this may not be possible,
see [Dataset Versioning](../dataset.md#dataset-versioning))
1. If the **DATASET HISTORY** section is not opened, click it.
1. If a snapshot is currently selected, click **RETURN TO CURRENT VERSION**.
1. Click **+ CREATE SNAPSHOT**.
1. Enter a version name, and optionally a description.
1. Click **CREATE**.
:::note
The WebApp (UI) does not currently support the automatic naming of snapshots with timestamps appended. You must provide a snapshot name.
:::note VERSION BROWSING
Snapshots are always taken of the current version. If you are currently viewing a previous snapshot, click
**RETURN TO CURRENT VERSION** to make snapshot creation available.
:::
You can view your datasets explicit version structure and create a more complex structure by switching to the advanced
view.
### Modifying Snapshot Name
To modify a Dataset version name:
1. Click the snapshot name on the top left of the dataset page
1. Edit the name,
1. Click <img src="/docs/latest/icons/ico-save.svg" alt="check" className="icon size-md space-sm" /> (check).
## Advanced View
When employing a non-trivial [version structure](../dataset.md#dataset-version-structure) of your dataset, the WebApp
(UI) presents the version history as a tree of versions.
![image](../../img/hyperdatasets/web-app/dataset_simple_adv_02.png)
### Creating Versions
To create a version, you must be in the advanced (version tree) view.
**To create a child version, do the following:**
1. If you are in the simple view, click **Switch to Advanced View**.
1. Click the (parent) version from which to create a child (inherit all frames).
1. Click **+ CREATE NEW VERSION**.
1. Enter a version name, and optionally a description.
@@ -68,46 +59,195 @@ To create a version, you must be in the advanced (version tree) view.
### Publishing Versions
Publish (make read-only) any Dataset version whose status is *Draft*. If the Dataset is in the simple structure,
and you publish the current version, then only the advanced view is available,
and you cannot create snapshots.
Publish (make read-only) any Dataset version whose status is *Draft*. If you publish a dataset's current version, the
dataset's structure will no longer be considered a simple version structure.
**To publish a version, do the following:**
1. If you are in the simple view, click **Switch to Advanced View**.
1. Click the version to publish.
1. Click **PUBLISH**.
1. Click **PUBLISH** again to confirm.
### Exporting Frames
Frame exports downloaded filtered frames as a JSON file.
**To export frames, do the following:**
* In the Thumbnails area, click **EXPORT FRAMES**. The frames JSON file downloads.
### Modifying Version Names
**To modify a Dataset version name, do the following:**
**To modify a Dataset version name:**
1. Click the dataset version name on the top left of the dataset page
1. Edit the name,
1. Click <img src="/docs/latest/icons/ico-save.svg" alt="check" className="icon size-md space-sm" /> (check).
* At the top right of the Dataset page, hover over the Dataset version name, click <img src="/docs/latest/icons/ico-edit.svg" className="icon size-md space-sm" /> , edit the name, and then click <img src="/docs/latest/icons/ico-save.svg" className="icon size-md space-sm" /> (check).
### Modifying Version Descriptions
**To modify a version description, do the following:**
* Expand the **INFO** area, hover over the **Description**, click <img src="/docs/latest/icons/ico-edit.svg" alt="Edit pencil" className="icon size-md space-sm" />,
edit the name, and then click <img src="/docs/latest/icons/ico-save.svg" alt="Check mark" className="icon size-md space-sm" /> (check).
### Deleting Versions
You can delete versions whose status is *Draft*.
**To delete the current version, do the following:**
1. Click the version to delete.
1. Click **DELETE**.
1. Click **DELETE** again to confirm.
## Version Information
Additional information about the selected dataset version is presented below the version selection panel in collapsible
panels:
### Version Info Panel
Provides general version information:
* Version ID
* Dataset ID
* Dataset name
* Status (*Draft* or *Published*)
* Creating user
* Version update time
* Version description.
**To modify a version description, do the following:**
1. Expand the **INFO** area
1. Hover over the **Description** element
1. Click <img src="/docs/latest/icons/ico-edit.svg" alt="Edit pencil" className="icon size-md space-sm" />
1. Edit the description
1. Click <img src="/docs/latest/icons/ico-save.svg" alt="Check mark" className="icon size-md space-sm" /> (check)
### Version Metadata Panel
Any version specific metadata, in JSON format.
The content icon color signifies whether any metadata is currently accompanying the dataset version.
**To edit a version's metadata:**
1. Click <img src="/docs/latest/icons/ico-metadata.svg" alt="Edit metadata" className="icon size-md space-sm" /> on the
**VERSION METADATA** panel to open the edit window
1. Edit the section contents (JSON format)
1. Click **OK**
### Version Label Statistics Panel
Label usage stats for this dataset version.
* Each label is listed along with the number of times it was used in the version
* The pie chart visualizes these stats. Hover over a chart slice and its associated label and usage
percentage will appear at the center of the chart.
## Version Frame Browser
The **Version Frame Browser** displays a preview of the contents of the selected dataset version.
The dataset version can be filtered by multiple criteria. The resulting frames can be exported as a JSON file.
To view further details about a specific frame, click on its preview, which will open the [Frame Viewer](webapp_datasets_frames.md#frame-viewer).
### Simple Frame Filtering
Simple frame filtering applies one annotation object (ROI) label and returns frames containing at least one annotation
with that label.
**To apply a simple frame filter,** select a label from the **LABEL FILTER** list.
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Simple filter example</summary>
<div className="cml-expansion-panel-content">
* Before filtering, the **Version Browser** in the image below contains seven frames.
![Unfiltered version browser](../../img/hyperdatasets/frame_filtering_01.png)
* A simple label filter for `person` shows three frames, each containing at least one ROI labeled `person`.
![Filtered version browser](../../img/hyperdatasets/frame_filtering_02.png)
</div>
</details>
### Advanced Frame Filtering
Advanced frame filtering applies sophisticated filtering logic, which is composed of as many frame filters as needed,
where each frame filter can be a combination of ROI, frame, and source rules.
* ROI rules use include and exclude logic to match frames by ROI label; an ROI label can match frames containing at least
one annotation object (ROI) with all labels in the rule.
* Frame rules and source rules use Lucene queries with AND, OR, and NOT logic. Frame rules apply to frame metadata.
* Source rules apply to frame source information.
**To apply advanced filters:**
1. In the **Version Browser**, click **Switch to advanced filters**.
1. In a **FRAME FILTER**, create one of the following rules:
* ROI rule
* Choose **Include** or **Exclude**, select ROI labels, and optionally set the confidence level range.
* To switch from the ROI dropdown list to a Lucene query mode, click <img src="/docs/latest/icons/ico-edit.svg" alt="Setting Gear" className="icon size-md space-sm" />.
* Frame rule - Enter a Lucene query using frame metadata fields in the format `meta.<key>:<value>`.
* Source rule - Enter a Lucene query using frame metadata fields in the format `sources.<key>:<value>`.
#### Filtering Examples
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">ROI Rules</summary>
<div className="cml-expansion-panel-content">
* Create one ROI rule for `person` shows the same three frames as the simple frame filter (above).
![Adding an ROI rule](../../img/hyperdatasets/frame_filtering_03.png)
* In the ROI rule, add a second label. Add `partially_occluded`. Only frames containing at least one ROI labeled as both
`person` and `partially_occluded` match the filter.
![Add label to ROI rule](../../img/hyperdatasets/frame_filtering_04.png)
* By opening a frame in the frame viewer, you can see an ROI labeled with both.
![Labeled ROIs in frame viewer](../../img/hyperdatasets/frame_filtering_05.png)
</div>
</details>
<br/>
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Frame Rules</summary>
<div className="cml-expansion-panel-content">
Filter by metadata using Lucene queries.
* Add a frame rule to filter by the metadata key `dangerous` for the value of `no`.
![Filter by metadata ](../../img/hyperdatasets/frame_filtering_08.png)
* Open a frame in the frame viewer to see its metadata.
![Frame metadata in frame viewer](../../img/hyperdatasets/frame_filtering_09.png)
</div>
</details>
<br/>
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Source Rules</summary>
<div className="cml-expansion-panel-content">
Filter by sources using Lucene queries.
* Add a source rule to filter for sources URIs with a wildcards.
![Filter by source](../../img/hyperdatasets/frame_filtering_10.png)
Lucene queries can also be used in ROI label filters and frame rules.
</div>
</details>
### Exporting Frames
To export (download) filtered datasets as a JSON file, click **EXPORT FRAMES**.
### Frame Browser Configuration
Click <img src="/docs/latest/icons/ico-settings.svg" alt="Setting Gear" className="icon size-md" /> to open the
frame browser configuration settings.
#### Grouping Previews
FrameGroups or SingleFrames can share the same `context_id` (URL). For example, users can set the same `context_id`
to multiple FrameGroups that represent frames in a single video.
Use the **Grouping** menu to select one of the following options:
* Split Preview - Show separate previews for each individual FrameGroup, regardless of shared context.
* Group by URL - Show a single preview for all FrameGroups with the same context
#### Preview Source
When using multi-source FrameGroups, users can choose which of the FrameGroups sources will be displayed as the preview.
Use the **PREVIEW SOURCE** menu to select from the list of sources.
Choose the `Default preview source` option to present the first available source.
:::note
If a FrameGroup doesn't have the selected preview source, the next available source will be used for the preview.
:::
* If you are in the simple view, click **Switch to Advanced View**.
* Click the version to delete.
* Click **DELETE**.
* Click **DELETE** again to confirm.

View File

@@ -53,6 +53,7 @@ The following table describes the actions that can be performed from the Datavie
| Archive | To more easily work with active Dataviews, move a Dataview to the archive. See [Archiving Dataviews](#archiving-dataviews). |
| Clone | Make an exact copy of a Dataview. |
| Move to project | To organize work and improve collaboration, move a Dataview to another project. |
| Custom action | When available, provides a mechanism to define your own custom action which will appear in the context menu. See [Custom UI Context Menu Actions](../../deploying_clearml/clearml_server_config.md#custom-ui-context-menu-actions).|
These actions can be accessed with the context menu (when right-clicking a Dataview or clicking the menu button <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Menu" className="icon size-md space-sm" />
in a Dataview's info panel).