mirror of
https://github.com/clearml/clearml-docs
synced 2025-04-02 12:21:08 +00:00
Disambiguate dataset and hyper-dataset (#88)
This commit is contained in:
parent
21071e9a07
commit
f2162ba9e5
@ -3,7 +3,7 @@ title: Projects
|
||||
---
|
||||
|
||||
Projects are contextual containers for [tasks](task.md) and [models](artifacts.md#models) (as well as [dataviews](../hyperdatasets/dataviews.md)
|
||||
when hyperdatasets are enabled), providing a logical structure similar to file system folders.
|
||||
when Hyper-Datasets are enabled), providing a logical structure similar to file system folders.
|
||||
An often useful method is to categorize components into projects according to models or objectives.
|
||||
Grouping into projects helps in identifying tasks, models, and dataviews when queried.
|
||||
|
||||
|
@ -5,7 +5,7 @@ title: Annotations
|
||||
With ClearML Enterprise, annotations can be applied to video and image frames. [Frames](single_frames.md) support
|
||||
two types of annotations: **Frame objects** and **Frame labels**.
|
||||
|
||||
Annotation Tasks can be used to efficiently organize the annotation of frames in Dataset versions (see
|
||||
Annotation Tasks can be used to efficiently organize the annotation of frames in Hyper-Dataset versions (see
|
||||
[Annotations Task Page](webapp/webapp_annotator.md)).
|
||||
|
||||
For information about how to view, create, and manage annotations using the WebApp, see [Annotating Images and Videos](webapp/webapp_annotator.md#annotating-images-and-video).
|
||||
|
@ -1,8 +1,8 @@
|
||||
---
|
||||
title: Hyper Datasets
|
||||
title: Hyper-Datasets
|
||||
---
|
||||
|
||||
ClearML's Hyper Datasets are an MLOps-oriented abstraction of your data, which facilitates traceable, reproducible model development
|
||||
ClearML's Hyper-Datasets are an MLOps-oriented abstraction of your data, which facilitates traceable, reproducible model development
|
||||
through parametrized data access and meta-data version control.
|
||||
|
||||
The basic premise is that a user-formed query is a full representation of the dataset used by the ML/DL process.
|
||||
|
@ -4,7 +4,7 @@ title: SingleFrames
|
||||
|
||||
|
||||
A `SingleFrame` contains metadata pointing to raw data, and other metadata and data, which supports experimentation and
|
||||
ClearML Enterprise's Git-like Dataset versioning.
|
||||
ClearML Enterprise's Git-like Hyper-Dataset versioning.
|
||||
|
||||
## Frame Components
|
||||
|
||||
@ -42,7 +42,7 @@ For more information, see [Masks](masks.md).
|
||||
|
||||
### Previews
|
||||
`previews` is a dictionary containing metadata for optional thumbnail images that can be used in the ClearML Enterprise WebApp (UI)
|
||||
to view selected images in a Dataset. `previews` includes the `uri` of the thumbnail image.
|
||||
to view selected images in a Hyper-Dataset. `previews` includes the `uri` of the thumbnail image.
|
||||
|
||||
For more information, see [Previews](previews.md).
|
||||
|
||||
@ -63,9 +63,9 @@ The panel below describes the details contained within a `frame`:
|
||||
* `id` (*string*) - The unique ID of this frame.
|
||||
* `blob` (*string*) - Raw data.
|
||||
* `context_id` (*string*) - Source URL.
|
||||
* `dataset` (*dict*) - The Dataset and version containing the frame.
|
||||
* `dataset` (*dict*) - The Hyper-Dataset and version containing the frame.
|
||||
|
||||
* `id` - ID of the Dataset.
|
||||
* `id` - ID of the Hyper-Dataset.
|
||||
* `version` - ID of the version.
|
||||
|
||||
* `meta` (*dict*) - Frame custom metadata. Any custom key-value pairs (`sources` and `rois` can also contain a meta
|
||||
|
@ -2,11 +2,11 @@
|
||||
title: Tasks
|
||||
---
|
||||
|
||||
Hyper Datasets extend the **ClearML** [**Task**](../fundamentals/task.md) with [Dataviews](dataviews.md)
|
||||
Hyper-Datasets extend the **ClearML** [**Task**](../fundamentals/task.md) with [Dataviews](dataviews.md)
|
||||
|
||||
## Usage
|
||||
|
||||
Hyper datasets are supported by the `allegroai` python package.
|
||||
Hyper-Datasets are supported by the `allegroai` python package.
|
||||
|
||||
### Connecting Dataviews to a Task
|
||||
|
||||
|
@ -2,7 +2,7 @@
|
||||
title: Working with Frames
|
||||
---
|
||||
|
||||
View and edit SingleFrames in the Dataset page. After selecting a Dataset version, the **Version Browser** shows a sample
|
||||
View and edit SingleFrames in the Dataset page. After selecting a Hyper-Dataset version, the **Version Browser** shows a sample
|
||||
of frames and enables viewing SingleFrames and FramesGroups, and edit SingleFrames, in the [frame viewer](#frame-viewer).
|
||||
Before opening the frame viewer, you can filter the frames by applying [simple](#simple-frame-filtering) or [advanced](#advanced-frame-filtering)
|
||||
filtering logic.
|
||||
@ -19,7 +19,7 @@ frame metadata, the raw data source URI, as well as providing navigation and vie
|
||||
|
||||
### Frame Viewer Controls
|
||||
|
||||
Use frame viewer controls to navigate between frames in a Dataset Version, and control frame changes and viewing.
|
||||
Use frame viewer controls to navigate between frames in a Hyper-Dataset Version, and control frame changes and viewing.
|
||||
|
||||
|Control Icon|Actions|Shortcut|
|
||||
|-----|------|----|
|
||||
@ -89,7 +89,7 @@ To save frames changes at any time, click **SAVE** (below the annotation list ar
|
||||
### Viewing FrameGroups
|
||||
|
||||
Viewing and editing frames in a FrameGroup is similar to viewing and editing SingleFrames.
|
||||
Click the FrameGroup in the Dataset. In the frame viewer, select SingleFrame to view / modify from
|
||||
Click the FrameGroup in the Hyper-Dataset. In the frame viewer, select SingleFrame to view / modify from
|
||||
a dropdown list in the **Current Source** section.
|
||||
|
||||

|
||||
|
@ -17,14 +17,14 @@ enables modifying Dataviews, including:
|
||||
|
||||
|
||||
The selection and control of input data can be modified in *Draft* experiments that are not [development experiments](../task.md#development-experiments).
|
||||
Do this by modifying the Dataview used by the experiment. The Dataview specifies the Dataset versions from which frames
|
||||
Do this by modifying the Dataview used by the experiment. The Dataview specifies the Hyper-Dataset versions from which frames
|
||||
are iterated and frame filters (see [Dataviews](webapp_dataviews.md).
|
||||
|
||||
**To choose a Dataview**, do any of the following:
|
||||
|
||||
* Create a new Dataview
|
||||
|
||||
* Click **+** and then follow the instructions below to select Dataset versions, filter frames, map labels (label translation),
|
||||
* Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation),
|
||||
and set label enumeration, data augmentation, and iteration controls.
|
||||
|
||||
* Select a different Dataview already associated with the experiment.
|
||||
@ -72,7 +72,7 @@ For more detailed information, see [Filtering](../dataviews.md#filtering).
|
||||
1. In the **FILTERING** area, click **EDIT**.
|
||||
1. For each frame filter:
|
||||
|
||||
1. Select the Dataset version to which the frame filter applies.
|
||||
1. Select the Hyper-Dataset version to which the frame filter applies.
|
||||
1. Add, change, or remove any combination of the following rules:
|
||||
|
||||
* ROI rule - Include or exclude frames containing any single ROI with any combination of labels in the Dataset
|
||||
@ -95,7 +95,7 @@ that are not mapped are ignored.
|
||||
1. In the **MAPPING** section, click **EDIT**
|
||||
* Add (**+**) or edit a mapping:
|
||||
|
||||
1. Select the Dataset and version whose labels will be mapped.
|
||||
1. Select the Hyper-Dataset and version whose labels will be mapped.
|
||||
|
||||
1. Select one or more labels to map.
|
||||
|
||||
@ -123,7 +123,7 @@ Modify the label enumeration assigned to output models.
|
||||
|
||||
## Data Augmentation
|
||||
|
||||
Modify the on-the-fly data augmentation applied to frames input from the select Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
|
||||
Modify the on-the-fly data augmentation applied to frames input from the select Hyper-Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength.
|
||||
|
||||
For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user