From f2162ba9e58d7c7ace28213a876b14137206369c Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Thu, 21 Oct 2021 09:40:53 +0300 Subject: [PATCH] Disambiguate dataset and hyper-dataset (#88) --- docs/fundamentals/projects.md | 2 +- docs/hyperdatasets/annotations.md | 2 +- docs/hyperdatasets/overview.md | 4 ++-- docs/hyperdatasets/single_frames.md | 8 ++++---- docs/hyperdatasets/task.md | 4 ++-- docs/hyperdatasets/webapp/webapp_datasets_frames.md | 6 +++--- docs/hyperdatasets/webapp/webapp_exp_modifying.md | 10 +++++----- 7 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/fundamentals/projects.md b/docs/fundamentals/projects.md index 699cc5e6..963f0184 100644 --- a/docs/fundamentals/projects.md +++ b/docs/fundamentals/projects.md @@ -3,7 +3,7 @@ title: Projects --- Projects are contextual containers for [tasks](task.md) and [models](artifacts.md#models) (as well as [dataviews](../hyperdatasets/dataviews.md) -when hyperdatasets are enabled), providing a logical structure similar to file system folders. +when Hyper-Datasets are enabled), providing a logical structure similar to file system folders. An often useful method is to categorize components into projects according to models or objectives. Grouping into projects helps in identifying tasks, models, and dataviews when queried. diff --git a/docs/hyperdatasets/annotations.md b/docs/hyperdatasets/annotations.md index 8fd34056..02f06637 100644 --- a/docs/hyperdatasets/annotations.md +++ b/docs/hyperdatasets/annotations.md @@ -5,7 +5,7 @@ title: Annotations With ClearML Enterprise, annotations can be applied to video and image frames. [Frames](single_frames.md) support two types of annotations: **Frame objects** and **Frame labels**. -Annotation Tasks can be used to efficiently organize the annotation of frames in Dataset versions (see +Annotation Tasks can be used to efficiently organize the annotation of frames in Hyper-Dataset versions (see [Annotations Task Page](webapp/webapp_annotator.md)). For information about how to view, create, and manage annotations using the WebApp, see [Annotating Images and Videos](webapp/webapp_annotator.md#annotating-images-and-video). diff --git a/docs/hyperdatasets/overview.md b/docs/hyperdatasets/overview.md index 4f54fba9..43c362f5 100644 --- a/docs/hyperdatasets/overview.md +++ b/docs/hyperdatasets/overview.md @@ -1,8 +1,8 @@ --- -title: Hyper Datasets +title: Hyper-Datasets --- -ClearML's Hyper Datasets are an MLOps-oriented abstraction of your data, which facilitates traceable, reproducible model development +ClearML's Hyper-Datasets are an MLOps-oriented abstraction of your data, which facilitates traceable, reproducible model development through parametrized data access and meta-data version control. The basic premise is that a user-formed query is a full representation of the dataset used by the ML/DL process. diff --git a/docs/hyperdatasets/single_frames.md b/docs/hyperdatasets/single_frames.md index 53b8956e..10ec31d9 100644 --- a/docs/hyperdatasets/single_frames.md +++ b/docs/hyperdatasets/single_frames.md @@ -4,7 +4,7 @@ title: SingleFrames A `SingleFrame` contains metadata pointing to raw data, and other metadata and data, which supports experimentation and -ClearML Enterprise's Git-like Dataset versioning. +ClearML Enterprise's Git-like Hyper-Dataset versioning. ## Frame Components @@ -42,7 +42,7 @@ For more information, see [Masks](masks.md). ### Previews `previews` is a dictionary containing metadata for optional thumbnail images that can be used in the ClearML Enterprise WebApp (UI) -to view selected images in a Dataset. `previews` includes the `uri` of the thumbnail image. +to view selected images in a Hyper-Dataset. `previews` includes the `uri` of the thumbnail image. For more information, see [Previews](previews.md). @@ -63,9 +63,9 @@ The panel below describes the details contained within a `frame`: * `id` (*string*) - The unique ID of this frame. * `blob` (*string*) - Raw data. * `context_id` (*string*) - Source URL. -* `dataset` (*dict*) - The Dataset and version containing the frame. +* `dataset` (*dict*) - The Hyper-Dataset and version containing the frame. - * `id` - ID of the Dataset. + * `id` - ID of the Hyper-Dataset. * `version` - ID of the version. * `meta` (*dict*) - Frame custom metadata. Any custom key-value pairs (`sources` and `rois` can also contain a meta diff --git a/docs/hyperdatasets/task.md b/docs/hyperdatasets/task.md index f72ec87e..666f954e 100644 --- a/docs/hyperdatasets/task.md +++ b/docs/hyperdatasets/task.md @@ -2,11 +2,11 @@ title: Tasks --- -Hyper Datasets extend the **ClearML** [**Task**](../fundamentals/task.md) with [Dataviews](dataviews.md) +Hyper-Datasets extend the **ClearML** [**Task**](../fundamentals/task.md) with [Dataviews](dataviews.md) ## Usage -Hyper datasets are supported by the `allegroai` python package. +Hyper-Datasets are supported by the `allegroai` python package. ### Connecting Dataviews to a Task diff --git a/docs/hyperdatasets/webapp/webapp_datasets_frames.md b/docs/hyperdatasets/webapp/webapp_datasets_frames.md index 4cfa1054..54b2a93f 100644 --- a/docs/hyperdatasets/webapp/webapp_datasets_frames.md +++ b/docs/hyperdatasets/webapp/webapp_datasets_frames.md @@ -2,7 +2,7 @@ title: Working with Frames --- -View and edit SingleFrames in the Dataset page. After selecting a Dataset version, the **Version Browser** shows a sample +View and edit SingleFrames in the Dataset page. After selecting a Hyper-Dataset version, the **Version Browser** shows a sample of frames and enables viewing SingleFrames and FramesGroups, and edit SingleFrames, in the [frame viewer](#frame-viewer). Before opening the frame viewer, you can filter the frames by applying [simple](#simple-frame-filtering) or [advanced](#advanced-frame-filtering) filtering logic. @@ -19,7 +19,7 @@ frame metadata, the raw data source URI, as well as providing navigation and vie ### Frame Viewer Controls -Use frame viewer controls to navigate between frames in a Dataset Version, and control frame changes and viewing. +Use frame viewer controls to navigate between frames in a Hyper-Dataset Version, and control frame changes and viewing. |Control Icon|Actions|Shortcut| |-----|------|----| @@ -89,7 +89,7 @@ To save frames changes at any time, click **SAVE** (below the annotation list ar ### Viewing FrameGroups Viewing and editing frames in a FrameGroup is similar to viewing and editing SingleFrames. -Click the FrameGroup in the Dataset. In the frame viewer, select SingleFrame to view / modify from +Click the FrameGroup in the Hyper-Dataset. In the frame viewer, select SingleFrame to view / modify from a dropdown list in the **Current Source** section. ![Frame dropdown menu in FrameGroup](../../img/hyperdatasets/framegroup_01.png) diff --git a/docs/hyperdatasets/webapp/webapp_exp_modifying.md b/docs/hyperdatasets/webapp/webapp_exp_modifying.md index db4d43c8..b80bf6ef 100644 --- a/docs/hyperdatasets/webapp/webapp_exp_modifying.md +++ b/docs/hyperdatasets/webapp/webapp_exp_modifying.md @@ -17,14 +17,14 @@ enables modifying Dataviews, including: The selection and control of input data can be modified in *Draft* experiments that are not [development experiments](../task.md#development-experiments). -Do this by modifying the Dataview used by the experiment. The Dataview specifies the Dataset versions from which frames +Do this by modifying the Dataview used by the experiment. The Dataview specifies the Hyper-Dataset versions from which frames are iterated and frame filters (see [Dataviews](webapp_dataviews.md). **To choose a Dataview**, do any of the following: * Create a new Dataview - * Click **+** and then follow the instructions below to select Dataset versions, filter frames, map labels (label translation), + * Click **+** and then follow the instructions below to select Hyper-Dataset versions, filter frames, map labels (label translation), and set label enumeration, data augmentation, and iteration controls. * Select a different Dataview already associated with the experiment. @@ -72,7 +72,7 @@ For more detailed information, see [Filtering](../dataviews.md#filtering). 1. In the **FILTERING** area, click **EDIT**. 1. For each frame filter: - 1. Select the Dataset version to which the frame filter applies. + 1. Select the Hyper-Dataset version to which the frame filter applies. 1. Add, change, or remove any combination of the following rules: * ROI rule - Include or exclude frames containing any single ROI with any combination of labels in the Dataset @@ -95,7 +95,7 @@ that are not mapped are ignored. 1. In the **MAPPING** section, click **EDIT** * Add (**+**) or edit a mapping: - 1. Select the Dataset and version whose labels will be mapped. + 1. Select the Hyper-Dataset and version whose labels will be mapped. 1. Select one or more labels to map. @@ -123,7 +123,7 @@ Modify the label enumeration assigned to output models. ## Data Augmentation -Modify the on-the-fly data augmentation applied to frames input from the select Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength. +Modify the on-the-fly data augmentation applied to frames input from the select Hyper-Dataset versions and filtered by the frame filters. Data augmentation is applied in steps, where each step applies a method, operation, and strength. For more detailed information, see [Data Augmentation](../dataviews.md#data-augmentation).