diff --git a/docs/hyperdatasets/webapp/webapp_datasets_versioning.md b/docs/hyperdatasets/webapp/webapp_datasets_versioning.md index a2e5d361..48f584eb 100644 --- a/docs/hyperdatasets/webapp/webapp_datasets_versioning.md +++ b/docs/hyperdatasets/webapp/webapp_datasets_versioning.md @@ -106,35 +106,15 @@ The dataset version's frames can be filtered by multiple criteria. The resulting To view the details of a specific frame, click on its preview, which will open the [Frame Viewer](webapp_datasets_frames.md#frame-viewer). -### Simple Frame Filtering -Simple frame filtering returns frames containing at least one annotation with a specified label. +### Frame Filtering -**To apply a simple frame filter,** select a label from the **LABEL FILTER** list. - - - -* The **FRAMES** tab in the image below contains 101 frames. - - -![Unfiltered version browser](../../img/hyperdatasets/frame_filtering_01.png#light-mode-only) -![Unfiltered version browser](../../img/hyperdatasets/frame_filtering_01_dark.png#dark-mode-only) - -* A simple label filter for `teddy bear` shows three frames, each containing at least one ROI labeled `teddy bear`. - -![Filtered version browser](../../img/hyperdatasets/frame_filtering_02.png#light-mode-only) -![Filtered version browser](../../img/hyperdatasets/frame_filtering_02_dark.png#dark-mode-only) - - - -### Advanced Frame Filtering - -Alternatively, a combination of ROI, frame, and source rules can be specified to apply more elaborate and specific +A combination of ROI, frame, and source rules can be specified to apply more elaborate and specific filters. -**To apply advanced filters:** -1. In the **FRAMES** tab, click Advanced filters (**Advanced filters**). +**To apply filters:** +1. In the **FRAMES** tab, click Advanced filters (**Filters**). 1. In a **FRAME FILTER**, create one of the following rules: - * ROI rule - Use "Include" and "Exclude" conditions to match frames according to ROI label. If the "Include" + * ROI rule - Use "Include" and "Exclude" conditions to match frames according to an ROI label. If the "Include" condition is used, frames match the rule if they contain at least one annotation object (ROI) with ALL labels in the rule. If the "Exclude" condition is used, frames match the rule if NONE of their ROIs contain the label. Multiple ROI rules in the same filter are evaluated independently against all frame ROIs. Meaning, a frame will match the filter @@ -166,12 +146,48 @@ described in the example above. "Frame Filter 2" specifies an ROI rule for the f To clear all filters click Clear filters. +### Vector Search + +**Vector search** finds the most similar frames to a specific reference vector. Frames are evaluated based on vector +embeddings that have been registered to them through the SDK. + +To find the frames most similar to one of the frames in the version: +1. Hover over the desired frame +1. Click Dot menu +1. Select `Find Nearest Frames By` +1. Choose the frame’s vector field that will be compared against the reference vector +1. Input the [search configuration](#search-configuration) + +To find the frames most similar to an arbitrary vector: +1. In the **FRAMES** tab, click Filter (filters) +1. Under **Vector search**, enter the vector values under **Reference vector**. +1. Input the [search configuration](#search-configuration) + +#### Search Configuration +* **Vector field** - Select the FrameGroup's vector field that will be compared against the reference vector. +* **Number of neighbors** - Choose how many nearest neighbors to show. +* **Search strategy** - Either of: + * `KNN` (K-nearest neighbors) + * `HNSW` (Hierarchical Navigable Small World) - Available with cosine similarity only. +* **Similarity function** - Select `Cosine similarity`, `Euclidean distance`, or `Dot product`. + +After entering the search configuration, click **Apply**. + +The frames are returned in order of their similarity to the reference vector. The calculated similarity is displayed on +each frame preview: + +![Vector Search](../../img/hyperdatasets/vector_search.png#light-mode-only) +![Vector Search](../../img/hyperdatasets/vector_search_dark.png#dark-mode-only) + +Vector search results adhere to configured frame filters. For example, if you filter for frames containing the ROI label +`cat`, the search will return only the nearest neighbors among frames with that ROI. ### Filtering Examples -* Create one ROI rule for the `teddy bear` label, which shows the same three frames as the simple frame filter (above). +* Create one ROI rule for the `teddy bear` label. Only frame containing at least one ROL labeled `teddy bear` match the + filter ![Adding an ROI rule](../../img/hyperdatasets/frame_filtering_03.png#light-mode-only) ![Adding an ROI rule](../../img/hyperdatasets/frame_filtering_03_dark.png#dark-mode-only) diff --git a/docs/img/hyperdatasets/frame_filtering_01.png b/docs/img/hyperdatasets/frame_filtering_01.png deleted file mode 100644 index 818504a4..00000000 Binary files a/docs/img/hyperdatasets/frame_filtering_01.png and /dev/null differ diff --git a/docs/img/hyperdatasets/frame_filtering_01_dark.png b/docs/img/hyperdatasets/frame_filtering_01_dark.png deleted file mode 100644 index 6a752ab3..00000000 Binary files a/docs/img/hyperdatasets/frame_filtering_01_dark.png and /dev/null differ diff --git a/docs/img/hyperdatasets/frame_filtering_02.png b/docs/img/hyperdatasets/frame_filtering_02.png deleted file mode 100644 index f8ef5364..00000000 Binary files a/docs/img/hyperdatasets/frame_filtering_02.png and /dev/null differ diff --git a/docs/img/hyperdatasets/frame_filtering_02_dark.png b/docs/img/hyperdatasets/frame_filtering_02_dark.png deleted file mode 100644 index 4921ba4f..00000000 Binary files a/docs/img/hyperdatasets/frame_filtering_02_dark.png and /dev/null differ diff --git a/docs/img/hyperdatasets/vector_search.png b/docs/img/hyperdatasets/vector_search.png new file mode 100644 index 00000000..60679961 Binary files /dev/null and b/docs/img/hyperdatasets/vector_search.png differ diff --git a/docs/img/hyperdatasets/vector_search_dark.png b/docs/img/hyperdatasets/vector_search_dark.png new file mode 100644 index 00000000..de830d41 Binary files /dev/null and b/docs/img/hyperdatasets/vector_search_dark.png differ