Merge branch 'clearml:main' into main

This commit is contained in:
pollfly 2025-02-17 13:56:17 +02:00 committed by GitHub
commit 2a9d9b2c8f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
10 changed files with 99 additions and 10 deletions

View File

@ -139,9 +139,9 @@ You can retrieve the Dataview frames using [`DataView.to_list()`](../references/
[`DataView.to_dict()`](../references/hyperdataset/dataview.md#to_dict), or [`DataView.get_iterator()`](../references/hyperdataset/dataview.md#get_iterator) [`DataView.to_dict()`](../references/hyperdataset/dataview.md#to_dict), or [`DataView.get_iterator()`](../references/hyperdataset/dataview.md#get_iterator)
(see [Accessing Frames](#accessing-frames)). (see [Accessing Frames](#accessing-frames)).
#### ROI Queries: ### ROI Query Examples
* **ROI query for a single label** #### ROI query for a single label
This example uses an ROI query to filter for frames containing at least one ROI with the label `cat`: This example uses an ROI query to filter for frames containing at least one ROI with the label `cat`:
@ -163,7 +163,7 @@ myDataView.add_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
* **ROI query for one label OR another** #### ROI query for one label OR another
This example uses an ROI query to filter for frames containing at least one ROI with either the label `cat` OR the label `dog`: This example uses an ROI query to filter for frames containing at least one ROI with either the label `cat` OR the label `dog`:
@ -188,7 +188,7 @@ myDataView.add_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
* **ROI query for two specific labels in the same ROI** #### ROI query for two specific labels in the same ROI
This example uses an ROI query to filter for frames containing at least one ROI with both the label `Car` AND the label `partly_occluded`: This example uses an ROI query to filter for frames containing at least one ROI with both the label `Car` AND the label `partly_occluded`:
@ -207,7 +207,7 @@ myDataView.add_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
* **ROI query for one label AND NOT another (Lucene query)** #### ROI query for one label AND NOT another (Lucene query)
This example uses an ROI query to filter for frames containing at least one ROI that has with the label `Car` AND DOES NOT This example uses an ROI query to filter for frames containing at least one ROI that has with the label `Car` AND DOES NOT
have the label `partly_occluded`: have the label `partly_occluded`:
@ -230,7 +230,7 @@ myDataView.add_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
* **ROI query for one label AND another label in different ROIs** #### ROI query for one label AND another label in different ROIs
This example uses an ROI query to filter for frames containing at least one ROI with the label `Car` and at least one This example uses an ROI query to filter for frames containing at least one ROI with the label `Car` and at least one
ROI with the label `Person`. The example demonstrates using the `roi_queries` parameter of [`DataView.add_multi_query()`](../references/hyperdataset/dataview.md#add_multi_query) ROI with the label `Person`. The example demonstrates using the `roi_queries` parameter of [`DataView.add_multi_query()`](../references/hyperdataset/dataview.md#add_multi_query)
@ -250,7 +250,7 @@ myDataview.add_multi_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
* **ROI query for one label AND NOT another label in different ROIs** #### ROI query for one label AND NOT another label in different ROIs
This example uses an ROI query to filter for frames containing at least one ROI with the label `Car` AND that DO NOT This example uses an ROI query to filter for frames containing at least one ROI with the label `Car` AND that DO NOT
contain ROIs with the label `Person`. To exclude an ROI, pass `must_not=True` in the [`DataView.RoiQuery`](../references/hyperdataset/dataview.md#roiquery) contain ROIs with the label `Person`. To exclude an ROI, pass `must_not=True` in the [`DataView.RoiQuery`](../references/hyperdataset/dataview.md#roiquery)
@ -311,7 +311,7 @@ myDataView.add_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
#### Frame Queries ### Frame Queries
Use frame queries to filter frames by ROI labels and/or frame metadata key-value pairs that a frame must include or Use frame queries to filter frames by ROI labels and/or frame metadata key-value pairs that a frame must include or
exclude for the Dataview to return the frame. exclude for the Dataview to return the frame.
@ -319,6 +319,8 @@ exclude for the Dataview to return the frame.
**Frame queries** match frame meta key-value pairs, ROI labels, or both. **Frame queries** match frame meta key-value pairs, ROI labels, or both.
They use the same logical OR, AND, NOT AND matching as ROI queries. They use the same logical OR, AND, NOT AND matching as ROI queries.
#### Frame Query by Metadata
This example demonstrates a frame query filtering for frames containing the meta key `city` value of `bremen`: This example demonstrates a frame query filtering for frames containing the meta key `city` value of `bremen`:
```python ```python
@ -336,6 +338,59 @@ myDataView.add_query(
list_of_frames = myDataView.to_list() list_of_frames = myDataView.to_list()
``` ```
#### Frame Query by Date and Time
Provided that a metadata field stores date and time values, you can query frames based on date ranges and specific time
intervals
##### Frame Query for Specific Date
This example demonstrates a frame query filtering for frames containing the meta key `updated` with the value of October
20th, 2024:
```python
# Add a frame query for frames with the meta key "updated" value of "2024-10-20"
myDataView.add_query(
dataset_name='myDataset',
version_name='version',
frame_query='meta.updated:[2024-10-20 TO 2024-10-20]'
)
# retrieving the actual SingleFrames / FrameGroups
# you can also iterate over the frames with `for frame in myDataView.get_iterator():`
list_of_frames = myDataView.to_list()
```
##### Frame Query for Date Range
This example demonstrates a frame query filtering for frames containing the meta key `updated` with any value between the
dates of October 20th and October 30th, 2024:
```python
# Add a frame query for frames with the meta key "updated" value between "2024-10-20" and "2024-10-30"
myDataView.add_query(
dataset_name='myDataset',
version_name='version',
frame_query='meta.updated:[2024-10-20 TO 2024-10-30]'
)
# retrieving the actual SingleFrames / FrameGroups
# you can also iterate over the frames with `for frame in myDataView.get_iterator():`
list_of_frames = myDataView.to_list()
```
##### Frame Query for Time Interval
This example demonstrates a frame query filtering for frames containing the meta key `updated` with any value between
`08:000` and `09:00` on October 20th, 2024:
```python
# Add a frame query for frames with the meta key's value between 08:00:00 and 09:00:00 on 2024-10-20
myDataView.add_query(
dataset_name='myDataset',
version_name='version',
frame_query='meta.<field_name>:[2024-10-20T08:00:00 TO 2024-10-20T09:00:00]'
)
```
### Controlling Query Iteration ### Controlling Query Iteration

View File

@ -163,7 +163,7 @@ described in the example above. "Frame Filter 2" specifies an ROI rule for the f
To clear all filters click <img src="/docs/latest/icons/ico-filter-reset.svg" alt="Clear filters" className="icon size-md" />. To clear all filters click <img src="/docs/latest/icons/ico-filter-reset.svg" alt="Clear filters" className="icon size-md" />.
#### Filtering Examples ### Filtering Examples
<Collapsible type="screenshot" title="ROI Rules"> <Collapsible type="screenshot" title="ROI Rules">
@ -199,7 +199,7 @@ To clear all filters click <img src="/docs/latest/icons/ico-filter-reset.svg" al
</Collapsible> </Collapsible>
<Collapsible type="screenshot" title="Frame Rules"> <Collapsible type="screenshot" title="Frame Rules: Metadata">
Filter by metadata using Lucene queries. Filter by metadata using Lucene queries.
@ -215,6 +215,40 @@ Filter by metadata using Lucene queries.
</Collapsible> </Collapsible>
<Collapsible type="screenshot" title="Frame Rules: Date and Time Fields">
If your dataset includes a metadata field that stores date and time information, you can filter
based on date ranges or specific time intervals.
Filter by date/time metadata fields using Lucene queries.
* **Data range filter**
* Add a frame rule to filter by the metadata key `updated` for the value of `[2024-10-20 TO 2024-10-20]`. The query
will match all frames where the `updated` value matches October 20th 2024. Use the format `meta.<field_name>.[YYYY-MM-DD TO YYYY-MM-DD]`.
![Filter by date](../../img/hyperdatasets/frame_filtering_11.png#light-mode-only)
![Filter by date](../../img/hyperdatasets/frame_filtering_11_dark.png#dark-mode-only)
* Open a frame in the frame viewer to see its metadata.
![Frame date metadata in frame viewer](../../img/hyperdatasets/frame_filtering_12.png#light-mode-only)
![Frame date metadata in frame viewer](../../img/hyperdatasets/frame_filtering_12_dark.png#dark-mode-only)
* **Time interval filter**
* Add a frame rule to filter by the metadata key `updated` for the value of `[2024-10-20T08:00:00 TO 2024-10-20T09:00:00]`.
The query will match all frames where the updated value is between 08:00 and 09:00 on October 20th 2024.
Use the format `meta.<field_name>.[YYYY-MM-DDThh:mm:ss TO YYYY-MM-DDThh:mm:ss]`.
![Filter by datetime](../../img/hyperdatasets/frame_filtering_13.png#light-mode-only)
![Filter by datetime](../../img/hyperdatasets/frame_filtering_13_dark.png#dark-mode-only)
* Open a frame in the frame viewer to see its metadata.
![Frame datetimee metadata in frame viewer](../../img/hyperdatasets/frame_filtering_14.png#light-mode-only)
![Frame datetime metadata in frame viewer](../../img/hyperdatasets/frame_filtering_14_dark.png#dark-mode-only)
</Collapsible>
<Collapsible type="screenshot" title="Source Rules"> <Collapsible type="screenshot" title="Source Rules">
Filter by sources using Lucene queries. Filter by sources using Lucene queries.

Binary file not shown.

After

Width:  |  Height:  |  Size: 613 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 612 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 MiB