clearml-docs/docs/hyperdatasets/sources.md

---
title: Sources
---

Each frame contains `sources`, a list of dictionaries containing: 
* Attributes of the source data (image raw data)
* A `URI` pointing to the source data (image or video)
* Sources for [masks](masks.md) used in semantic segmentation
* Image [previews](previews.md), which are thumbnails used in the ClearML Enterprise WebApp (UI).

`sources` does not contain: 
* `rois` even though ROIs are directly associated with the images and `masks` in `sources`
* ROI metadata, because ROIs can be used over multiple frames. 
  
Instead, frames contain a top-level `rois` array, which is a list of ROI dictionaries, where each dictionary contains a 
list of source IDs. Those IDs connect `sources` to ROIs.

## Examples 

The examples below demonstrate the `sources` section of a Frame for different types of content.

### Example 1: Video Sources

This example demonstrates `sources` for video.

<details className="cml-expansion-panel info">
<summary className="cml-expansion-panel-summary">Example 1</summary>
<div className="cml-expansion-panel-content">

```json
/* video from one of four cameras on car */
"sources": [
    {
        "id": "front",
        "content_type": "video/mp4",
        "width": 800,
        "height": 600,
        "uri": "https://s3.amazonaws.com/my_cars/car_1/front.mp4",
        "timestamp": 1234567889,
        "meta" :{
            "angle":45,
            "fov":129
        },
    },
    {
        "id": "rear",
        "uri": "https://s3.amazonaws.com/my_cars/car_1/rear.mp4",
        "content_type": "video/mp4",
        "timestamp": 1234567889
    }
    
```

</div>
</details>
<br/>

The `sources` example above details a video from a car that has two cameras. One camera
is the source with the ID `front` and the other is the source with the ID `rear`.

`sources` includes the following information about the Frame:
* `content_type` - The video is an `mp4` file
* `width` and `height` - Each frame in the video is `800` pixels by `600` pixels,
* `uri` - The raw data is located in `s3.amazonaws.com/my_cars/car_1/front.mp4` and `s3.amazonaws.com/my_cars/car_1/rear.mp4` 
  (the front and rear camera, respectively) 
* `timestamp` - This indicates the absolute position of the frame in the video
* `meta` - Additional metadata is included for the angle of the camera (`angle`) and its field of vision (`fov`).

:::note
Sources includes a variety of content types. This example shows mp4 video.
:::

### Example 2: Images Sources

This example demonstrates `sources` images.
<details className="cml-expansion-panel info">
<summary className="cml-expansion-panel-summary">Example 2</summary>
<div className="cml-expansion-panel-content">        

```json
/* camera images */
"sources": [
    {
        "id": "default",
        "content_type": "png",
        "width": 800,
        "height": 600,
        "uri": "https://s3.amazonaws.com/my_images/imag1000.png",
        "timestamp": 0,
    }
```

</div>
</details>
<br/>

The `sources` of this frame contains the following information:
* `content_type` - This frame contains an image in `png` format. 
* `width` and `height` - It is `800` px by `600` px, 
* `uri` - The raw data is located in `https://s3.amazonaws.com/my_images/imag1000.png`
* `timestamp` is 0 (timestamps are used for video).


### Example 3: Sources and Regions of Interest

This example demonstrates `sources` for video, `masks`, and `preview`. 

<details className="cml-expansion-panel info">
<summary className="cml-expansion-panel-summary">Example 3</summary>
<div className="cml-expansion-panel-content">

```json

{
  "timestamp": 1234567889, 
  "context_id": "car_1", 
  "meta": {
    "velocity": "60"
  }, 
  "sources": [
    {
      "id": "front",
      "content_type": "video/mp4",
      "width": 800,
      "height": 600,
      "uri": "https://s3.amazonaws.com/my_cars/car_1/front.mp4",
      "timestamp": 1234567889,
      "meta" :{
        "angle":45,
        "fov":129
      }, 
      "preview": {
        "content_type": "image/jpg",
        "uri": "https://s3.amazonaws.com/my_previews/car_1/front_preview.jpg",
        "timestamp": 0
      },
      "masks": [
        {
          "id": "seg",
          "content_type": "video/mp4",
          "uri": "https://s3.amazonaws.com/seg_masks/car_1/front_seg.mp4",
          "timestamp": 1234567889
        },
        {
          "id": "instances_seg",
          "content_type": "video/mp4",
          "uri": "https://s3.amazonaws.com/seg_masks/car_1/front_instance_seg.mp4",
          "timestamp": 1234567889
        }
      ]
    },
    {
      "id": "rear",
      "uri": "https://s3.amazonaws.com/my_cars/car_1/rear.mp4",
      "content_type": "video/mp4",
      "timestamp": 1234567889
    }
  ],
  "rois": [
    {
      "sources":["front"],
      "label": ["right_lane"],
      "mask": {
        "id": "seg",
        "value": [-1, 1, 255]
      }
    },
    {
      "sources": ["front"],
      "label": ["bike"],
      "poly":[30, 50, 50,50, 100,50, 100,100],
      "meta": {
        "velocity": 5.4
      }
    },
    {
      "sources": ["front", "rear"],
      "label": ["car"],
      "poly":[30, 50, 50,50, 100,50, 100,100]
    }
  ]
}
```

</div>
</details>
<br/>

This frame shows the `masks` section in `sources`, and the top-level `rois` array.

In `sources`,  the `masks` subsection contains the sources for the two masks associated with the raw data. 

The raw mask data is located in:

* `https://s3.amazonaws.com/my_cars/car_1/front.mp4`
* `https://s3.amazonaws.com/my_previews/car_1/front_preview.jpg`

In `rois`, the `mask` section is associated with its `masks` source using the `id` key. 
In this example: 
* In the `rois` array, there is a region of interest that has a `mask` with the ID `seg` and an RGB
  value
* The `masks` section in `sources` contains the location of each mask. The first dictionary of `masks`
details the mask with the ID `seg`. The ID connects it to the `seg` mask in `rois`

`sources` also contains the source of a preview. It is located in: `https://s3.amazonaws.com/my_previews/car_1/front_preview.jpg`.