Update docs

docs/deploying_clearml/enterprise_deploy/appgw_install_compose.md

@@ -6,6 +6,14 @@ title: Docker-Compose Deployment
 The Application Gateway is available under the ClearML Enterprise plan.
 :::
 
+The AI Application Gateway enables external HTTP(S) or direct TCP access to ClearML tasks and applications running on 
+nodes. The gateway is configured with an endpoint or external address, making these services accessible from the user's 
+machine, outside the workload’ network.
+
+This guide describes how to install and run the ClearML AI Application Gateway using docker-compose for environments 
+where you manage both the ClearML Server and the workload nodes.
+
+
 ## Requirements
 
 * Linux OS (x86) machine  
@@ -155,3 +163,11 @@ Each router's `runtime.env` file should include:
   ```
   
 Make sure `LISTEN_QUEUE_NAME` is set in the  [`docker-compose` environment variables](#docker-compose-file) for each router instance.
+
+## Monitoring and Testing the Gateway
+
+Once your gateway is deployed, you can monitor its status, view routed tasks, and run connectivity tests in 
+ClearML WebApp, under **Settings > Application Gateway**.
+
+![App Gateway Test](../../img/settings_app_gateway_test.png#light-mode-only)
+![App Gateway Test](../../img/settings_app_gateway_test_dark.png#dark-mode-only)

docs/deploying_clearml/enterprise_deploy/appgw_install_compose_hosted.md

@@ -163,4 +163,12 @@ to `false` in the `runtime.env` file.
    LISTEN_QUEUE_NAME=queue3,queue4  
    ```
    
- Make sure `LISTEN_QUEUE_NAME` is set in the  [`docker-compose` environment variables](#docker-compose-file) for each router instance.
+ Make sure `LISTEN_QUEUE_NAME` is set in the  [`docker-compose` environment variables](#docker-compose-file) for each router instance.
+ 
+## Monitoring and Testing the Gateway
+
+Once your gateway is deployed, you can monitor its status, view routed tasks, and run connectivity tests in 
+ClearML WebApp, under **Settings > Application Gateway**.
+
+![App Gateway Test](../../img/settings_app_gateway_test.png#light-mode-only)
+![App Gateway Test](../../img/settings_app_gateway_test_dark.png#dark-mode-only)

docs/deploying_clearml/enterprise_deploy/appgw_install_k8s.md

@@ -104,4 +104,12 @@ Replace the placeholders with the following values:
   namespace. The App Gateway monitors the same namespace for AI workloads (e.g. remote IDE applications). The App Gateway has a 
   namespace-limited scope, meaning it can only detect and manage tasks within its 
   assigned namespace.
-* `<CHART_VERSION>` - Version recommended by the ClearML Support Team.
+* `<CHART_VERSION>` - Version recommended by the ClearML Support Team.
+
+## Monitoring and Testing the Gateway
+
+Once your gateway is deployed, you can monitor its status, view routed tasks, and run connectivity tests in 
+ClearML WebApp, under **Settings > Application Gateway**.
+
+![App Gateway Test](../../img/settings_app_gateway_test.png#light-mode-only)
+![App Gateway Test](../../img/settings_app_gateway_test_dark.png#dark-mode-only)

docs/hyperdatasets/webapp/webapp_datasets.md

@@ -84,6 +84,8 @@ of a dataset card to open its context menu and access dataset actions:
 
 </div>
 
+* **Copy Dataset ID**
+* **View Dataset Schema** - View the dataset's frame document schema. This shows the frames’ fields and their types.
 * **Rename** - Change the dataset's name
 * **Add Tag** - Add label to the dataset to help easily classify groups of datasets.
 * **Edit Metadata** - Modify dataset-level metadata. This will open the metadata edit window, where you can edit the section

docs/hyperdatasets/webapp/webapp_datasets_frames.md

@@ -22,6 +22,18 @@ and frame metadata, as well as view frame masks of your dataset version frames.
 ![Frame viewer](../../img/hyperdatasets/dataset_example_frame_editor.png#light-mode-only)
 ![Frame viewer](../../img/hyperdatasets/dataset_example_frame_editor_dark.png#dark-mode-only)
 
+### Viewing FrameGroup Data
+
+FrameGroup information is organized across several panels in the Frame Viewer: 
+* **FrameGroup Details** – Displays all FrameGroup details except metadata. The details include: general fields (e.g. 
+  `context_id`, dataset ID and version, timestamp, etc.) as well as annotation (e.g. `rois`, their `labels`, `confidence`, etc.) 
+  * Click <img src="/docs/latest/icons/ico-metadata.svg" alt="edit metadata" className="icon size-md space-sm" /> to view the details in a larger window. 
+  * Click <img src="/docs/latest/icons/ico-code-file.svg" alt="view schema" className="icon size-md space-sm" /> to view the dataset's frame document schema. This covers all possible fields in the dataset
+* **FrameGroup Metadata** – Displays FrameGroup-level metadata. Displays only the contents of the `frame.meta` field—typically used for structured metadata or custom key-value data.
+  * Click <img src="/docs/latest/icons/ico-metadata.svg" alt="edit metadata" className="icon size-md space-sm" /> to modify the metadata field.
+* **Annotations** - Review and modify frame annotations. For more information, see [Masks](#masks) and [Annotations](#annotations).
+
+
 ### Frame Viewer Controls
 
 Use frame viewer controls to navigate between frames in a Hyper-Dataset Version, and control frame changes and viewing.
@@ -113,7 +125,7 @@ If an annotation applies to all frames in a FrameGroup, it is displayed with a `
 
 </div>
 
-## Masks 
+## Masks
 
 Use the **MASKS** panel to select which masks to apply over the frame. 
 

docs/hyperdatasets/webapp/webapp_datasets_versioning.md

@@ -106,35 +106,15 @@ The dataset version's frames can be filtered by multiple criteria. The resulting
 
 To view the details of a specific frame, click on its preview, which will open the [Frame Viewer](webapp_datasets_frames.md#frame-viewer).
 
-### Simple Frame Filtering
-Simple frame filtering returns frames containing at least one annotation with a specified label.
+### Frame Filtering
 
-**To apply a simple frame filter,** select a label from the **LABEL FILTER** list.
-
-<Collapsible type="screenshot" title="Simple filter example">
-
-* The **FRAMES** tab in the image below contains 101 frames.
-
-
-![Unfiltered version browser](../../img/hyperdatasets/frame_filtering_01.png#light-mode-only)
-![Unfiltered version browser](../../img/hyperdatasets/frame_filtering_01_dark.png#dark-mode-only)
-
-* A simple label filter for `teddy bear` shows three frames, each containing at least one ROI labeled `teddy bear`.
-
-![Filtered version browser](../../img/hyperdatasets/frame_filtering_02.png#light-mode-only)
-![Filtered version browser](../../img/hyperdatasets/frame_filtering_02_dark.png#dark-mode-only)
-
-</Collapsible>
-
-### Advanced Frame Filtering
-
-Alternatively, a combination of ROI, frame, and source rules can be specified to apply more elaborate and specific 
+A combination of ROI, frame, and source rules can be specified to apply more elaborate and specific 
 filters.
 
-**To apply advanced filters:**
-1. In the **FRAMES** tab, click <img src="/docs/latest/icons/ico-advanced-filters.svg" alt="Advanced filters" className="icon size-md space-sm" /> (**Advanced filters**).
+**To apply filters:**
+1. In the **FRAMES** tab, click <img src="/docs/latest/icons/ico-advanced-filters.svg" alt="Advanced filters" className="icon size-md space-sm" /> (**Filters**).
 1. In a **FRAME FILTER**, create one of the following rules:
-   * ROI rule - Use "Include" and "Exclude" conditions to match frames according to ROI label. If the "Include" 
+   * ROI rule - Use "Include" and "Exclude" conditions to match frames according to an ROI label. If the "Include" 
    condition is used, frames match the rule if they contain at least one annotation object (ROI) with ALL labels in the 
    rule. If the "Exclude" condition is used, frames match the rule if NONE of their ROIs contain the label. Multiple ROI 
    rules in the same filter are evaluated independently against all frame ROIs. Meaning, a frame will match the filter 
@@ -166,12 +146,48 @@ described in the example above. "Frame Filter 2" specifies an ROI rule for the f
 
 To clear all filters click <img src="/docs/latest/icons/ico-filter-reset.svg" alt="Clear filters" className="icon size-md" />. 
 
+### Vector Search
+
+**Vector search** finds the most similar frames to a specific reference vector. Frames are evaluated based on vector 
+embeddings that have been registered to them through the SDK.
+
+To find the frames most similar to one of the frames in the version: 
+1. Hover over the desired frame
+1. Click <img src="/docs/latest/icons/ico-dots-v-menu.svg" alt="Dot menu" className="icon size-md space-sm" />
+1. Select `Find Nearest Frames By`
+1. Choose the frame’s vector field  that will be compared against the reference vector
+1. Input the [search configuration](#search-configuration)
+
+To find the frames  most similar to an arbitrary vector:
+1. In the **FRAMES** tab, click  <img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" /> (filters)
+1. Under **Vector search**, enter the vector values under **Reference vector**.
+1. Input the [search configuration](#search-configuration)
+
+#### Search Configuration
+* **Vector field** - Select the FrameGroup's vector field that will be compared against the reference vector.
+* **Number of neighbors** - Choose how many nearest neighbors to show.
+* **Search strategy** - Either of:
+  * `KNN` (K-nearest neighbors) 
+  * `HNSW` (Hierarchical Navigable Small World) - Available with cosine similarity only.
+* **Similarity function** - Select `Cosine similarity`, `Euclidean distance`, or `Dot product`. 
+
+After entering the search configuration, click **Apply**.
+
+The frames are returned in order of their similarity to the reference vector. The calculated similarity is displayed on 
+each frame preview:
+
+![Vector Search](../../img/hyperdatasets/vector_search.png#light-mode-only)
+![Vector Search](../../img/hyperdatasets/vector_search_dark.png#dark-mode-only)
+
+Vector search results adhere to configured frame filters. For example, if you filter for frames containing the ROI label
+`cat`, the search will return only the nearest neighbors among frames with that ROI.
 
 ### Filtering Examples
 
 <Collapsible type="screenshot" title="ROI Rules"> 
 
-* Create one ROI rule for the `teddy bear` label, which shows the same three frames as the simple frame filter (above).
+* Create one ROI rule for the `teddy bear` label. Only frame containing at least one ROL labeled `teddy bear` match the 
+  filter
 
 ![Adding an ROI rule](../../img/hyperdatasets/frame_filtering_03.png#light-mode-only)
 ![Adding an ROI rule](../../img/hyperdatasets/frame_filtering_03_dark.png#dark-mode-only)

docs/webapp/settings/webapp_settings_app_gw.md

@@ -0,0 +1,66 @@
+---
+title: Application Gateway
+---
+
+The ClearML [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md) facilitates setting up secure, 
+authenticated access to jobs running on your compute nodes from external networks (see application gateway installation
+for [Kubernetes](../../deploying_clearml/enterprise_deploy/appgw_install_k8s.md), [Docker-Compose Self-Hosted Deployment](../../deploying_clearml/enterprise_deploy/appgw_install_compose.md) 
+or [Docker-Compose Hosted Deploynment](../../deploying_clearml/enterprise_deploy/appgw_install_compose_hosted.md)).
+
+The **Application Gateway** table lets you monitor all active application gateway routers, as well as verify gateway functionality. The table shows each router’s:
+* Name 
+* Externally accessible URL 
+* Test Status: The result of the most recent connectivity test 
+* Last Tested: The time the router was last tested 
+
+![Application Gateway table](../../img/settings_app_gateway.png#light-mode-only)
+![Application Gateway table](../../img/settings_app_gateway_dark.png#dark-mode-only)
+
+Click on a router to open its details panel, which includes:
+* **Info**: General router information
+  * Router details 
+    * Uptime 
+    * Last update time 
+    * Router version 
+  * Routed Tasks table: ClearML tasks currently available for access through the router
+    * Task name: Click to navigate to the task page
+    * Endpoint: Exposed application URL
+    * Owner: User who initiated the task
+
+  ![Application Gateway info](../../img/settings_app_gateway_info.png#light-mode-only)
+  ![Application Gateway info](../../img/settings_app_gateway_info_dark.png#dark-mode-only)
+
+* **Test Details**: Administrators can run a test to verify that a gateway is functioning correctly: Identifying routed 
+  tasks and creating accessible network endpoints for them. The test launches a small task in the target network 
+  (specified through the desired ClearML queue), and checks that the router successfully creates a route to that task, 
+  and routes the network traffic to it. 
+
+  To run a test:
+  1. Hover over the **Test Details** panel **>** Click **Test**
+  1. Input a queue that is serviced by agents in the network environment the router should provide access to  
+  1. Click **Test**
+  
+  <br/>
+
+  :::note
+  Testing is only supported when both the ClearML WebApp and the gateway endpoint are served over secure (HTTPS) protocols.
+  :::
+
+  The **Test Details** tab displays: 
+  * Status - Result of the most recent test
+  * Status message
+  * Status reason
+  * Started time
+  * Completed time 
+  * Run time
+  * Queue - Queue where test task was enqueued for execution
+  * Worker - ClearML Agent that executed the test task
+  * Test Task name
+  * Task ID 
+  * Browser endpoint
+  * Endpoint 
+
+  ![Application Gateway test](../../img/settings_app_gateway_test.png#light-mode-only)
+  ![Application Gateway test](../../img/settings_app_gateway_test_dark.png#dark-mode-only)
+
+* **Test log**: Console output of the most recent router test. 

docs/webapp/settings/webapp_settings_overview.md

@@ -24,6 +24,8 @@ The Settings page consists of the following sections:
   * [Identity Providers](webapp_settings_id_providers.md) (ClearML Enterprise Server) - Manage server identity providers
   * [Resource Configuration](webapp_settings_resource_configs.md) (ClearML Enterprise Server) - Define the available resources and the way in which they 
   will be allocated to different workloads 
+  * [Application Gateway](webapp_settings_app_gw.md) (ClearML Enterprise Server) - Monitor all active application 
+  gateways, and verify gateway functionality
   * [Billing & Usage](webapp_settings_usage_billing.md) (ClearML Hosted Service) - View current billing details and usage information 
   * [Storage Cleanup](webapp_settings_storage_credentials.md) (ClearML Enterprise Server) - Configure storage provider access credentials to 
   enable ClearML to delete artifacts stored in cloud storage when tasks and models are deleted

docs/webapp/settings/webapp_settings_users.md

@@ -92,7 +92,7 @@ Service accounts are members of the `Users` group, meaning they can access the r
 impersonation is enabled, a task run by the service account (i.e. by an agent or autoscaler using the service accounts' 
 credentials) is executed as if by the owner of the task, meaning it will have access to the task owner's configuration 
 vaults and to the resources that the task owner has access to. Impersonating an admin user does not mean the task's code 
-will have admin privileges.
+will have admin privileges (see [Setting a Service Account as Administrator](#setting-a-service-account-as-administrator)).
 
 In case impersonation is not enabled: 
 * If you run an agent with `--use_owner_token` then the agent will fail. 
@@ -103,6 +103,25 @@ In case impersonation is not enabled:
 When a service account is created, an initial set of credentials is automatically generated. The dialog displays new 
 credentials, formatted as a ready-to-copy configuration file section.
 
+### Setting a Service Account as Administrator
+
+You can grant a service account administrator privileges, giving it the same access and capabilities as an admin user.
+
+#### Prerequisite: Enable Admin Privileges for Service Accounts
+
+To allow admin roles to be assigned to service accounts, you must enable support for admin service accounts on the server 
+in one of the following ways:
+* **Set as an environment variable**: Set `CLEARML__services__organization__allow_service_user_admins=true` in the `apiserver` service environment
+* **Edit the services/organization.conf file**: Set `allow_service_user_admins: true`
+
+#### Assigning Admin Role to a Service Account
+
+To assign an admin role to a service account: 
+1. Go to the **Service Accounts** table.
+1. Click the service account you want to modify to open its details panel.
+1. Select **Admin** toggle to grant administrator access.
+
+
 ### Service Account Credentials 
 
 To generate new credentials for a service account:

sidebars.js

@@ -581,6 +581,7 @@ module.exports = {
                         'webapp/settings/webapp_settings_access_rules',
                         'webapp/settings/webapp_settings_id_providers',
                         'webapp/settings/webapp_settings_resource_configs',
+                        'webapp/settings/webapp_settings_app_gw',
                         'webapp/settings/webapp_settings_usage_billing',
                         'webapp/settings/webapp_settings_storage_credentials'
                     ]

static/icons/ico-code-file.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24">
+    <path d="M21,8h-6V2l6,6Zm-7,1h7v11c0,1.10455-.89539,2-2,2H5c-1.10455,0-2-.89545-2-2V4c0-1.10461,.89545-2,2-2H14v7Zm1.35889,9.86047l3.64111-3.63538-3.64111-3.64105v1.90112l1.73993,1.73993-1.73993,1.73511v1.90027Zm-6.71783-7.27643l-3.64105,3.64105,3.64105,3.63538v-1.90033l-1.73987-1.73505,1.73987-1.73993v-1.90112Zm5.04529-.91028l-1.31671-.26874-2.09302,9.61047,1.31671,.26874,2.09302-9.61047Z" fill="#8492c2"/>
+</svg>