Update docs (#793)

docs/webapp/applications/apps_overview.md

@@ -13,10 +13,6 @@ Use ClearML's GUI Applications to manage ML workloads and automatically run your
 Configure and launch app instances, then track their execution from the app dashboard.
 
 ClearML provides the following applications:
-* [**GPU Compute**](apps_gpu_compute.md) - Launch cloud machines on demand and optimize their usage according to a 
-  defined budget--no previous setup necessary
-* [**AWS Autoscaler**](apps_aws_autoscaler.md) - Optimize AWS EC2 instance usage according to a defined instance budget
-* [**GCP Autoscaler**](apps_gcp_autoscaler.md) - Optimize GCP instance usage according to a defined instance budget
 * [**Hyperparameter Optimization**](apps_hpo.md) - Find the parameter values that yield the best performing models
 * **Nvidia Clara** - Train models using Nvidia's Clara framework
 * [**Project Dashboard**](apps_dashboard.md) - High-level project monitoring with Slack alerts
@@ -25,6 +21,12 @@ ClearML provides the following applications:
 * [**Jupyter Lab**](apps_jupyter_lab.md) - Launch a Jupyter Lab session on a remote machine (available under ClearML Enterprise Plan)
 * [**VS Code**](apps_vscode.md) - Launch a VS Code session on a remote machine (available under ClearML Enterprise Plan)
 
+:::info Autoscalers
+Autoscaling ([GPU Compute](apps_gpu_compute.md), [AWS Autoscaler](apps_aws_autoscaler.md), and [GCP Autoscaler](apps_gcp_autoscaler.md))
+was previously available through the Applications page. The autoscaler functionality has been moved to the [Orchestration page](https://app.clear.ml/workers-and-queues/autoscalers) 
+in the WebApp. 
+:::
+
 ## App Pages Layout
 Each application's page is split into two sections:
 * App Instance List - Launch new app instances and view previously launched instances. Click on an instance to view its 

docs/webapp/webapp_exp_comparing.md

@@ -8,7 +8,8 @@ differences in experiments' results and their causes. You can view the differenc
 objects, and other details.
 * Hyperparameters
   * [Values](#side-by-side-textual-comparison) - Compare parameters and their values
-  * [Parallel coordinates](#parallel-coordinates-mode) - View the impact of hyperparameters on a selected metric
+  * [Parallel coordinates](#parallel-coordinates-mode) - View the impact of hyperparameters on selected metrics
+  * [Scatter plot](#scatter-plot) - View the correlation between a selected hyperparameter and metric
 * Scalars - Compare experiment metrics:
   * [Values](#tabular-scalar-comparison) - Compare minimal, maximal or last reported values in a concise comparison 
   table
@@ -59,6 +60,7 @@ The comparison pages provide the following views:
 * [Side-by-side textual comparison](#side-by-side-textual-comparison)
 * [Tabular scalar comparison](#tabular-scalar-comparison)
 * [Parallel coordinates](#parallel-coordinates-mode) for parameter impact on metric
+* [Scatter plot](#scatter-plot)
 * [Overlaid plot comparison](#plot-comparison)
 * Side-by-side [debug sample](#side-by-side-debug-sample-comparison) and [plot](#plot-comparison) comparison
 
@@ -101,11 +103,10 @@ Switch on the **Show row extremes** toggle to highlight each variant's maximum a
 
 ### Parallel Coordinates Mode
 
-The **Hyperparameters** tab's **Parallel Coordinates** comparison shows experiments' hyperparameter impact on a specific metric.
-
-**To compare by metric:**
-1. Under **Performance Metric**, select a metric to compare for
-1. Select the metric values to use in the plot:
+The **Hyperparameters** tab's **Parallel Coordinates** comparison shows experiments' hyperparameter impact on specified 
+metrics:
+1. Under **Performance Metrics**, select a metrics to compare for
+1. Select the values to use for each metric in the plot (can select multiple):
     * LAST - The final value, or the most recent value, for currently running experiments 
     * MIN - Minimal value 
     * MAX - Maximal value
@@ -121,6 +122,23 @@ To focus on a specific experiment, hover over its name in the graph legend.
 
 To hide an experiment, click its name in the graph legend (click again to bring back).
 
+### Scatter Plot 
+The **Hyperparameters** tab's **Scatter Plot** comparison shows experiments' correlation between a selected 
+hyperparameter and metric.
+
+To show the value distribution:
+* Select the **Plot Axes**:
+   1. Under Y-axis select the metric and the metric values to use in the plot:
+      * **LAST** - The final value, or the most recent value, for currently running experiments
+      * **MIN** - Minimal value
+      * **MAX** - Maximal value
+   1. Under X-axis select the hyperparameter.
+
+Hovering over each datapoint in the resulting plot will show the experiment name and the metric and parameter value for that 
+point. You can add additional metrics and hyperparameters values to the datapoint tooltip through **ADDITIONAL DATA POINT INFORMATION**.
+
+![Comparison scatter plot](../img/webapp_compare_scatter.png)
+
 ### Plot Comparison
 The **Scalars** (Graph view) and **Plots** tabs compare experiments' plots.
 

docs/webapp/webapp_model_comparing.md

@@ -49,7 +49,7 @@ The comparison tabs provides the following views:
 ### Side-by-side Textual Comparison
 
 In the **Details** and **Network** tabs, you can view differences in the models' nominal 
-values. **Details** displays the models' general information, labels, and metadata. **Network** displays the models' 
+values. **Details** displays the models' general information, labels, metadata, and lineage. **Network** displays the models' 
 configuration. Each model's 
 information is displayed in a column, so each field is lined up side-by-side. 
 

docs/webapp/webapp_orchestration_dash.md

@@ -43,7 +43,9 @@ Category sections display the resource count and utilization for:
 
 Hover over any of this data to see the number of currently idle machines. 
 
-Use the **Event Log** to view updates of worker events: worker addition/removal, worker has become idle/busy. 
+Use the **Event Log** to view updates of worker events: worker addition/removal, worker has become idle/busy. Hover over 
+the log to download (<img src="/docs/latest/icons/ico-download.svg" alt="Download" className="icon size-md space-sm" />) 
+it or open the expanded view (<img src="/docs/latest/icons/ico-maximize.svg" alt="Maximize" className="icon size-sm space-sm" />).
 
 ## Resource Graph
 

docs/webapp/webapp_profile.md

@@ -8,19 +8,21 @@ To navigate to the Settings page, click the <img src="/docs/latest/icons/ico-me.
 button in the top right corner of the web UI screen, then click **Settings**. 
 
 The Settings page consists of the following sections:
-* [Profile](#profile) - You basic user information
-* [Configuration](#configuration) - Control general system behavior settings and input storage access credentials
-* [Workspace](#workspace)  
-    * [ClearML credentials](#clearml-credentials) - Create client credentials for ClearML and ClearML Agent to use 
-    * [Configuration vault](#configuration-vault) (ClearML Enterprise Server) - Define global ClearML client settings
-      that are applied to all ClearML and ClearML Agent instances (which use the workspace's access 
-      credentials)
-* [Administrator Vaults](#administrator-vaults) (ClearML Enterprise Server) - Manage user-group level configuration 
-  vaults to apply ClearML client settings to all members of the user groups
-* [Users & Groups](#users--groups) - Manage the users that have access to a workspace
-* [Access Rules](#access-rules) (ClearML Enterprise Server) - Manage per-resource access privileges 
-* [Identity Providers](#identity-providers) (ClearML Enterprise Server) - Manage server identity providers
-* [Usage & Billing](#usage--billing) (ClearML Hosted Service) - View current usage information and billing details 
+* User Settings:
+  * [Profile](#profile) - You basic user information
+  * [Configuration](#configuration) - Control general system behavior settings and input storage access credentials
+  * [Workspace](#workspace)  
+      * [ClearML credentials](#clearml-credentials) - Create client credentials for ClearML and ClearML Agent to use 
+      * [Configuration vault](#configuration-vault) (ClearML Enterprise Server) - Define global ClearML client settings
+        that are applied to all ClearML and ClearML Agent instances (which use the workspace's access 
+        credentials)
+* Administrator Settings:
+  * [Administrator Vaults](#administrator-vaults) (ClearML Enterprise Server) - Manage user-group level configuration 
+    vaults to apply ClearML client settings to all members of the user groups
+  * [Users & Groups](#users--groups) - Manage the users that have access to a workspace
+  * [Access Rules](#access-rules) (ClearML Enterprise Server) - Manage per-resource access privileges 
+  * [Identity Providers](#identity-providers) (ClearML Enterprise Server) - Manage server identity providers
+  * [Usage & Billing](#usage--billing) (ClearML Hosted Service) - View current usage information and billing details 
 
 ## Profile 
 The profile tab presents user information.
@@ -238,6 +240,88 @@ Removed users lose access to your workspace's resources (tasks, models, etc.) an
 revoked. Tasks and associated artifacts logged to your workspace by a removed user will remain in your workspace. The 
 user can only rejoin your workspace when you re-invite them. 
 
+### Service Accounts
+
+:::important Enterprise Feature
+This feature is available under the ClearML Enterprise plan.
+:::
+
+Service accounts are ClearML users that provide services with access to the ClearML API, but not the 
+UI. Administrators can create access credentials for service accounts to use them for different ClearML Agents, 
+automations, and more. 
+
+A service account has all the privileges of a normal user in ClearML, with the following limitations:
+* Service accounts cannot be used to access the UI
+* Service accounts can be used to facilitate running tasks under the identity of each task's owner ("Impersonation"). 
+  * Used to run an agent using the command-line, this will allow you to specify the `--use-owner-token` option.
+  * Used to run the ClearML Agent Helm Chart, this will allow you to specify `values.agentk8s.useOwnerToken: true` option.
+  * Used to run an Autoscaler application, this will allow you to make use of the `Apply Task Owner Vault Configuration`
+  option.
+
+:::info Access Rules 
+When [access controls](#access-rules) are provisioned, they apply to service accounts the same as for ClearML users.
+Therefore, in order to use a service account to run an agent in daemon mode, the service account must have access to the 
+queue the agent will service.
+:::
+
+The **SERVICE ACCOUNTS** table lists workspace service accounts. 
+Each row of the table includes: 
+* Name - Service account name 
+* [User groups](#user-groups)
+* User ID
+* Credentials - Number of credentials currently available to the account
+* Last active time
+
+Hover over a service account in the table to **Edit** or **Delete** it.
+
+![Service accounts](../img/settings_service_accounts.png)
+
+#### Creating a Service Account
+
+To create a service account:
+1. Click **+ ADD SERVICE ACCOUNT**
+2. In the **ADD SERVICE ACCOUNT** modal input a name for the new account. Select `Allow impersonation` to allow the 
+   service account to assume the identity of a task owner 
+4. Click **Save**
+
+:::info Impersonation 
+Service accounts are members of the `Users` group, meaning they can access the resources available to all users. When 
+impersonation is enabled, a task run by the service account (i.e. by an agent or autoscaler using the service accounts' 
+credentials) is executed as if by the owner of the task, meaning it will have access to the task owner's configuration 
+vaults and to the resources that the task owner has access to. Impersonating an admin user does not mean the task's code 
+will have admin privileges.
+
+In case impersonation is not enabled: 
+* If you run an agent with `--use_owner_token` then the agent will fail. 
+* If you run an agent without `--use_owner_token`, the task will run with the service account's access rules, so make 
+  sure the account uses resources it has access to
+:::
+
+When a service account is created, an initial set of credentials is automatically generated. The dialog displays new 
+credentials, formatted as a ready-to-copy configuration file section.
+
+#### Service Account Credentials 
+
+To generate new credentials for a service account:
+1. Hover over the account's row on the table
+2. Click the <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" /> button, which
+   opens the editing panel
+3. Click **Create new credentials**
+
+The dialog displays new credentials, formatted as a ready-to-copy configuration file section.
+
+To revoke a set of credentials:
+1. In the editing panel, hover of the relevant credential's row
+2. Click the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> button
+
+#### Deleting Service Account
+Deleting a service account will revoke its credentials, causing agents using the account's credentials to fail. 
+Tasks and associated artifacts logged to your workspace by a service account will remain in your workspace.
+
+To delete a service account:
+1. Hover over the account's row on the table
+1. Click the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> button
+
 ### User Groups
 
 :::important Enterprise Feature
@@ -249,7 +333,7 @@ Administrators can define user groups, which can be used for access privilege ma
 multiple user groups.
 
 The system includes three pre-configured groups that can't be removed: 
-* `Users` - All users. Can't be modified
+* `Users` - All users (including [service accounts](#service-accounts)). Can't be modified
 * `Admins` - Have RW access to all resources (except queue modification), and can grant users / user groups access 
   permissions to workspace resources
 * `Queue admins` - Can create / delete / rename queues
@@ -288,8 +372,8 @@ otherwise provided individually or to another group they are members of).
 This feature is available under the ClearML Enterprise plan
 :::
 
-Workspace administrators can use the **Access Rules** page to manage workspace permissions, by specifying which users 
-and/or user groups have access permissions to the following workspace resources:
+Workspace administrators can use the **Access Rules** page to manage workspace permissions, by specifying which users,
+service accounts, and/or user groups have access permissions to the following workspace resources:
  
 * [Projects](../fundamentals/projects.md)
 * [Tasks](../fundamentals/task.md) 
@@ -308,9 +392,10 @@ Access privileges can be viewed, defined, and edited in the **Access Rules** tab
    specific project or task), click the input box, and select the object from the list that appears. Filter the 
    list by typing part of the desired object name
 1. Select the permission type - **Read Only** or **Read & Modify**
-1. Assign users and/or [user groups](#user-groups) to be given access. Click the desired input box, and select the 
-   users / groups from the list that appears. Filter the list by typing part of the desired object name. To revoke 
-   access, hover over a user's or group's row and click the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> 
+1. Assign users, [service accounts](#service-accounts), and/or [user groups](#user-groups) to be given access. Click the 
+   desired input box, and select the users / service accounts / groups from the list that appears. Filter the list by 
+   typing part of the desired object name. To revoke 
+   access, hover over a user's, service account's, or group's row and click the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> 
    button
 1. Click **SAVE**
 
@@ -324,22 +409,22 @@ not have access to another task in the project, unless explicitly granted.
 1. Hover over the access rule's row on the table
 1. Click the <img src="/docs/latest/icons/ico-edit.svg" alt="Edit Pencil" className="icon size-md" /> button
 1. Change the resource, resource object, and permission type as desired
-1. Edit access rule users / groups (see details [here](#creating-access-rules))
+1. Edit access rule users / service accounts / groups (see details [here](#creating-access-rules))
 1. Click **SAVE**
 
 ### Deleting Access Rules
 1. Hover over the access rule's row on the **Access Rules** table
 1. Click the <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> button
 
-All users and access groups who had been assigned to the deleted access rule, will lose the access privileges granted by
-that rule (unless otherwise provided by a different rule)
+All users, service accounts, and user groups who had been assigned to the deleted access rule, will lose the access privileges granted by
+that rule (unless otherwise provided by a different rule).
 
 ### Filtering Access Rules Table
 
 The access rules table can be filtered by resource type and by target resource and users / groups. 
 * **To filter by resource**, click the **View** dropdown menu and select the desired resource
-* **To filter by target resource or users / groups**, click <img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" />
-on the respective column and select the users / groups to view from the list that appears. 
+* **To filter by target resource or users / groups / service accounts**, click <img src="/docs/latest/icons/ico-filter-off.svg" alt="Filter" className="icon size-md" />
+on the respective column and select the users / groups / service accounts to view from the list that appears. 
 
 ## Identity Providers
 

sidebars.js

@@ -40,9 +40,9 @@ module.exports = {
         {'Cloud Autoscaling': [
             'cloud_autoscaling/autoscaling_overview',
              {'Autoscaler Apps': [
-                    {type: 'ref', id: 'webapp/applications/apps_gpu_compute'},
-                    {type: 'ref', id: 'webapp/applications/apps_aws_autoscaler'},
-                    {type: 'ref', id: 'webapp/applications/apps_gcp_autoscaler'},
+                    'webapp/applications/apps_gpu_compute',
+                    'webapp/applications/apps_aws_autoscaler',
+                    'webapp/applications/apps_gcp_autoscaler',
                  ]
              }
              ]
@@ -119,9 +119,6 @@ module.exports = {
             {
                 'ClearML Applications': [
                     'webapp/applications/apps_overview',
-                    'webapp/applications/apps_gpu_compute',
-                    'webapp/applications/apps_aws_autoscaler',
-                    'webapp/applications/apps_gcp_autoscaler',
                     'webapp/applications/apps_hpo',
                     'webapp/applications/apps_dashboard',
                     'webapp/applications/apps_task_scheduler',