From 085841ab0daf317ce2d863d5d88f0dc6a8065ba5 Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Mon, 10 Feb 2025 10:17:24 +0200 Subject: [PATCH] Terminology update (#1035) --- docs/apps/clearml_session.md | 14 +++++++------- docs/apps/clearml_task.md | 12 ++++++------ docs/clearml_agent.md | 6 +++--- docs/clearml_agent/clearml_agent_docker.md | 3 +-- .../clearml_agent_execution_env.md | 2 +- .../clearml_agent/executable_exp_containers.md | 6 +++--- .../clearml_agent/exp_environment_containers.md | 6 +++--- docs/guides/docker/extra_docker_shell_script.md | 2 +- docs/guides/ide/remote_jupyter_tutorial.md | 4 ++-- docs/webapp/applications/apps_jupyter_lab.md | 4 ++-- docs/webapp/applications/apps_ssh_session.md | 4 ++-- docs/webapp/applications/apps_vscode.md | 4 ++-- docs/webapp/settings/webapp_settings_profile.md | 2 +- docs/webapp/webapp_exp_tuning.md | 17 ++++++++--------- 14 files changed, 42 insertions(+), 44 deletions(-) diff --git a/docs/apps/clearml_session.md b/docs/apps/clearml_session.md index ea778caa..386d32ea 100644 --- a/docs/apps/clearml_session.md +++ b/docs/apps/clearml_session.md @@ -41,7 +41,7 @@ error, you are good to go. clearml-session ``` - You can add flags to set a Docker image, the remote SSH port, JupyterLab/VS Code versions, and more. See [CLI options](#command-line-options) + You can add flags to set a container image, the remote SSH port, JupyterLab/VS Code versions, and more. See [CLI options](#command-line-options) for all configuration options. `clearml-session` creates a new [Task](../fundamentals/task.md) that is responsible for setting up the SSH and @@ -57,8 +57,8 @@ error, you are good to go. launches it. 1. Once the agent finishes the initial setup of the interactive Task, the local `cleaml-session` connects to the host - machine via SSH, and tunnels both SSH and IDE over the SSH connection. If a Docker is specified, the - IDE environment runs inside the Docker. + machine via SSH, and tunnels both SSH and IDE over the SSH connection. If a container is specified, the + IDE environment runs inside of it. 1. The CLI outputs access links to the remote JupyterLab and VS Code sessions: @@ -106,8 +106,8 @@ To connect to an existing session: 1. Click on the JupyterLab / VS Code link that is outputted, or connect directly to the SSH session ## Features -### Running in Docker -To run a session inside a Docker container, use the `--docker` flag and enter the docker image to use in the interactive +### Running in a Container +To run a session inside a container, use the `--docker` flag and enter the image to use in the interactive session. ### Kubernetes Support @@ -224,8 +224,8 @@ clearml-session --continue-session --store-workspace ~/workspace | `--disable-fingerprint-check` | If set, bypass the remote SSH server fingerprint verification process | `none` | | `--disable-session-cleanup` | If `True`, previous interactive sessions are not deleted | `false`| | `--disable-store-defaults` | If set, do not store current setup as new default configuration| `none`| -| `--docker`| Select the docker image to use in the interactive session |`nvidia/cuda:11.6.2-runtime-ubuntu20.04` or previously used docker image| -| `--docker-args` | Add additional arguments for the docker image to use in the interactive session | `none` or the previously used docker-args | +| `--docker`| Select the image to use in the interactive session |`nvidia/cuda:11.6.2-runtime-ubuntu20.04` or previously used image| +| `--docker-args` | Add additional arguments for the docker image to use in the interactive session | `none` or the previously used `docker-args` | | `--force_dropbear`| Force using `dropbear` instead of SSHd |`none`| | `--git-credentials` | If `True`, local `.git-credentials` file is sent to the interactive session.| `false`| | `--init-script` | Specify a BASH init script file to be executed when the interactive session is being set up | `none` or previously entered BASH script | diff --git a/docs/apps/clearml_task.md b/docs/apps/clearml_task.md index cd390f6c..dd7fb019 100644 --- a/docs/apps/clearml_task.md +++ b/docs/apps/clearml_task.md @@ -18,7 +18,7 @@ line arguments, Python module dependencies, and a requirements.txt file! ## How Does ClearML Task Work? 1. Execute `clearml-task`, specifying the ClearML target project and task name, along with your script (and repository / commit / branch). - Optionally, specify an execution queue and Docker image to use. + Optionally, specify an execution queue and container image to use. 1. `clearml-task` does its magic! It creates a new [ClearML Task](../fundamentals/task.md), and, if so directed, enqueues it for execution by a ClearML Agent. 1. While the Task is running on the remote machine, all its console outputs are logged in real-time, alongside your @@ -26,9 +26,9 @@ line arguments, Python module dependencies, and a requirements.txt file! (a link to your task details page in the ClearML Web UI is printed as ClearML Task creates the task). ## Execution Configuration -### Docker -Specify a Docker container to run the code in with the `--docker ` option. -The ClearML Agent pulls it from Docker Hub or a Docker artifactory automatically. +### Container +Specify a container to run the code in with the `--docker ` option. +The ClearML Agent pulls it from Docker Hub or a container artifactory automatically. ### Package Dependencies `clearml-task` automatically finds the `requirements.txt` file in remote repositories. @@ -61,8 +61,8 @@ errors in identifying the correct default branch. | `--branch` | Select repository branch / tag. By default, latest commit from the master branch | No | | `--commit` | Select commit ID to use. By default, latest commit, or local commit ID when using local repository | No | | `--cwd` | Working directory to launch the script from. Relative to repo root or local `--folder` | No | -| `--docker` | Select the Docker image to use in the remote task | No | -| `--docker_bash_setup_script` | Add a bash script to be executed inside the Docker before setting up the task's environment | No | +| `--docker` | Select the container image to use in the remote task | No | +| `--docker_bash_setup_script` | Add a bash script to be executed inside the container before setting up the task's environment | No | | `--docker_args` | Add Docker arguments. Pass a single string in the following format: `--docker_args ""`. For example: `--docker_args "-v some_dir_1:other_dir_1 -v some_dir_2:other_dir_2"` | No | | `--folder` | Execute the code from a local folder. Notice, it assumes a git repository already exists. Current state of the repo (commit ID and uncommitted changes) is logged and replicated on the remote machine | No | | `--import-offline-session`| Specify the path to the offline session you want to import.| No | diff --git a/docs/clearml_agent.md b/docs/clearml_agent.md index a6ce6b07..d6be238c 100644 --- a/docs/clearml_agent.md +++ b/docs/clearml_agent.md @@ -29,7 +29,7 @@ The preceding diagram demonstrates a typical flow where an agent executes a task 1. Enqueue a task for execution on the queue. 1. The agent pulls the task from the queue. -1. The agent launches a docker container in which to run the task's code. +1. The agent launches a container in which to run the task's code. 1. The task's execution environment is set up: 1. Execute any custom setup script configured. 1. Install any required system packages. @@ -39,8 +39,8 @@ The preceding diagram demonstrates a typical flow where an agent executes a task 1. The task's script/code is executed. :::note Python Version -ClearML Agent uses the Python version available in the environment or docker in which it executes the code. It does not -install Python, so make sure to use a docker or environment with the version you need. +ClearML Agent uses the Python version available in the environment or container in which it executes the code. It does not +install Python, so make sure to use a container or environment with the version you need. ::: While the agent is running, it continuously reports system metrics to the ClearML Server (these can be monitored in the diff --git a/docs/clearml_agent/clearml_agent_docker.md b/docs/clearml_agent/clearml_agent_docker.md index fe42f1ef..99bdb1e2 100644 --- a/docs/clearml_agent/clearml_agent_docker.md +++ b/docs/clearml_agent/clearml_agent_docker.md @@ -39,8 +39,7 @@ clearml-agent build --id --docker --target You can add the Docker container as the base Docker image to a task, using one of the following methods: -- Using the **ClearML Web UI** - See [Base Docker image](../webapp/webapp_exp_tuning.md#base-docker-image) on the "Tuning - Tasks" page. +- Using the **ClearML Web UI** - See [Default Container](../webapp/webapp_exp_tuning.md#default-container). - In the ClearML configuration file - Use the ClearML configuration file [`agent.default_docker`](../configs/clearml_conf.md#agentdefault_docker) options. diff --git a/docs/clearml_agent/clearml_agent_execution_env.md b/docs/clearml_agent/clearml_agent_execution_env.md index 6c70817f..0180da86 100644 --- a/docs/clearml_agent/clearml_agent_execution_env.md +++ b/docs/clearml_agent/clearml_agent_execution_env.md @@ -26,7 +26,7 @@ If you are using pyenv to control the environment where you use ClearML Agent, y * Install poetry with the deprecated `get-poetry.py` installer ::: -## Docker Mode +## Docker Mode :::note notes * Docker Mode is only supported in Linux. * Docker Mode requires docker service v19.03 or higher installed. diff --git a/docs/guides/clearml_agent/executable_exp_containers.md b/docs/guides/clearml_agent/executable_exp_containers.md index d1a06eaf..910fb819 100644 --- a/docs/guides/clearml_agent/executable_exp_containers.md +++ b/docs/guides/clearml_agent/executable_exp_containers.md @@ -3,7 +3,7 @@ title: Executable Task Containers --- This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)'s [`build`](../../clearml_agent/clearml_agent_ref.md#build) -command to package a task into an executable container. In this example, you will build a Docker image that, when +command to package a task into an executable container. In this example, you will build a Container image that, when run, will automatically execute the [keras_tensorboard.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) script. @@ -44,8 +44,8 @@ script. If the container will not make use of a GPU, add the `--cpu-only` flag. ::: - This command will create a Docker container, set up with the execution environment for this task in the - specified `--target` folder. When the Docker container is launched, it will clone the task specified with `id` and + This command will create a container, set up with the execution environment for this task in the + specified `--target` folder. When the container is launched, it will clone the task specified with `id` and execute the clone (as designated by the `--entry-point` parameter). 1. Run the Docker, pointing to the new container: diff --git a/docs/guides/clearml_agent/exp_environment_containers.md b/docs/guides/clearml_agent/exp_environment_containers.md index fa95166a..6bc2145d 100644 --- a/docs/guides/clearml_agent/exp_environment_containers.md +++ b/docs/guides/clearml_agent/exp_environment_containers.md @@ -3,7 +3,7 @@ title: Task Environment Containers --- This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)'s [`build`](../../clearml_agent/clearml_agent_ref.md#build) -command to build a Docker container replicating the execution environment of an existing task. ClearML Agents can make +command to build a container replicating the execution environment of an existing task. ClearML Agents can make use of such containers to execute tasks without having to set up their environment every time. A use case for this would be manual hyperparameter optimization, where a base task can be used to create a container to @@ -36,7 +36,7 @@ be used when running optimization tasks. ``` This ID will be used in the following section. -## Building the Docker Container +## Building the Container Execute the following command to build the container. Input the ID of the task created above. ```console @@ -56,7 +56,7 @@ Committing docker container to: new_docker sha256:460453b93ct1989fd1c6637c236e544031c4d378581433fc0b961103ce206af1 ``` -## Using the New Docker Container +## Using the New Container Make use of the container you've just built by having a ClearML agent make use of it for executing a new task: 1. In the [ClearML Web UI](../../webapp/webapp_overview.md), go to the "examples" project, "Keras with TensorBoard diff --git a/docs/guides/docker/extra_docker_shell_script.md b/docs/guides/docker/extra_docker_shell_script.md index d1eda979..ae920334 100644 --- a/docs/guides/docker/extra_docker_shell_script.md +++ b/docs/guides/docker/extra_docker_shell_script.md @@ -2,7 +2,7 @@ title: Extra Docker Shell Script --- -When using `clearml-agent`, an agent recreates an entire execution environment, be it by pulling the docker container or +When using `clearml-agent`, an agent recreates an entire execution environment, be it by pulling a container or installing specified packages, and then executes the code on a remote machine. The Agent takes into account required Python packages, but sometimes, when using a Docker container, a user may need to use additional, non-Python tools. diff --git a/docs/guides/ide/remote_jupyter_tutorial.md b/docs/guides/ide/remote_jupyter_tutorial.md index 7054f9ed..69d464cb 100644 --- a/docs/guides/ide/remote_jupyter_tutorial.md +++ b/docs/guides/ide/remote_jupyter_tutorial.md @@ -27,7 +27,7 @@ clearml-session --docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 --packages This sets the following arguments: -* `--docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` - Docker image +* `--docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` - Container image * `--packages "clearml" "tensorflow>=2.2" "keras"` - Required Python packages @@ -39,7 +39,7 @@ name is `DevOps`. ::: After launching the command, the `clearml-agent` listening to the `default` queue spins a remote Jupyter environment with -the specifications. It will automatically connect to the docker on the remote machine. +the specifications. It will automatically connect to the container on the remote machine. The console should display the session's configuration details: diff --git a/docs/webapp/applications/apps_jupyter_lab.md b/docs/webapp/applications/apps_jupyter_lab.md index fb25e8a8..74dcd6a6 100644 --- a/docs/webapp/applications/apps_jupyter_lab.md +++ b/docs/webapp/applications/apps_jupyter_lab.md @@ -51,8 +51,8 @@ to open the app's instance launch form. * Repository * Branch * Commit -* **Docker** - * Image - Docker image used to run the IDE in +* **Container** + * Image - Container image used to run the IDE in * Docker arguments - `docker run` arguments, as a single string * **Extra Packages** - Extra Python packages to be installed * **Queue** - The queue serviced by the ClearML Agent that will execute the Jupyter Lab session diff --git a/docs/webapp/applications/apps_ssh_session.md b/docs/webapp/applications/apps_ssh_session.md index 113bb001..cf4ef5eb 100644 --- a/docs/webapp/applications/apps_ssh_session.md +++ b/docs/webapp/applications/apps_ssh_session.md @@ -68,8 +68,8 @@ values from the file, which can be modified before launching the app instance * Repository * Branch * Commit -* **Docker** - Input details to run the session in Docker container - * Image - Docker image to launch +* **Container** - Input details to run the session in Docker container + * Image - Container image to launch * Docker Arguments - Additional arguments for the Docker image * Init Script - Bash script that is executed upon container boot (comments are supported only at the beginning of the line) diff --git a/docs/webapp/applications/apps_vscode.md b/docs/webapp/applications/apps_vscode.md index 277683f2..3ad8b076 100644 --- a/docs/webapp/applications/apps_vscode.md +++ b/docs/webapp/applications/apps_vscode.md @@ -51,8 +51,8 @@ to open the app's instance launch form. * Repository * Branch * Commit -* **Docker** - * Image - Docker image used to run the IDE in +* **Container** + * Image - container image used to run the IDE in * Docker arguments - `docker run` arguments, as a single string * **Queue** - The queue serviced by the ClearML Agent that will execute the VS Code session * **Maximum idle time** (hours) - Maximum time of inactivity, after which the session will shut down. Configure idleness diff --git a/docs/webapp/settings/webapp_settings_profile.md b/docs/webapp/settings/webapp_settings_profile.md index 306792cc..38c0f00b 100644 --- a/docs/webapp/settings/webapp_settings_profile.md +++ b/docs/webapp/settings/webapp_settings_profile.md @@ -27,7 +27,7 @@ Disable for default desktop scale. * **Block running user's scripts in the browser** - Block any user and 3rd party scripts from running anywhere in the WebApp. Note that if enabled, the WebApp will not display debug samples, [Hyper-Dataset frame previews](../../hyperdatasets/previews.md), and embedded resources in [reports](../webapp_reports.md). -* **Hide specific container arguments** - Specify which Docker environment variable values should be hidden in logs. +* **Hide specific container arguments** - Specify which container environment variable values should be hidden in logs. When printed, the variable values are replaced with `********`. By default, `CLEARML_API_SECRET_KEY`, `CLEARML_AGENT_GIT_PASS`, `AWS_SECRET_ACCESS_KEY`, and `AZURE_STORAGE_KEY` values are redacted. To modify the hidden container argument list, click **Edit**. diff --git a/docs/webapp/webapp_exp_tuning.md b/docs/webapp/webapp_exp_tuning.md index b9261bcf..6c6ddd96 100644 --- a/docs/webapp/webapp_exp_tuning.md +++ b/docs/webapp/webapp_exp_tuning.md @@ -39,7 +39,7 @@ Web UI**, edit any of the following * [Source code](#source-code) * [Output destination for artifacts](#output-destination) -* [Base Docker image](#base-docker-image) +* [Default container](#default-container) * [Hyperparameters](#hyperparameters) - Parameters, TensorFlow Definitions, command line options, environment variables, and user-defined properties :::note @@ -71,23 +71,22 @@ and/or Reset functions. -#### Base Docker Image -Select a pre-configured Docker that **ClearML Agent** will use to remotely execute this task (see [Building Docker containers](../clearml_agent/clearml_agent_docker.md)). +#### Default Container +Select a pre-configured container that the [ClearML Agent](../clearml_agent.md) will use to remotely execute this task (see [Building Docker containers](../clearml_agent/clearml_agent_docker.md)). -**To add, change, or delete a base Docker image:** +**To add, change, or delete a default container:** -* In **EXECUTION** **>** **AGENT CONFIGURATION** **>** **BASE DOCKER IMAGE** **>** hover **>** **EDIT** **>** - Enter the base Docker image. +* In **EXECUTION** **>** **CONTAINER** **>** hover **>** **EDIT** **>** + Enter the default container image. :::important For a ClearML Agent to execute the task in a container, the agent must be running in -Docker mode: +[Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode): ```bash -clearml-agent daemon --queue --docker [optional default docker image to use] +clearml-agent daemon --queue --docker [optional default container image to use] ``` -For more information, see [Docker Mode](../clearml_agent/clearml_agent_execution_env.md#docker-mode). ::: #### Output Destination