Add ClearML session command line options (#18)

This commit is contained in:
pollfly 2021-07-20 10:45:22 +03:00 committed by GitHub
parent b0e148a3fe
commit 63ed502115
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 87 additions and 29 deletions

View File

@ -11,26 +11,47 @@ in the UI and send it for long-term training on a remote machine.
**If you are not that lucky**, this section is for you :) **If you are not that lucky**, this section is for you :)
## What does Clearml Session do? ## What does ClearML Session do?
`clearml-session` is a feature that allows to launch a session of Jupyterlab and VS Code, and to execute code on a remote `clearml-session` is a feature that allows to launch a session of Jupyterlab and VS Code, and to execute code on a remote
machine that better meets resource needs. With this feature, local links are provided, which can be used to access machine that better meets resource needs. With this feature, local links are provided, which can be used to access
JupyterLab and VS Code on a remote machine over a secure and encrypted SSH connection. JupyterLab and VS Code on a remote machine over a secure and encrypted SSH connection.
![image](../img/clearml_session_jupyter.png) <details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">Jupyter-Lab Window</summary>
<div className="cml-expansion-panel-content">
![image](../img/session_jupyter.png)
</div>
</details>
<br/>
<details className="cml-expansion-panel screenshot">
<summary className="cml-expansion-panel-summary">VS Code Window</summary>
<div className="cml-expansion-panel-content">
![image](../img/session_vs_code.png)
</div>
</details>
<!--![image](../img/clearml_session_jupyter.png)-->
## How it Works ## How it Works
ClearML allows to leverage a resource (e.g. GPU or CPU machine) by utilizing the [ClearML Agent](../clearml_agent). ClearML allows to leverage a resource (e.g. GPU or CPU machine) by utilizing the [ClearML Agent](../clearml_agent).
A ClearML Agent will be executed on target machine, and ClearML Session will instruct it to execute the Jupyter \ VSCode server to develop remotely. A ClearML Agent will run on a target machine, and ClearML Session will instruct it to execute the Jupyter \ VS Code
After entering a `clearml-session` command with all server to develop remotely.
specifications: After entering a `clearml-session` command with all specifications:
1. `clearml-session` creates a new [Task](../fundamentals/task.md) that is responsible for setting up the SSH and 1. `clearml-session` creates a new [Task](../fundamentals/task.md) that is responsible for setting up the SSH and
JupyterLab / VSCode environment, according to your specifications, on the host machine. JupyterLab / VS Code environment according to your specifications on the host machine.
1. The Task is enqueued to the queue ClearML Agent listens to and then executed by it. It will download the appropriate server and execute it. 1. The Task is enqueued, and a ClearML Agent pulls and executes it. The agent downloads the appropriate server and
launches it.
1. Once the Agent finishes the initial setup of the interactive Task, the local `cleaml-session` connects to the host 1. Once the agent finishes the initial setup of the interactive Task, the local `cleaml-session` connects to the host
machine via SSH, and tunnels both SSH and JupyterLab over the SSH connection. If a specific Docker was specified, the machine via SSH, and tunnels both SSH and JupyterLab over the SSH connection. If a specific Docker was specified, the
JupyterLab environment will run inside the Docker. JupyterLab environment will run inside the Docker.
@ -52,16 +73,16 @@ specifications:
To run a session inside a Docker container, use the `--docker` flag and enter the docker image to use in the interactive To run a session inside a Docker container, use the `--docker` flag and enter the docker image to use in the interactive
session. session.
### Passing requirements ### Installing requirements
`clearml-session` can download required Python packages. `clearml-session` can install required Python packages when setting up the remote environment. A `requirement.txt` file
A `requirement.txt` file can be attached to the command using `--requirements </file/location.txt>`. can be attached to the command using `--requirements </file/location.txt>`.
Alternatively, packages can be manually specified, using `--packages "<package_name>"` Alternatively, packages can be manually specified, using `--packages "<package_name>"`
(for example `--packages "keras" "clearml"`) and they'll be automatically installed. (for example `--packages "keras" "clearml"`), and they'll be automatically installed.
### Passing Git credentials ### Accessing a git repository
To send local .git-credentials file to the interactive session, add a `--git-credentials` flag and set it to `True`. To access a git repository remotely, add a `--git-credentials` flag and set it to `true`, so the local .git-credentials
This is helpful if working on private git repositories and allows for seemless cloning and tracking of git references, file will be sent to the interactive session. This is helpful if working on private git repositories, and it allows for seamless
including untracked changes. cloning and tracking of git references, including untracked changes.
### Re-launching and shutting down sessions ### Re-launching and shutting down sessions
If a `clearml-session` was launched locally and is still running on a remote machine, users can easily reconnect to it. If a `clearml-session` was launched locally and is still running on a remote machine, users can easily reconnect to it.
@ -72,7 +93,7 @@ to an existing session will show up:
Connect to active session id=c7302b564aa945408aaa40ac5c69399c [Y]/n?` Connect to active session id=c7302b564aa945408aaa40ac5c69399c [Y]/n?`
``` ```
If multiple sessions were launched from a local machine and are still active, choose the session to reconnect to: If multiple sessions were launched from a local machine and are still active, choose the desired session:
```console ```console
Active sessions: Active sessions:
@ -84,11 +105,13 @@ Connect to session [0-1] or 'N' to skip
To shut down a remote session, which will free the `clearml-agent` and close the CLI, enter "Shutdown". If a session To shut down a remote session, which will free the `clearml-agent` and close the CLI, enter "Shutdown". If a session
is shutdown, there is no option to reconnect to it. is shutdown, there is no option to reconnect to it.
### Connecting to existing session ### Connecting to an existing session
If a `clearml-session` is running remotely, it's possible to continue working on the session from any machine. Starting a If a `clearml-session` is running remotely, it's possible to continue working on the session from any machine.
session initializes a Task with a unique ID in the ClearML Server. To connect to an existing session: When `clearml-session` is launched, it initializes a task with a unique ID in the ClearML Server.
1. Go to the ClearML UI, find the interactive session Task (by default it's in project "DevOps").
1. Click on the ID button to the right of the Task name, and copy the unique ID. To connect to an existing session:
1. Go to the web UI, find the interactive session task (by default, it's in project "DevOps").
1. Click on the ID button to the right of the task name, and copy the unique ID.
1. Enter the following command: `clearml-session --attach <session_id>`. 1. Enter the following command: `clearml-session --attach <session_id>`.
1. Click on the JupyterLab / VS Code link that is outputted, or connect directly to the SSH session 1. Click on the JupyterLab / VS Code link that is outputted, or connect directly to the SSH session
@ -107,3 +130,27 @@ The Task must be connected to a git repository, since currently single script de
1. Enter the following command: `clearml-session --debugging-session <experiment_id_here>` 1. Enter the following command: `clearml-session --debugging-session <experiment_id_here>`
1. Click on the JupyterLab / VS Code link, or connect directly to the SSH session. 1. Click on the JupyterLab / VS Code link, or connect directly to the SSH session.
1. In JupyterLab / VS Code, access the experiment's repository in the `environment/task_repository` folder. 1. In JupyterLab / VS Code, access the experiment's repository in the `environment/task_repository` folder.
### Command line options
<div className="tbl-cmd">
| Command line options | Description | Default value |
|-----|---|---|
| `--jupyter-lab` | Download a Jupyter-Lab environment | `true` |
| `--vscode-server` | Download a VSCode environment | `true` |
| `--public-ip` | Register the public IP of the remote machine (if you are running the session on a public cloud) | Session runs on the machine whose agent is executing the session|
| `--init-script` | Specify a BASH init script file to be executed when the interactive session is being set up | `none` or previously entered BASH script |
| `--user-folder` | Specify the path for the session's remote base folder for the session | Home folder(`~/`) or previously entered user folder path |
| `--config-file` | Specify a path to another configuration file for `clearml-session` to store its previous state | `.clearml_session.json` or previously entered configuration file |
| `--remote-gateway` | Specify a gateway IP to pass to the interactive session, if an external address needs to be accessed | `none`|
| `--base-task-id` | Pass the ID of a task that will become the base task, so the session will use its configurations | `none` or the previously entered base task |
| `--disable-keepalive` | Disable transparent proxy that keep sockets alive to maintain the connection to the remote resource | `false` |
| `--queue-excluded-tag` | The queue option list will exclude queues with specified tags. See the `tags` parameter in the [queues.create](../references/api/endpoints#post-queuescreate) API call | `none` |
| `--queue-include-tag` | The queue option list will include only queues with specified tags. See the `tags` parameter in the [queues.create](../references/api/endpoints#post-queuescreate) API call | `none` |
| `--skip-docker-network` | Pass the `--network host` flag to the Docker that is launching the remote session. See [Networking using the host network](https://docs.docker.com/network/network-tutorial-host/) | `false`|
| `--username`| Set your own SSH username for the interactive session | `root` or a previously used username |
| `--password`| Set your own SSH password for the interactive session | A randomly generated password or a previously used one |
</div>

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

View File

@ -492,6 +492,17 @@ html[data-theme="light"] .footer__link-item[href*="stackoverflow"] {
border-bottom-right-radius: var(--ifm-card-border-radius); border-bottom-right-radius: var(--ifm-card-border-radius);
} }
.markdown table code {
font-size: 12px;
}
@media only screen and (min-width: 1400px) {
.tbl-cmd tbody tr td:first-child code {
white-space: nowrap;
}
}
html[data-theme="dark"] .markdown thead th { html[data-theme="dark"] .markdown thead th {
color: var(--ifm-color-gray-600); color: var(--ifm-color-gray-600);
} }