ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-04-22 15:27:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add new heading to ClearML Session example (#46)

Browse Source
This commit is contained in:
pollfly 2021-08-23 11:03:34 +03:00 committed by GitHub
parent 0048cea0d8
commit 0767ffdd85
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 75 additions and 65 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

140
docs/guides/ide/remote_jupyter_tutorial.md
View File

@ -16,76 +16,84 @@ where a `clearml-agent` will run and spin an instance of the remote session.
## Steps ## Steps
1. Execute the `clearml-session` command with the following command line options:
```bash ### Step 1: Launch `clearml-session`
clearml-session --docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 --packages "clearml" "tensorflow>=2.2" "keras" --queue default
``` Execute the `clearml-session` command with the following command line options:
```bash
clearml-session --docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 --packages "clearml" "tensorflow>=2.2" "keras" --queue default
```
* Enter a docker image `--docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` * Enter a docker image `--docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04`
* Enter required python packages `--packages "clearml" "tensorflow>=2.2" "keras"` * Enter required python packages `--packages "clearml" "tensorflow>=2.2" "keras"`
* Specify the resource queue `--queue default`. * Specify the resource queue `--queue default`.
<br/> :::note
There is an option to enter a project name using `--project <name>`. If no project is input, the default project
name is "DevOps"
:::
After launching the command, the `clearml-agent` listening to the `default` queue spins a remote Jupyter environment with
the specifications. It will automatically connect to the docker on the remote machine.
The terminal should return output with the session's configuration details, which should look something like this:
:::note ```console
There is an option to enter a project name using `--project <name>`. If no project is inputted, the default project Interactive session config:
name is "DevOps" {
::: "base_task_id": null,
"git_credentials": false,
1. After launching the command, the `clearml-agent` listening to the `default` queue spins a remote Jupyter environment "jupyter_lab": true,
with the specifications. It will automatically connect to the docker on the remote machine. "password": "0879348ae41fb944004ff89b9103f09592ec799f39ae34e5b71afb46976d5c83",
"queue": "default",
The terminal should return output with the session's configuration details, which should look something like this: "vscode_server": true
}
```console ```
Interactive session config:
{
"base_task_id": null,
"git_credentials": false,
"jupyter_lab": true,
"password": "0879348ae41fb944004ff89b9103f09592ec799f39ae34e5b71afb46976d5c83",
"queue": "default",
"vscode_server": true
}
```
1. Press `Y` when the CLI will ask whether to `Launch interactive session [Y]/n?`. Press 'Y' or 'Enter'. ### Step 2: Launch interactive session
The terminal should output information regarding the status of the environment-building process, which should look When the CLI asks whether to `Launch interactive session [Y]/n?`, press 'Y' or 'Enter'.
something like this:
```console
Creating new session
New session created [id=35c0af81ae6541589dbae1efb747f388]
Waiting for remote machine allocation [id=35c0af81ae6541589dbae1efb747f388]
.Status [queued]
...Status [in_progress]
Remote machine allocated
Setting remote environment [Task id=35c0af81ae6541589dbae1efb747f388]
Setup process details: https://app.community.clear.ml/projects/60893b87b0b642679fde00db96e90359/experiments/35c0af81ae6541589dbae1efb747f388/output/log
Waiting for environment setup to complete [usually about 20-30 seconds]
```
Then the CLI will output a link to the ready environment: The terminal should output information regarding the status of the environment-building process, which should look
```console something like this:
Interactive session is running:
SSH: ssh root@localhost -p 8022 [password: c5d19b3c0fa9784ba4f6aeb568c1e036a4fc2a4bc7f9bfc54a2c198d64ceb9c8] ```console
Jupyter Lab URL: http://localhost:8878/?token=ff7e5e8b9e5493a01b1a72530d18181320630b95f442b419 Creating new session
VSCode server available at http://localhost:8898/ New session created [id=35c0af81ae6541589dbae1efb747f388]
``` Waiting for remote machine allocation [id=35c0af81ae6541589dbae1efb747f388]
.Status [queued]
...Status [in_progress]
Remote machine allocated
Setting remote environment [Task id=35c0af81ae6541589dbae1efb747f388]
Setup process details: https://app.community.clear.ml/projects/60893b87b0b642679fde00db96e90359/experiments/35c0af81ae6541589dbae1efb747f388/output/log
Waiting for environment setup to complete [usually about 20-30 seconds]
```
1. Click on the JupyterLab link, which will open the remote session ### Step 3: Connect to remote notebook
Then the CLI will output a link to the ready environment:
```console
Interactive session is running:
SSH: ssh root@localhost -p 8022 [password: c5d19b3c0fa9784ba4f6aeb568c1e036a4fc2a4bc7f9bfc54a2c198d64ceb9c8]
Jupyter Lab URL: http://localhost:8878/?token=ff7e5e8b9e5493a01b1a72530d18181320630b95f442b419
VSCode server available at http://localhost:8898/
```
Click on the JupyterLab link, which will open the remote session
1. Now, let's execute some code in the remote session! Open up a new Notebook. Now, let's execute some code in the remote session!
### Step 4: Execute code
1. In the first cell of the notebook, clone the [ClearML Repo](https://github.com/allegroai/events).
1. Open up a new Notebook.
1. In the first cell of the notebook, clone the [ClearML Repo](https://github.com/allegroai/clearml).
!git clone https://github.com/allegroai/clearml.git !git clone https://github.com/allegroai/clearml.git
1. In the second cell of the notebook, we are going to run this [script](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) 1. In the second cell of the notebook, we are going to run this [script](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py)
from the repository that we cloned. from the repository that we cloned.
@ -95,11 +103,13 @@ where a `clearml-agent` will run and spin an instance of the remote session.
Look in the script, and notice that it makes use of ClearML, Keras, and TensorFlow, but we don't need to install these Look in the script, and notice that it makes use of ClearML, Keras, and TensorFlow, but we don't need to install these
packages in Jupyter, because we specified them in the `--packages` flag of `clearml-session`. packages in Jupyter, because we specified them in the `--packages` flag of `clearml-session`.
1. To shut down the remote session, which will free the `clearml-agent` and close the CLI. Enter "Shutdown". ### Step 5: Shut down remote session
To shut down the remote session, which will free the `clearml-agent` and close the CLI, enter "Shutdown".
```console ```console
Connection is up and running Connection is up and running
Enter "r" (or "reconnect") to reconnect the session (for example after suspend) Enter "r" (or "reconnect") to reconnect the session (for example after suspend)
Ctrl-C (or "quit") to abort (remote session remains active) Ctrl-C (or "quit") to abort (remote session remains active)
or "Shutdown" to shutdown remote interactive session or "Shutdown" to shutdown remote interactive session
``` ```