2021-05-13 23:48:51 +00:00
---
title: Remote Jupyter Tutorial
---
2023-08-09 10:28:25 +00:00
In this tutorial you will learn how to launch a remote interactive session on Jupyter Notebook using `clearml-session` .
You will be using two machines. A local one, where you will be using an interactive session of Jupyter, and a remote machine,
2021-05-13 23:48:51 +00:00
where a `clearml-agent` will run and spin an instance of the remote session.
## Prerequisites
* `clearml-session` package installed (`pip install clearml-session`)
* At least one `clearml-agent` running on a **remote** host. See [installation details ](../../clearml_agent.md#installation ).
Configure the `clearml-agent` to listen to the `default` queue (`clearml-agent daemon --queue default`)
* An SSH client installed on machine being used. To verify, open terminal, execute `ssh` , and if no error is received,
it should be good to go.
## Steps
2021-08-23 08:03:34 +00:00
### Step 1: Launch `clearml-session`
2022-01-24 13:42:17 +00:00
Execute the following command:
2021-08-23 08:03:34 +00:00
```bash
clearml-session --docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 --packages "clearml" "tensorflow>=2.2" "keras" --queue default
```
2022-01-24 13:42:17 +00:00
This sets the following arguments:
* `--docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` - Docker image
2021-05-13 23:48:51 +00:00
2022-01-24 13:42:17 +00:00
* `--packages "clearml" "tensorflow>=2.2" "keras"` - Required Python packages
2021-05-13 23:48:51 +00:00
2022-01-24 13:42:17 +00:00
* `--queue default` - Selected queue to launch the session from
2021-08-23 08:03:34 +00:00
:::note
2022-01-18 11:23:47 +00:00
Enter a project name using `--project <name>` . If no project is input, the default project
2021-08-23 08:03:34 +00:00
name is "DevOps"
:::
2021-05-13 23:48:51 +00:00
2021-08-23 08:03:34 +00:00
After launching the command, the `clearml-agent` listening to the `default` queue spins a remote Jupyter environment with
the specifications. It will automatically connect to the docker on the remote machine.
2021-05-13 23:48:51 +00:00
2021-08-23 08:03:34 +00:00
The terminal should return output with the session's configuration details, which should look something like this:
2021-05-13 23:48:51 +00:00
2021-08-23 08:03:34 +00:00
```console
Interactive session config:
{
"base_task_id": null,
"git_credentials": false,
"jupyter_lab": true,
"password": "0879348ae41fb944004ff89b9103f09592ec799f39ae34e5b71afb46976d5c83",
"queue": "default",
"vscode_server": true
}
```
2021-09-09 10:17:46 +00:00
### Step 2: Launch Interactive Session
2021-08-23 08:03:34 +00:00
When the CLI asks whether to `Launch interactive session [Y]/n?` , press 'Y' or 'Enter'.
The terminal should output information regarding the status of the environment-building process, which should look
something like this:
2021-05-13 23:48:51 +00:00
2021-08-23 08:03:34 +00:00
```console
Creating new session
New session created [id=35c0af81ae6541589dbae1efb747f388]
Waiting for remote machine allocation [id=35c0af81ae6541589dbae1efb747f388]
.Status [queued]
...Status [in_progress]
Remote machine allocated
Setting remote environment [Task id=35c0af81ae6541589dbae1efb747f388]
2022-03-13 13:07:06 +00:00
Setup process details: https://app.clear.ml/projects/60893b87b0b642679fde00db96e90359/experiments/35c0af81ae6541589dbae1efb747f388/output/log
2021-08-23 08:03:34 +00:00
Waiting for environment setup to complete [usually about 20-30 seconds]
```
2021-09-09 10:17:46 +00:00
### Step 3: Connect to Remote Notebook
2021-08-23 08:03:34 +00:00
Then the CLI will output a link to the ready environment:
```console
Interactive session is running:
SSH: ssh root@localhost -p 8022 [password: c5d19b3c0fa9784ba4f6aeb568c1e036a4fc2a4bc7f9bfc54a2c198d64ceb9c8]
Jupyter Lab URL: http://localhost:8878/?token=ff7e5e8b9e5493a01b1a72530d18181320630b95f442b419
VSCode server available at http://localhost:8898/
```
Click on the JupyterLab link, which will open the remote session
2021-05-13 23:48:51 +00:00
2021-08-23 08:03:34 +00:00
Now, let's execute some code in the remote session!
2021-09-09 10:17:46 +00:00
### Step 4: Execute Code
2021-08-23 08:03:34 +00:00
1. Open up a new Notebook.
2023-08-09 10:28:25 +00:00
1. In the first cell of the notebook, clone the [ClearML repository ](https://github.com/allegroai/clearml ):
2021-05-13 23:48:51 +00:00
!git clone https://github.com/allegroai/clearml.git
2023-08-09 10:28:25 +00:00
1. In the second cell of the notebook, run this [script ](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py )
from the cloned repository:
2021-05-13 23:48:51 +00:00
%run clearml/examples/frameworks/keras/keras_tensorboard.py
2023-08-09 10:28:25 +00:00
Look in the script, and notice that it makes use of ClearML, Keras, and TensorFlow, but you don't need to install these
packages in Jupyter, because you specified them in the `--packages` flag of `clearml-session` .
2021-05-13 23:48:51 +00:00
2021-09-09 10:17:46 +00:00
### Step 5: Shut Down Remote Session
2021-08-23 08:03:34 +00:00
To shut down the remote session, which will free the `clearml-agent` and close the CLI, enter "Shutdown".
2021-05-13 23:48:51 +00:00
2021-08-23 08:03:34 +00:00
```console
Connection is up and running
Enter "r" (or "reconnect") to reconnect the session (for example after suspend)
Ctrl-C (or "quit") to abort (remote session remains active)
or "Shutdown" to shutdown remote interactive session
```