Update ClearML Session page (#795)

docs/apps/clearml_session.md

@@ -143,6 +143,18 @@ To access a git repository remotely, add a `--git-credentials` flag and set it t
 file is sent to the interactive session. This is helpful if working on private git repositories, and it allows for seamless 
 cloning and tracking of git references, including untracked changes. 
 
+#### Uploading Local Files to Remote Session
+You can upload local files and directories from your local machine into the remote session by specifying their path with 
+`--upload-files <file_path>`. The entire content of the directory or file will be copied into your remote 
+`clearml-session` container under the `~/session-files/` directory.
+
+```commandline
+clearml-session --upload-files /mnt/data/stuff
+```
+
+You can upload your files in conjunction with the `--store-workspace` option to easily move workloads between local 
+development machines and remote machines with persistent workspace synchronization. See [Storing and Synchronizing Workspace](#storing-and-synchronizing-workspace).
+
 #### Starting a Debugging Session 
 You can debug previously executed experiments registered in the ClearML system on a remote interactive session. 
 Input into `clearml-session` the ID of a Task to debug, then `clearml-session` clones the experiment's git repository and 
@@ -158,6 +170,31 @@ The Task must be connected to a git repository, since currently single script de
 1. Click on the JupyterLab / VS Code link, or connect directly to the SSH session.
 1. In JupyterLab / VS Code, access the experiment's repository in the `environment/task_repository` folder. 
 
+#### Storing and Synchronizing Workspace
+You can store and sync your interactive session workspace with the `--store-workspace` option. `clearml-session`
+will automatically create a snapshot of your entire workspace when shutting it down, and later restore in a 
+new session on any remote machine. 
+
+Specify the remote workspace root-folder by adding `--store-workspace <workspace_root_folder>` to the command line. In 
+the remote session container, put all your code and data under the `<workspace_root_folder>` directory. When your 
+session is shut down, the workspace folder will be automatically packaged and stored on the ClearML file server. 
+
+```commandline
+clearml-session --store-workspace ~/workspace --docker python:3.10-bullseye
+```
+
+In your next `clearml-session` execution, specify `--store-workspace <workspace_root_folder>` again and `clearml-session` 
+will grab the previous workspace snapshot and restore it into the new remote container in `<workspace_root_folder>`.
+
+```commandline
+clearml-session --store-workspace ~/workspace --docker python:3.10-bullseye
+```
+
+To continue a specific session and restore its workspace, specify the session ID with `--continue-session <session_id>`:
+
+```commandline
+clearml-session --continue-session <session_id> --store-workspace ~/workspace --docker python:3.10-bullseye
+```
 
 ### Command Line Options
 
@@ -166,42 +203,42 @@ The Task must be connected to a git repository, since currently single script de
 | Command line options | Description | Default value |
 |-----|---|---|
 | `--attach`| Attach to running interactive session | Previous session|
-| `--project`| Set the project name to the interactive session task| `DevOps` |
+| `--base-task-id` | Pass the ID of a task that will become the base task, so the session will use its configurations | `none` or the previously entered base task |
-| `--shutdowm`, `-S`| Shut down an active session | Previous session|
+| `--config-file` | Specify a path to another configuration file for `clearml-session` to store its previous state | `.clearml_session.json` or previously entered configuration file |  
+| `--continue-session` | Pass the session of a previous session to continue, restoring your workspace (see `--store-workspace`) | `none` |
+| `--debugging-session` | Pass existing Task ID, create a copy of the experiment on a remote machine, and launch Jupyter/SSH for interactive access. Example `--debugging-session <task_id>`| `none`|
 | `--disable-session-cleanup` | If `True`, previous interactive sessions are not deleted | `false`|
-| `--requirements`| Specify `requirements.txt` file to install when setting the interactive session. | `none` or previously used requirements (can be overridden by calling `--packages`)|
-| `--packages`| Additional packages to add. Supports version numbers. Example: `--packages torch==1.7 tqdm` | Previously added packages.| 
-| `--upload-files`|  Specify local files/folders to upload to the remote session|`None`|
-| `--git-credentials` | If `True`, local `.git-credentials` file is sent to the interactive session.| `false`|
 | `--docker`| Select the docker image to use in the interactive session on |`nvidia/cuda:11.6.2-runtime-ubuntu20.04` or previously used docker image|
 | `--docker-args` | Add additional arguments for the docker image to use in the interactive session | `none` or the previously used docker-args |
-| `--debugging-session` | Pass existing Task ID, create a copy of the experiment on a remote machine, and launch Jupyter/SSH for interactive access. Example `--debugging-session <task_id>`| `none`|
+| `--force_dropbear`| Force using `dropbear` instead of SSHd |`None`| 
-| `--queue`| Select the queue to launch the interactive session on | Previously used queue|
+| `--git-credentials` | If `True`, local `.git-credentials` file is sent to the interactive session.| `false`|
-| `--shell` | Open the SSH session directly. Notice, quiting the SSH session will not shut down the remote session|`None`|
-| `--jupyter-lab` | Install a JupyterLab on interactive session | `true` |
-| `--vscode-server` | Install VSCode on interactive session | `true` |
-| `--vscode-version` | Set VSCode server (code-server) version, as well as VSCode python extension version <vscode:python-ext> (example: "3.7.4:2020.10.332292344")| `4.14.1:2023.12.0`|
-| `--vscode-extensions` |Install additional VSCode extensions and VSCode python extensions (example: `ms-python.python,ms-python.black-formatter,ms-python.pylint,ms-python.flake8`)|`None`|
-| `--public-ip` | If `true`, register the public IP of the remote machine (if you are running the session on a public cloud) | `false` - Session runs on the machine whose agent is executing the session|
-| `--remote-ssh-port`| Set the remote SSH server port, running on the agent's machine | 10022|
 | `--init-script` | Specify a BASH init script file to be executed when the interactive session is being set up | `none` or previously entered BASH script |
-| `--user-folder` | Specify the path for the session's remote base folder for the session | Home folder(`~/`) or previously entered user folder path |
+| `--jupyter-lab` | Install a JupyterLab on interactive session | `true` |
-| `--config-file` | Specify a path to another configuration file for `clearml-session` to store its previous state | `.clearml_session.json` or previously entered configuration file |  
-| `--remote-gateway` | Specify a gateway IP to pass to the interactive session, if an external address needs to be accessed | `none`|
-| `--base-task-id` | Pass the ID of a task that will become the base task, so the session will use its configurations | `none` or the previously entered base task |
 | `--keepalive` | If set, enables transparent proxy that keep sockets alive to maintain the connection to the remote resource | `false` - do not use transparent socket for mitigating connection drops | 
+| `--packages`| Additional packages to add. Supports version numbers. Example: `--packages torch==1.7 tqdm` | Previously added packages.| 
+| `--password`| Set your own SSH password for the interactive session | A randomly generated password or a previously used one |
+| `--project`| Set the project name to the interactive session task| `DevOps` |
+| `--public-ip` | If `true`, register the public IP of the remote machine (if you are running the session on a public cloud) | `false` - Session runs on the machine whose agent is executing the session|
+| `--queue`| Select the queue to launch the interactive session on | Previously used queue|
 | `--queue-excluded-tag` | The queue option list will exclude queues with specified tags. See the `tags` parameter in the [queues.create](../references/api/queues.md#post-queuescreate) API call | `none` |
 | `--queue-include-tag` | The queue option list will include only queues with specified tags. See the `tags` parameter in the [queues.create](../references/api/queues.md#post-queuescreate) API call | `none` |
-| `--skip-docker-network` | Don't pass the `--network host` flag to the Docker that is launching the remote session. See [Networking using the host network](https://docs.docker.com/network/network-tutorial-host/) | `false`|
+| `--remote-gateway` | Specify a gateway IP to pass to the interactive session, if an external address needs to be accessed | `none`|
-| `--username`| Set your own SSH username for the interactive session | `root` or a previously used username | 
+| `--remote-ssh-port`| Set the remote SSH server port, running on the agent's machine | 10022|
-| `--password`| Set your own SSH password for the interactive session | A randomly generated password or a previously used one |
+| `--requirements`| Specify `requirements.txt` file to install when setting the interactive session. | `none` or previously used requirements (can be overridden by calling `--packages`)|
-| `--force_dropbear`| Force using `dropbear` instead of SSHd |`None`| 
-| `--store-workspace` |  Upload/Restore remote workspace folder and extract it into next the session. Use with `--continue-session` to continue your previous work from your exact container state | `none` |
-| `--continue-session` | Pass the session of a previous session to continue, restoring your workspace (see `--store-workspace`) | `none` |
 | `--session-name` | Set the name of the interactive session Task| `none` |
 | `--session-tags` | Add tags to the interactive session for increased visibility | `none` |
-| `--version`| Display the clearml-session utility version| N/A|
+| `--shell` | Open the SSH session directly. Notice, quiting the SSH session will not shut down the remote session|`None`|
+| `--shutdowm`, `-S`| Shut down an active session | Previous session|
+| `--skip-docker-network` | Don't pass the `--network host` flag to the Docker that is launching the remote session. See [Networking using the host network](https://docs.docker.com/network/network-tutorial-host/) | `false`|
+| `--store-workspace` |  Upload/Restore remote workspace folder and extract it into next the session. Use with `--continue-session` to continue your previous work from your exact container state | `none` |
+| `--upload-files`|  Specify local files/folders to upload to the remote session|`None`|
+| `--user-folder` | Specify the path for the session's remote base folder for the session | Home folder(`~/`) or previously entered user folder path |
+| `--username`| Set your own SSH username for the interactive session | `root` or a previously used username | 
 | `--verbose` | Increase verbosity of logging | `none` |
+| `--version`| Display the clearml-session utility version| N/A|
+| `--vscode-extensions` |Install additional VSCode extensions and VSCode python extensions (example: `ms-python.python,ms-python.black-formatter,ms-python.pylint,ms-python.flake8`)|`None`|
+| `--vscode-server` | Install VSCode on interactive session | `true` |
+| `--vscode-version` | Set VSCode server (code-server) version, as well as VSCode python extension version <vscode:python-ext> (example: "3.7.4:2020.10.332292344")| `4.14.1:2023.12.0`|
 | `--yes`, `-y`| Automatic yes to prompts; assume "yes" as answer to all prompts and run non-interactively |N/A|
 
 </div>