ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-05-15 09:56:15 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add clearml-task import offline session option (#251)

Browse Source
This commit is contained in:
pollfly 2022-05-15 15:26:38 +03:00 committed by GitHub
parent 26db382df9
commit b5ee71c21c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 34 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/apps/clearml_task.md
View File

@ -75,6 +75,7 @@ errors in identifying the correct default branch.
| `--task-type` | Set the task type. Optional values: training, testing, inference, data_processing, application, monitor, controller, optimizer, service, qc, custom | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> | | `--task-type` | Set the task type. Optional values: training, testing, inference, data_processing, application, monitor, controller, optimizer, service, qc, custom | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--skip-task-init` | If set, `Task.init()` call is not added to the entry point, and is assumed to be called within the script | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> | | `--skip-task-init` | If set, `Task.init()` call is not added to the entry point, and is assumed to be called within the script | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--base-task-id` | Use a pre-existing task in the system, instead of a local repo / script. Essentially clones an existing task and overrides arguments / requirements | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> | | `--base-task-id` | Use a pre-existing task in the system, instead of a local repo / script. Essentially clones an existing task and overrides arguments / requirements | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--import-offline-session`| Specify the path to the offline session you want to import.| <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
</div> </div>

12
docs/faq.md
View File

@ -490,12 +490,16 @@ See [`Task.init`](references/sdk/task.md#taskinit).
**Can I run ClearML Task while working offline?** <a id="offline_mode"></a> **Can I run ClearML Task while working offline?** <a id="offline_mode"></a>
Yes! You can use ClearML's Offline Mode, in which all the data and logs that a task captures from the code are stored in a Yes! You can use ClearML's Offline Mode, in which all the data and logs that a task captures from the code are stored in a
local folder local folder.
Before initializing a task, use the [Task.set_offline](references/sdk/task.md#taskset_offline) Before initializing a task, use the [Task.set_offline](references/sdk/task.md#taskset_offline)
class method and set the `offline_mode` argument to `True`. When executed, this returns the Task ID and a path to the local class method and set the `offline_mode` argument to `True`. When executed, this returns the Task ID and a path to the
session folder. In order to upload to the ClearML Server the local execution data that the Task captured offline, use the session folder. In order to upload to the ClearML Server the execution data that the Task captured offline, do one of the
[Task.import_offline_session](references/sdk/task.md#taskimport_offline_session) method. See [Storing Task Data Offline](guides/set_offline.md). following:
* Use the `import-offline-session <session_path>` option of the [clearml-task](apps/clearml_task.md) CLI
* Use the [Task.import_offline_session](references/sdk/task.md#taskimport_offline_session) method.
See [Storing Task Data Offline](guides/set_offline.md).

37
docs/guides/set_offline.md
View File

@ -21,7 +21,7 @@ task = Task.init(project_name="examples", task_name="my_task")
# Rest of code is executed. All data is logged locally and not onto the server # Rest of code is executed. All data is logged locally and not onto the server
``` ```
The method returns the Task ID and a path to the local session folder: The method returns the Task ID and a path to the session folder:
```console ```console
ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9 ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9
@ -33,25 +33,38 @@ ClearML Task: Offline session stored in /home/user/.clearml/cache/offline/b78684
All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped. The All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped. The
session's zip folder's location is `~/.clearml/cache/offline/<task_id>.zip`. session's zip folder's location is `~/.clearml/cache/offline/<task_id>.zip`.
## Uploading Local Session ## Uploading Session Data
In order to upload to the ClearML Server the local execution data that the Task captured offline, use the Upload the session's execution data that the Task captured offline to the ClearML Server using one of the following:
[Task.import_offline_session](../references/sdk/task.md#taskimport_offline_session) method. This method will upload the
Task's full execution details and outputs.
```python
from clearml import Task
Task.import_offline_session(session_folder_zip="path/to/session/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip") * `clearml-task` CLI
```
```bash
clearml-task --import-offline-session "/home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip"
```
In the `session_folder_zip` argument, insert the path to the zip folder containing the session. Pass the path to the zip folder containing the session with the `--import-offline-session` parameter.
* [Task.import_offline_session](../references/sdk/task.md#taskimport_offline_session) method.
This method returns a link to the Task's results page on the ClearML Server: ```python
from clearml import Task
Task.import_offline_session(
session_folder_zip="/home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip"
)
```
In the `session_folder_zip` argument, insert the path to the zip folder containing the session.
Both options will upload the Task's full execution details and outputs and return a link to the Task's results page on
the ClearML Server:
```console ```console
ClearML: Importing offline session from /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip ClearML: Importing offline session from /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip
ClearML results page: https://app.community.clear.ml/projects/4043a1657f374e9298649c6ba72ad233/experiments/bb8b0f6fa0f94536a0d27fb55f02d3a5/output/log ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72ad233/experiments/bb8b0f6fa0f94536a0d27fb55f02d3a5/output/log
``` ```
The session details can be viewed in the ClearML WebApp, in the "my_task" experiment of the "examples" The session details can be viewed in the ClearML WebApp, in the "my_task" experiment of the "examples"