From 55fb19644ddd0355a6b06c0bc8d634704aa77df2 Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Thu, 18 May 2023 12:17:21 +0300 Subject: [PATCH] Add CLEARML_OFFLINE MODE info (#568) --- docs/clearml_sdk/task_sdk.md | 40 ++++++++++++++++++++++-------------- docs/faq.md | 23 ++++++++++++++++++--- docs/guides/set_offline.md | 34 +++++++++++++++--------------- 3 files changed, 63 insertions(+), 34 deletions(-) diff --git a/docs/clearml_sdk/task_sdk.md b/docs/clearml_sdk/task_sdk.md index 61a46543..ad088eda 100644 --- a/docs/clearml_sdk/task_sdk.md +++ b/docs/clearml_sdk/task_sdk.md @@ -404,34 +404,44 @@ Function tasks must be created from within a regular task, created by calling `T ### Offline Mode You can work with tasks in Offline Mode, in which all the data and logs that the Task captures are stored in a local -folder, which can later be uploaded to the [ClearML Server](../deploying_clearml/clearml_server.md). +session folder, which can later be uploaded to the [ClearML Server](../deploying_clearml/clearml_server.md). -Before initializing a Task, use the [`Task.set_offline`](../references/sdk/task.md#taskset_offline) class method and set -the `offline_mode` argument to `True`. The method returns the Task ID and a path to the session folder. +You can enable offline mode in one of the following ways: +* Before initializing a task, use the [`Task.set_offline`](../references/sdk/task.md#taskset_offline) class method and set +the `offline_mode` argument to `True` + + ```python + from clearml import Task + # Use the set_offline class method before initializing a Task + Task.set_offline(offline_mode=True) + # Initialize a Task + task = Task.init(project_name="examples", task_name="my_task") + ``` + +* Before running a task, set `CLEARML_OFFLINE_MODE=1` :::caution -Notice that the `Task.set_offline` method only works with tasks created using `Task.init` and not with those created +Offline mode only works with tasks created using `Task.init` and not with those created using the `Task.create` method. ::: -```python -from clearml import Task -# Use the set_offline class method before initializing a Task -Task.set_offline(offline_mode=True) -# Initialize a Task -task = Task.init(project_name="examples", task_name="my_task") - -# Rest of code is executed. All data is logged locally and not onto the server -``` - All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped. +The task's console output displays the task ID and a path to the folder with the captured information: + +```console +ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9 +... +... +ClearML Task: Offline session stored in /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip +``` + Upload the execution data that the Task captured offline to the ClearML Server using one of the following: * [`clearml-task`](../apps/clearml_task.md) CLI ```bash clearml-task --import-offline-session "path/to/session/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip" ``` - Pass the path to the zip folder containing the session with the `--import-offline-session` parameter + Pass the path to the zip folder containing the captured information with the `--import-offline-session` parameter * [`Task.import_offline_session`](../references/sdk/task.md#taskimport_offline_session) class method ```python diff --git a/docs/faq.md b/docs/faq.md index e2167a1d..aa14ccc9 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -503,9 +503,26 @@ See [`Task.init`](references/sdk/task.md#taskinit). Yes! You can use ClearML's Offline Mode, in which all the data and logs that a task captures from the code are stored in a local folder. -Before initializing a task, use the [`Task.set_offline`](references/sdk/task.md#taskset_offline) -class method and set the `offline_mode` argument to `True`. When executed, this returns the Task ID and a path to the -session folder. In order to upload to the ClearML Server the execution data that the Task captured offline, do one of the +You can enable offline mode in one of the following ways: +* Before initializing a task, use the [`Task.set_offline`](../references/sdk/task.md#taskset_offline) class method and set +the `offline_mode` argument to `True` +* Before running a task, set `CLEARML_OFFLINE_MODE=1` + +:::caution +Offline mode only works with tasks created using `Task.init` and not with those created +using the `Task.create` method. +::: + +The task's console output displays the task ID and a path to the folder with the session's captured information: + +```console +ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9 +... +... +ClearML Task: Offline session stored in /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip +``` + +In order to upload to the ClearML Server the execution data that the Task captured offline, do one of the following: * Use the `import-offline-session ` option of the [clearml-task](apps/clearml_task.md) CLI * Use the [`Task.import_offline_session`](references/sdk/task.md#taskimport_offline_session) method. diff --git a/docs/guides/set_offline.md b/docs/guides/set_offline.md index c7bdd565..33561c0c 100644 --- a/docs/guides/set_offline.md +++ b/docs/guides/set_offline.md @@ -4,30 +4,34 @@ title: Storing Task Data Offline If your computer is offline, or you do not want a Task's data and logs stored in the ClearML Server, use the **Offline Mode** option. In this mode, all the data and logs that the Task captures from the code are stored in a -local folder, which can be later uploaded to the [ClearML Server](../deploying_clearml/clearml_server.md). +local session folder, which can be later uploaded to the [ClearML Server](../deploying_clearml/clearml_server.md). ## Setting Task to Offline Mode -Before initializing a Task, use the [`Task.set_offline`](../references/sdk/task.md#taskset_offline) class method and set the -`offline_mode` argument to `True`. +You can enable offline mode in one of the following ways: +* Before initializing a task, use the [`Task.set_offline`](../references/sdk/task.md#taskset_offline) class method and set +the `offline_mode` argument to `True` + + ```python + from clearml import Task + # Use the set_offline class method before initializing a Task + Task.set_offline(offline_mode=True) + # Initialize a Task + task = Task.init(project_name="examples", task_name="my_task") + ``` + +* Before running a task, set `CLEARML_OFFLINE_MODE=1` :::caution -Notice that the `Task.set_offline` method only works with tasks created using `Task.init` and not with those created +Offline mode only works with tasks created using `Task.init` and not with those created using the `Task.create` method. ::: -```python -from clearml import Task -# Use the set_offline class method before initializing a Task -Task.set_offline(offline_mode=True) -# Initialize a Task -task = Task.init(project_name="examples", task_name="my_task") +All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped. The +session's zip folder's location is `~/.clearml/cache/offline/.zip`. -# Rest of code is executed. All data is logged locally and not onto the server -``` - -The method returns the Task ID and a path to the session folder: +The task's console output displays the task ID and a path to the folder with the session's captured information: ```console ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9 @@ -36,8 +40,6 @@ ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9 ClearML Task: Offline session stored in /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip ``` -All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped. The -session's zip folder's location is `~/.clearml/cache/offline/.zip`. ## Uploading Session Data