3.1 KiB
title |
---|
Storing Task Data Offline |
If your computer is offline, or you do not want a Task's data and logs stored in the ClearML Server, use the Offline Mode option. In this mode, all the data and logs that the Task captures from the code are stored in a local folder, which can be later uploaded to the ClearML Server.
Setting Task to Offline Mode
Before initializing a Task, use the Task.set_offline class method and set the
offline_mode
argument to True
.
from clearml import Task
# Use the set_offline class method before initializing a Task
Task.set_offline(offline_mode=True)
# Initialize a Task
task = Task.init(project_name="examples", task_name="my_task")
# Rest of code is executed. All data is logged locally and not onto the server
The method returns the Task ID and a path to the session folder:
ClearML Task: created new task id=offline-372657bb04444c25a31bc6af86552cc9
...
...
ClearML Task: Offline session stored in /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip
All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped. The
session's zip folder's location is ~/.clearml/cache/offline/<task_id>.zip
.
Uploading Session Data
Upload the session's execution data that the Task captured offline to the ClearML Server using one of the following:
-
clearml-task
CLIclearml-task --import-offline-session "/home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip"
Pass the path to the zip folder containing the session with the
--import-offline-session
parameter. -
Task.import_offline_session method.
from clearml import Task Task.import_offline_session( session_folder_zip="/home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip" )
In the
session_folder_zip
argument, insert the path to the zip folder containing the session.You can also use the offline task to update the execution of an existing previously executed task by providing the previously executed task’s ID. To avoid overwriting metrics, you can specify the initial iteration offset with
iteratiion_offset
.Task.import_offline_session( session_folder_zip="path/to/session/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip", previous_task_id="12345679", iteration_offset=1500 )
Both options will upload the Task's full execution details and outputs and return a link to the Task's results page on the ClearML Server:
ClearML: Importing offline session from /home/user/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip
ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72ad233/experiments/bb8b0f6fa0f94536a0d27fb55f02d3a5/output/log
The session details can be viewed in the ClearML WebApp, in the "my_task" experiment of the "examples" project, as specified when initializing the Task.