clearml-docs/docs/guides/services/cleanup_service.md

66 lines
3.2 KiB
Markdown
Raw Normal View History

2021-05-13 23:48:51 +00:00
---
title: Cleanup Service
---
2021-12-23 12:02:45 +00:00
The [cleanup service](https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py)
demonstrates how to use the `clearml.backend_api.session.client.APIClient` class to implement a service that deletes old
archived tasks and their associated files: model checkpoints, other artifacts, and debug samples.
2023-10-01 07:31:48 +00:00
Modify the cleanup service's parameters to specify which archived experiments to delete and when to delete them.
2021-12-23 12:02:45 +00:00
### Running the Cleanup Service
:::info Self deployed ClearML server
A template `Cleanup Service` task is available in the `DevOps Services` project. You can clone it, adapt its [configuration](#configuration)
to your needs, and enqueue it for execution directly from the ClearML UI.
:::
Configure the task execution by modifying the `args` dictionary:
* `delete_threshold_days` - Tasks older than this number of days will be deleted. The default value is 30 days.
* `cleanup_period_in_days` - Repeat the cleanup service at this interval, in days. The default value is 1.0 (run once a day).
2023-01-25 11:25:29 +00:00
* `force_delete` - If `False` (default), delete only Draft tasks. If `True`, allows deletion of tasks in any status.
* `run_as_service` - If `True` (default), the task will be enqueued for remote execution (default queue: "services"). Otherwise, the script will execute locally.
2021-12-23 12:02:45 +00:00
:::note Remote Execution
If `run_as_service` is set to `True`, make sure a `clearml-agent` is assigned to the `services` queue.
:::
Now that the script is configured, execute it:
```bash
python cleanup_service.py
```
A new task called `Cleanup Service` is created in the `DevOps` project on your ClearML server. The script output should
look similar to:
```console
ClearML Task: created new task id=8126c0af800f4903be07421aa344d7b3
2022-03-13 13:07:06 +00:00
ClearML results page: https://app.clear.ml/projects/608e9039/experiments/81261aa34d7b3/output/log
2021-12-23 12:02:45 +00:00
Cleanup service started
Starting cleanup
Deleting 100 tasks
```
This is followed by details from the cleanup.
2021-05-13 23:48:51 +00:00
2021-09-09 10:17:46 +00:00
## The Cleanup Service Code
2021-05-13 23:48:51 +00:00
[cleanup_service.py](https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py) creates
2021-12-23 12:02:45 +00:00
an `APIClient` object that establishes a session with the ClearML Server, and accomplishes the cleanup by calling:
* [`Tasks.get_all`](../../references/api/tasks.md#post-tasksget_all) to get a list of Tasks to delete, providing the following parameters:
2021-05-13 23:48:51 +00:00
* `system_tags` - Get only Tasks tagged as `archived`.
2023-01-25 11:25:29 +00:00
* `status_changed` - Get Tasks whose last status change is older than the delete threshold (in seconds).
2021-12-23 12:02:45 +00:00
* [`Task.delete`](../../references/sdk/task.md#delete) - Delete a Task.
## Configuration
2023-10-01 07:31:48 +00:00
The experiment's hyperparameters are explicitly logged to ClearML using the [`Task.connect`](../../references/sdk/task.md#connect)
method. View them in the WebApp, in the experiment's **CONFIGURATION** page under **HYPERPARAMETERS > General**.
2021-12-23 12:02:45 +00:00
The task can be reused. Clone the task, edit its parameters, and enqueue the task to run in ClearML Agent [services mode](../../clearml_agent.md#services-mode).
![Cleanup service configuration](../../img/example_cleanup_configuration.png)
## Console
2023-10-01 07:31:48 +00:00
All console output appears in the experiment's **CONSOLE**.
2021-05-13 23:48:51 +00:00
2021-12-23 12:02:45 +00:00
![Cleanup service console](../../img/examples_cleanup_console.png)