clearml-docs/docs/guides/ide/google_colab.md

61 lines
2.4 KiB
Markdown
Raw Normal View History

2021-06-20 22:00:16 +00:00
---
title: ClearML Agent on Google Colab
---
2023-02-16 10:17:53 +00:00
[Google Colab](https://colab.research.google.com) is a common development environment for data scientists. It supports a convenient IDE as well as
2023-08-15 10:48:06 +00:00
compute provided by Google.
2021-10-05 06:02:44 +00:00
2021-12-22 08:54:04 +00:00
Users can transform a Google Colab instance into an available resource in ClearML using [ClearML Agent](../../clearml_agent.md).
2021-06-20 22:00:16 +00:00
2022-05-19 06:59:10 +00:00
This tutorial goes over how to create a ClearML worker node in a Google Colab notebook. Once the worker is up
2023-08-15 10:48:06 +00:00
and running, users can send Tasks to be executed on Google Colab's hardware.
2021-06-20 22:00:16 +00:00
## Prerequisites
2023-04-16 09:32:48 +00:00
* Be signed up for ClearML (or have a server deployed).
2024-03-12 09:24:42 +00:00
* Have a Google account to access Google Colab.
2021-06-20 22:00:16 +00:00
## Steps
2022-03-16 11:08:47 +00:00
1. Open up [this Google Colab notebook](https://colab.research.google.com/github/allegroai/clearml/blob/master/examples/clearml_agent/clearml_colab_agent.ipynb).
2021-06-20 22:00:16 +00:00
1. Run the first cell, which installs all the necessary packages:
```
!pip install git+https://github.com/allegroai/clearml
!pip install clearml-agent
```
1. Run the second cell, which exports this environment variable:
```
! export MPLBACKEND=TkAg
```
This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.
2023-08-15 10:48:06 +00:00
1. Create new credentials. Go to your [**Settings**](https://app.clear.ml/settings/workspace-configuration) page > **WORKSPACE** section.
2022-01-23 07:50:24 +00:00
Under **App Credentials**, click **+ Create new credentials**, and copy the information that pops up.
2021-06-20 22:00:16 +00:00
2023-08-15 10:48:06 +00:00
1. Set the credentials. In the third cell, enter your own credentials:
2021-06-20 22:00:16 +00:00
```python
from clearml import Task
2022-01-23 07:50:24 +00:00
Task.set_credentials(
2022-03-13 13:07:06 +00:00
api_host="https://api.clear.ml",
web_host="https://app.clear.ml",
files_host="https://files.clear.ml",
2022-01-23 07:50:24 +00:00
key='6ZHX9UQMYL874A1NE8',
secret='=2h6#%@Y&m*tC!VLEXq&JI7QhZPKuJfbaYD4!uUk(t7=9ENv'
2021-06-20 22:00:16 +00:00
)
```
1. In the fourth cell, launch a `clearml-agent` that will listen to the `default` queue:
```
!clearml-agent daemon --queue default
```
2022-08-15 14:31:02 +00:00
For additional options for running `clearml-agent`, see the [clearml-agent reference](../../clearml_agent/clearml_agent_ref.md).
2021-06-20 22:00:16 +00:00
2023-08-09 10:28:25 +00:00
After executing cell 4, the worker appears in the [**Orchestration**](../../webapp/webapp_workers_queues.md)
2021-06-20 22:00:16 +00:00
page of your server. Clone experiments and enqueue them to your hearts content! The `clearml-agent` will fetch
experiments and execute them using the Google Colab hardware.