ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add Hyper-Datasets

Browse Source
This commit is contained in:
allegroai
2021-06-21 01:00:16 +03:00
parent 31b3b52cac
commit 81bcabcb10
76 changed files with 3319 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
docs/guides/ide/google_colab.md Normal file
View File

@@ -0,0 +1,60 @@
---
title: ClearML Agent on Google Colab
---
[Google Colab](https://colab.research.google.com) is a common development environment for data scientists. It offers a convenient IDE as well as
compute provided by google.<br/>
Users can transform a Google Colab instance into an available resource in ClearML using [Clearml Agent](../../clearml_agent.md).
In this tutorial, we will go over how to create a ClearML worker node in a Google Colab notebook. Once the worker is up
and running, users can send Tasks to be executed on the Google Colab's HW.
## Prerequisites
* Be signed up for ClearML (Or have a server deployed).
* Have a Google account to access Google Colab
## Steps
1. Open up [this Google Colab notebook](https://colab.research.google.com/github/pollfly/clearml/blob/master/examples/clearml_agent/clearml_colab_agent.ipynb).
1. Run the first cell, which installs all the necessary packages:
```
!pip install git+https://github.com/allegroai/clearml
!pip install clearml-agent
```
1. Run the second cell, which exports this environment variable:
```
! export MPLBACKEND=TkAg
```
This environment variable makes Matplotlib work in headless mode, so it won't output graphs to the screen.
1. Create new credentials.
Go to your **profile** in the [ClearML WebApp](https://app.community.clear.ml). Under the **WORKSPACES** section,
go to **App Credentials**, click **+ Create new credentials**, and copy the information that pops up.
1. Set the credentials.
In the third cell, enter your own credentials:
```python
from clearml import Task
Task.set_credentials(api_host="https://api.community.clear.ml",
web_host="https://app.community.clear.ml",
files_host="https://files.community.clear.ml",
key='6ZHX9UQMYL874A1NE8',
secret='=2h6#%@Y&m*tC!VLEXq&JI7QhZPKuJfbaYD4!uUk(t7=9ENv'
)
```
1. In the fourth cell, launch a `clearml-agent` that will listen to the `default` queue:
```
!clearml-agent daemon --queue default
```
For additional options for running `clearml-agent`, see the [clearml-agent reference](../../references/clearml_agent_ref.md).
After cell 4 is executed, the worker should now appear in the [**Workers & Queues**](../../webapp/webapp_workers_queues.md)
page of your server. Clone experiments and enqueue them to your hearts content! The `clearml-agent` will fetch
experiments and execute them using the Google Colab hardware.

18
docs/guides/ide/integration_pycharm.md
View File

@@ -2,23 +2,21 @@
title: Integration for PyCharm
---
The **ClearML PyCharm plugin** enables syncing a local execution configuration to a remote execution machine:
The **ClearML PyCharm plugin** enables syncing a local execution configuration to a remote executor machine:
* Sync local repository information to a remote debug machine.
* Multiple users can use the same resource for execution without compromising private credentials.
* Run the [ClearML Agent](../../fundamentals/agents_and_queues.md) on default VMs/Containers.
* Run the [ClearML Agent](../../clearml_agent.md) on default VMs/Containers.
## Installation
**To install the ClearML PyCharm plugin, do the following:**
1. Download the latest plugin version from the [Releases page](https://github.com/allegroai/clearml-pycharm-plugin/releases).
1. Install the plugin in PyCharm from local disk:
![image](../../img/examples_ide_pycharm.png)
![image](../../img/ide_pycharm_plugin_from_disk.png)
## Optional: ClearML configuration parameters
@@ -29,17 +27,13 @@ the settings in the ClearML configuration file.
**To set ClearML configuration parameters:**
1. In PyCharm, open **Settings**.
1. In PyCharm, open **Settings** **>** **Tools** **>** **ClearML**.
1. Click **Tools**.
1. Click **ClearML**.
1. Configure ClearML server information:
1. Configure your ClearML server information:
1. API server (for example: ``http://localhost:8008``)
1. Web server (for example: ``http://localhost:8080``)
1. File server (for example: ``http://localhost:8081``)
1. Add **ClearML** user credentials key/secret.
![image](../../img/clearml_pycharm_plugin/pycharm_config_params.png)
![image](../../img/ide_pycharm_config_params.png)