clearml-docs/docs/clearml_agent/clearml_agent_setup.md

163 lines
6.3 KiB
Markdown
Raw Normal View History

2024-07-15 12:53:41 +00:00
---
title: Setup
---
## Installation
:::note
If ClearML was previously configured, follow [this](#adding-clearml-agent-to-a-configuration-file) to add
ClearML Agent specific configurations
:::
To install ClearML Agent, execute
```bash
pip install clearml-agent
```
:::info
Install ClearML Agent as a system Python package and not in a Python virtual environment.
An agent that runs in Virtual Environment Mode or Conda Environment Mode needs to create virtual environments, and
it can't do that when running from a virtual environment.
:::
## Configuration
1. In a terminal session, execute
```bash
clearml-agent init
```
The setup wizard prompts for ClearML credentials (see [here](../webapp/webapp_profile.md#clearml-credentials) about obtaining credentials).
```
Please create new clearml credentials through the settings page in your `clearml-server` web app,
or create a free account at https://app.clear.ml/settings/webapp-configuration
In the settings > workspace page, press "Create new credentials", then press "Copy to clipboard".
Paste copied configuration here:
```
If the setup wizard's response indicates that a configuration file already exists, follow the instructions [here](#adding-clearml-agent-to-a-configuration-file).
The wizard does not edit or overwrite existing configuration files.
1. At the command prompt `Paste copied configuration here:`, copy and paste the ClearML credentials and press **Enter**.
The setup wizard confirms the credentials.
```
Detected credentials key="********************" secret="*******"
```
1. **Enter** to accept the default server URL, which is detected from the credentials or enter a ClearML web server URL.
A secure protocol, https, must be used. **Do not use http.**
```
WEB Host configured to: [https://app.clear.ml]
```
:::note
If you are using a self-hosted ClearML Server, the default URL will use your domain.
:::
1. Do as above for API, URL, and file servers.
1. The wizard responds with your configuration:
```
CLEARML Hosts configuration:
Web App: https://app.clear.ml
API: https://api.clear.ml
File Store: https://files.clear.ml
Verifying credentials ...
Credentials verified!
```
1. Enter your Git username and password. Leave blank for SSH key authentication or when only using public repositories.
This is needed for cloning repositories by the agent.
```
Enter git username for repository cloning (leave blank for SSH key authentication): []
Enter password for user '<username>':
```
The setup wizard confirms your git credentials.
```
Git repository cloning will be using user=<username> password=<password>
```
1. Enter an additional artifact repository, or press **Enter** if not required.
This is needed for installing Python packages not found in pypi.
```
Enter additional artifact repository (extra-index-url) to use when installing python packages (leave blank if not required):
```
The setup wizard completes.
```
New configuration stored in /home/<username>/clearml.conf
CLEARML-AGENT setup completed successfully.
```
The configuration file location depends upon the operating system:
* Linux - `~/clearml.conf`
* Mac - `$HOME/clearml.conf`
* Windows - `\User\<username>\clearml.conf`
1. Optionally, configure ClearML options for **ClearML Agent** (default docker, package manager, etc.). See the [ClearML Configuration Reference](../configs/clearml_conf.md)
and the [ClearML Agent Environment Variables reference](../clearml_agent/clearml_agent_env_var.md).
:::note
The ClearML Enterprise server provides a [configuration vault](../webapp/webapp_profile.md#configuration-vault), the contents
of which are categorically applied on top of the agent-local configuration
:::
### Adding ClearML Agent to a Configuration File
In case a `clearml.conf` file already exists, add a few ClearML Agent specific configurations to it.<br/>
**Adding ClearML Agent to a ClearML configuration file:**
1. Open the ClearML configuration file for editing. Depending upon the operating system, it is:
* Linux - `~/clearml.conf`
* Mac - `$HOME/clearml.conf`
* Windows - `\User\<username>\clearml.conf`
1. After the `api` section, add your `agent` section. For example:
```
agent {
# Set GIT user/pass credentials (if user/pass are set, GIT protocol will be set to https)
git_user=""
git_pass=""
# all other domains will use public access (no user/pass). Default: always send user/pass for any VCS domain
git_host=""
# Force GIT protocol to use SSH regardless of the git url (Assumes GIT user/pass are blank)
force_git_ssh_protocol: false
# unique name of this worker, if None, created based on hostname:process_id
# Overridden with os environment: CLEARML_WORKER_NAME
worker_id: ""
}
```
View a complete ClearML Agent configuration file sample including an `agent` section [here](https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf).
1. Save the configuration.
### Dynamic Environment Variables
Dynamic ClearML Agent environment variables can be used to override any configuration setting that appears in the [`agent`](../configs/clearml_conf.md#agent-section)
section of the `clearml.conf`.
The environment variable's name should be `CLEARML_AGENT__AGENT__<configuration-path>`, where `<configuration-path>`
represents the full path to the configuration field being set. Elements of the configuration path should be separated by
`__` (double underscore). For example, set the `CLEARML_AGENT__AGENT__DEFAULT_DOCKER__IMAGE` environment variable to
deploy an agent with a different value to what is specified for `agent.default_docker.image` in the clearml.conf.
:::note NOTES
* Since configuration fields may contain JSON-parsable values, make sure to always quote strings (otherwise the agent
might fail to parse them)
* To comply with environment variables standards, it is recommended to use only upper-case characters in
environment variable keys. For this reason, ClearML Agent will always convert the configuration path specified in the
dynamic environment variable's key to lower-case before overriding configuration values with the environment variable
value.
:::