clearml-docs/docs/clearml_agent/clearml_agent_setup.md

title
Setup

Installation

:::note If ClearML was previously configured, follow this to add ClearML Agent specific configurations :::

To install ClearML Agent, execute

pip install clearml-agent

:::info Install ClearML Agent as a system Python package and not in a Python virtual environment. An agent that runs in Virtual Environment Mode or Conda Environment Mode needs to create virtual environments, and it can't do that when running from a virtual environment. :::

Configuration

1. In a terminal session, execute

   clearml-agent init
   

   The setup wizard prompts for ClearML credentials (see here about obtaining credentials).

   Please create new clearml credentials through the settings page in your `clearml-server` web app, 
   or create a free account at https://app.clear.ml/settings/webapp-configuration
   
   In the settings > workspace page, press "Create new credentials", then press "Copy to clipboard".
   
   Paste copied configuration here:    
   

   If the setup wizard's response indicates that a configuration file already exists, follow the instructions here. The wizard does not edit or overwrite existing configuration files.

2. At the command prompt Paste copied configuration here:, copy and paste the ClearML credentials and press Enter. The setup wizard confirms the credentials.

   Detected credentials key="********************" secret="*******"
   

3. Enter to accept the default server URL, which is detected from the credentials or enter a ClearML web server URL.

   A secure protocol, https, must be used. Do not use http.

   WEB Host configured to: [https://app.clear.ml] 
   

   :::note If you are using a self-hosted ClearML Server, the default URL will use your domain.
   :::

4. Do as above for API, URL, and file servers.

5. The wizard responds with your configuration:

   CLEARML Hosts configuration:
   Web App: https://app.clear.ml
   API: https://api.clear.ml
   File Store: https://files.clear.ml
   
   Verifying credentials ...
   Credentials verified!
   

6. Input the default output URI. Model checkpoints (snapshots) and task artifacts will be stored in this output location.

   Default Output URI (used to automatically store models and artifacts): (N)one/ClearML (S)erver/(C)ustom [None]
   

7. Enter your Git username and password. Leave blank for SSH key authentication or when only using public repositories.

   This is needed for cloning repositories by the agent.

   Enter git username for repository cloning (leave blank for SSH key authentication): []
   Enter password for user '<username>':
   

   The setup wizard confirms your git credentials.

   Git repository cloning will be using user=<username> password=<password>        
   

8. Enter an additional artifact repository, or press Enter if not required.

   This is needed for installing Python packages not found in pypi.

   Enter additional artifact repository (extra-index-url) to use when installing python packages (leave blank if not required):
   

   The setup wizard completes.

   New configuration stored in /home/<username>/clearml.conf
   CLEARML-AGENT setup completed successfully.
   

   The configuration file location depends upon the operating system:

   • Linux - ~/clearml.conf
   • Mac - $HOME/clearml.conf
   • Windows - \User\<username>\clearml.conf

9. Optionally, configure ClearML options for ClearML Agent (default docker, package manager, etc.). See the ClearML Configuration Reference and the ClearML Agent Environment Variables reference.

:::note The ClearML Enterprise server provides a configuration vault, the contents of which are categorically applied on top of the agent-local configuration :::

Adding ClearML Agent to a Configuration File

In case a clearml.conf file already exists, add a few ClearML Agent specific configurations to it.

Adding ClearML Agent to a ClearML configuration file:

1. Open the ClearML configuration file for editing. Depending upon the operating system, it is:

   • Linux - ~/clearml.conf
   • Mac - $HOME/clearml.conf
   • Windows - \User\<username>\clearml.conf

2. After the api section, add your agent section. For example:

   agent {
       # Set GIT user/pass credentials (if user/pass are set, GIT protocol will be set to https)
       git_user=""
       git_pass=""
       # all other domains will use public access (no user/pass). Default: always send user/pass for any VCS domain
       git_host=""
   
       # Force GIT protocol to use SSH regardless of the git url (Assumes GIT user/pass are blank)
       force_git_ssh_protocol: false
   
       # unique name of this worker, if None, created based on hostname:process_id
       # Overridden with os environment: CLEARML_WORKER_NAME
       worker_id: ""
   }   
   

   View a complete ClearML Agent configuration file sample including an agent section here.

3. Save the configuration.

Dynamic Environment Variables

Dynamic ClearML Agent environment variables can be used to override any configuration setting that appears in the agent section of the clearml.conf.

The environment variable's name should be CLEARML_AGENT__AGENT__<configuration-path>, where <configuration-path> represents the full path to the configuration field being set. Elements of the configuration path should be separated by __ (double underscore). For example, set the CLEARML_AGENT__AGENT__DEFAULT_DOCKER__IMAGE environment variable to deploy an agent with a different value to what is specified for agent.default_docker.image in the clearml.conf.

:::note NOTES

• Since configuration fields may contain JSON-parsable values, make sure to always quote strings (otherwise the agent might fail to parse them)
• To comply with environment variables standards, it is recommended to use only upper-case characters in environment variable keys. For this reason, ClearML Agent will always convert the configuration path specified in the dynamic environment variable's key to lower-case before overriding configuration values with the environment variable value. :::