Add clearml-agent build tutorials (#319)

docs/clearml_agent.md

@@ -457,22 +457,27 @@ Build a Docker container that when launched executes a specific experiment, or a
   docker run <new-docker-name>
   ```
 
+Check out [this tutorial](guides/clearml_agent/executable_exp_containers.md) for building executable experiment 
+containers.
+
 ### Base Docker Container
 
-Build a Docker container according to the execution environment of a specific Task.
+Build a Docker container according to the execution environment of a specific task.
 
 ```bash
 clearml-agent build --id <task-id> --docker --target <new-docker-name>
 ```
 
-
-It's possible to add the Docker container as the base Docker image to a Task (experiment), using one of the following methods:
+It's possible to add the Docker container as the base Docker image to a task (experiment), using one of the following methods:
 
 - Using the **ClearML Web UI** - See [Base Docker image](webapp/webapp_exp_tuning.md#base-docker-image) on the "Tuning
   Experiments" page.
 - In the ClearML configuration file - Use the ClearML configuration file [agent.default_docker](configs/clearml_conf.md#agentdefault_docker)
   options.
 
+Check out [this tutorial](guides/clearml_agent/exp_environment_containers.md) for building a Docker container 
+replicating the execution environment of an existing task.
+
 ## Google Colab
 
 ClearML Agent can run on a [Google Colab](https://colab.research.google.com/) instance. This helps users to leverage 

docs/guides/clearml_agent/executable_exp_containers.md

@@ -0,0 +1,59 @@
+---
+title: Executable Experiment Containers
+---
+
+This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)’s [`build`](../../clearml_agent/clearml_agent_build.md) 
+command to package an experiment into an executable container. In this example, you will build a Docker image that, when 
+run, will automatically execute the [keras_tensorboard.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py)
+script.
+
+## Prerequisites
+* [`clearml-agent`](../../clearml_agent.md#installation) installed and configured
+* [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured
+* [clearml](https://github.com/allegroai/clearml) repo cloned (`git clone https://github.com/allegroai/clearml.git`)
+
+## Creating the ClearML Experiment
+1. Set up the experiment’s execution environment:
+   
+   ```console
+   cd clearml/examples/frameworks/keras
+   pip install -r requirements.txt
+   ```
+
+1. Run the experiment:
+   
+   ```console
+   python keras_tensorboard.py
+   ```
+   This creates a ClearML task called "Keras with TensorBoard example" in the "examples" project.
+
+   Note the task ID in the console output when running the script above:
+
+   ```console
+   ClearML Task: created new task id=<TASK_ID>
+   ```
+   This ID will be used in the following section.
+
+## Building and Launching a Containerized Task
+1. Execute the following command to build the container. Input the ID of the task created above.  
+   ```console
+   clearml-agent build --id <TASK_ID> --docker --target new-docker --entry-point clone_task
+   ```
+
+   :::tip
+   If the container will not make use of a GPU, add the `--cpu-only` flag
+   :::
+
+   This command will create a Docker container, set up with the execution environment for this experiment in the 
+   specified `--target` folder. When the Docker container is launched, it will clone the task specified with `id` and 
+   execute the clone (as designated by the `--entry-point` parameter).
+
+1. Run the Docker, pointing to the new container:
+
+   ```console
+   docker run new-docker
+   ```
+
+   The task will be executed inside the container. Task details can be viewed in the [ClearML Web UI](../../webapp/webapp_overview.md).
+
+For additional ClearML Agent options, see the [ClearML Agent reference page](../../clearml_agent/clearml_agent_ref.md).

docs/guides/clearml_agent/exp_environment_containers.md

@@ -0,0 +1,83 @@
+---
+title: Experiment Environment Containers
+---
+
+This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)’s [`build`](../../clearml_agent/clearml_agent_build.md) 
+command to build a Docker container replicating the execution environment of an existing task. ClearML Agents can make 
+use of such containers to execute tasks without having to set up their environment every time. 
+
+A use case for this would be manual hyperparameter optimization, where a base task can be used to create a container to 
+be used when running optimization tasks.
+
+## Prerequisites
+* [`clearml-agent`](../../clearml_agent.md#installation) installed and configured
+* [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured
+* [clearml](https://github.com/allegroai/clearml) repo cloned (`git clone https://github.com/allegroai/clearml.git`)
+  
+## Creating the ClearML Experiment
+1. Set up the experiment’s execution environment:
+   
+   ```console
+   cd clearml/examples/frameworks/keras
+   pip install -r requirements.txt
+   ```
+
+1. Run the experiment:
+   
+   ```console
+   python keras_tensorboard.py
+   ```
+   This creates a ClearML task called "Keras with TensorBoard example" in the "examples" project.
+
+   Note the task ID in the console output when running the script above:
+
+   ```console
+   ClearML Task: created new task id=<TASK_ID>
+   ```
+   This ID will be used in the following section.
+
+## Building the Docker Container
+
+Execute the following command to build the container. Input the ID of the task created above. 
+```console
+clearml-agent build --id <TASK_ID> --docker --target new_docker
+```
+
+:::tip
+If the container will not make use of a GPU, add the `--cpu-only` flag
+:::
+
+This will create a container with the specified task’s execution environment in the `--target` folder. 
+When the Docker build completes, the console output shows:
+
+```console
+Docker build done
+Committing docker container to: new_docker
+sha256:460453b93ct1989fd1c6637c236e544031c4d378581433fc0b961103ce206af1
+```
+
+## Using the New Docker Container
+Make use of the container you've just built by having a ClearML agent make use of it for executing a new experiment:	
+
+1. In the [ClearML Web UI](../../webapp/webapp_overview.md), go to the "examples" project, "Keras with TensorBoard 
+   example" task (the one executed [above](#creating-the-clearml-experiment)).
+1. [Clone](../../webapp/webapp_exp_reproducing.md) the experiment.
+1. In the cloned experiment, go to the **EXECUTION** tab **>** **CONTAINER** section. Under **IMAGE**, insert the name 
+   of the new Docker image, `new_docker`. See [Tuning Experiments](../../webapp/webapp_exp_tuning.md) for more task 
+   modification options. 
+1. Enqueue the cloned experiment to the `default` queue.
+1. Launch a `clearml-agent` in [Docker Mode](../../clearml_agent.md#docker-mode) and assign it to the `default` queue:
+   ```console
+   clearml-agent daemon --docker --queue default
+   ```
+
+   :::tip
+   If the agent will not make use of a GPU, add the `--cpu-only` flag
+   :::
+
+   This agent will pull the enqueued task and run it using the `new_docker` image to create the execution environment. 
+   In the task’s **CONSOLE** tab, one of the first logs should be:
+   
+   ```console
+   Executing: ['docker', 'run', ..., 'CLEARML_DOCKER_IMAGE=new_docker', ...].
+   ```

sidebars.js

@@ -106,6 +106,7 @@ module.exports = {
             {'Advanced': ['guides/advanced/execute_remotely', 'guides/advanced/multiple_tasks_single_process']},
             {'Automation': ['guides/automation/manual_random_param_search_example', 'guides/automation/task_piping']},
             {'ClearML Task': ['guides/clearml-task/clearml_task_tutorial']},
+            {'ClearML Agent': ['guides/clearml_agent/executable_exp_containers', 'guides/clearml_agent/exp_environment_containers']},
             {'Datasets': ['guides/datasets/data_man_cifar_classification', 'guides/datasets/data_man_python']},
             {'Distributed': ['guides/distributed/distributed_pytorch_example', 'guides/distributed/subprocess_example']},
             {'Docker': ['guides/docker/extra_docker_shell_script']},