ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-02-07 05:20:07 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Add reproducing execution env guide (#694)

Browse Source
This commit is contained in:
pollfly 2023-10-18 12:18:45 +03:00 committed by GitHub
parent 328d3729ec
commit 8a8bdc15c2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 42 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
docs/guides/clearml_agent/reproduce_exp.md Normal file
View File

@ -0,0 +1,41 @@
---
title: Recreating Experiment Environments
---
<div class="vid" >
<iframe style={{position: 'absolute', top: '0', left: '0', bottom: '0', right: '0', width: '100%', height: '100%'}}
src="https://www.youtube.com/embed/WTVrchczD34?si=2mZoMi4QdGl4MnUe"
title="YouTube video player"
frameborder="0"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; fullscreen"
allowfullscreen>
</iframe>
</div>
<br/>
Sometimes, you may need to recreate your experiment environment on a different machine, but you haven't committed your
code.
ClearML logs everything needed to reproduce your experiment and its environment (uncommitted changes, used packages, and
more), making it easy to reproduce your experiment's execution environment using ClearML.
You can reproduce the execution environment of any experiment you’ve run with ClearML on any workload:
1. Go to the experiment page of the task you want to reproduce in the [ClearML WebApp](../../webapp/webapp_overview.md),
:::tip
Use the UI's [filtering and sorting](../../webapp/webapp_exp_table.md#filtering-columns) to find the best performing experiments
:::
1. Copy the desired experiment's ID
1. Use the ClearML Agent's [`build`](../../clearml_agent/clearml_agent_ref.md#build) command to rebuild the experiment's
execution environment. Input the experiment's ID and the target local folder, where the environment will be created:
```commandline
clearml-agent build --id <task_id> --target <target_folder>
```
After running this command, the target folder will contain that task's original code with uncommitted changes applied,
as well as a complete recreated virtual environment
2. Activate the virtual environment using the activation script. Once done, you'll find all of your environment's packages
already installed in the environment
And that's it! Your experiment's environment and your code has been reproduced!

2
sidebars.js
View File

@ -143,7 +143,7 @@ module.exports = {
{'Advanced': ['guides/advanced/execute_remotely', 'guides/advanced/multiple_tasks_single_process']},
{'Automation': ['guides/automation/manual_random_param_search_example', 'guides/automation/task_piping']},
{'ClearML Task': ['guides/clearml-task/clearml_task_tutorial']},
{'ClearML Agent': ['guides/clearml_agent/executable_exp_containers', 'guides/clearml_agent/exp_environment_containers']},
{'ClearML Agent': ['guides/clearml_agent/executable_exp_containers', 'guides/clearml_agent/exp_environment_containers', 'guides/clearml_agent/reproduce_exp']},
{'Datasets': ['clearml_data/data_management_examples/data_man_cifar_classification', 'clearml_data/data_management_examples/data_man_python']},
{'Distributed': ['guides/distributed/distributed_pytorch_example', 'guides/distributed/subprocess_example']},
{'Docker': ['guides/docker/extra_docker_shell_script']},