clearml-docs/docs/guides/distributed/subprocess_example.md

39 lines
1.7 KiB
Markdown
Raw Normal View History

2021-05-13 23:48:51 +00:00
---
title: Subprocess
---
The [subprocess_example.py](https://github.com/allegroai/clearml/blob/master/examples/distributed/subprocess_example.py)
script demonstrates multiple subprocesses interacting and reporting to a main Task. The following happens in the script:
* This script initializes a main Task and spawns subprocesses, each for an instances of that Task.
2022-01-19 12:26:14 +00:00
* Each Task in a subprocess references the main Task by calling [Task.current_task](../../references/sdk/task.md#taskcurrent_task),
2021-05-13 23:48:51 +00:00
which always returns the main Task.
* The Task in each subprocess reports the following to the main Task:
* Hyperparameters - Additional, different hyperparameters.
2021-05-18 22:31:01 +00:00
* Console - Text logged to the console as the Task in each subprocess executes.
2021-05-13 23:48:51 +00:00
* When the script runs, it creates an experiment named `Popen example` which is associated with the `examples` project.
## Hyperparameters
2021-12-14 13:12:30 +00:00
ClearML automatically logs the command line options defined with `argparse`. A parameter dictionary is logged by
2022-01-19 12:26:14 +00:00
connecting it to the Task using a call to the [Task.connect](../../references/sdk/task.md#connect) method.
2021-05-13 23:48:51 +00:00
2021-12-14 13:12:30 +00:00
```python
additional_parameters = {
'stuff_' + str(randint(0, 100)): 'some stuff ' + str(randint(0, 100))
}
Task.current_task().connect(additional_parameters)
```
2021-05-13 23:48:51 +00:00
2022-05-26 06:54:41 +00:00
Command line options appear in **CONFIGURATION** **>** **HYPER PARAMETERS** **>** **Args**.
2021-05-13 23:48:51 +00:00
![image](../../img/examples_subprocess_example_01.png)
Parameter dictionaries appear in **General**.
![image](../../img/examples_subprocess_example_01a.png)
2021-05-18 22:31:01 +00:00
## Console
2021-05-13 23:48:51 +00:00
2022-05-22 07:27:30 +00:00
Output to the console, including the text messages from the Task in each subprocess, appear in **CONSOLE**.
2021-05-13 23:48:51 +00:00
![image](../../img/examples_subprocess_example_02.png)