clearml-docs/docs/guides/distributed/subprocess_example.md

39 lines
1.6 KiB
Markdown
Raw Normal View History

2021-05-13 23:48:51 +00:00
---
title: Subprocess
---
The [subprocess_example.py](https://github.com/allegroai/clearml/blob/master/examples/distributed/subprocess_example.py)
script demonstrates multiple subprocesses interacting and reporting to a main Task. The following happens in the script:
* This script initializes a main Task and spawns subprocesses, each for an instances of that Task.
2023-09-11 10:33:30 +00:00
* Each Task in a subprocess references the main Task by calling [`Task.current_task()`](../../references/sdk/task.md#taskcurrent_task),
2021-05-13 23:48:51 +00:00
which always returns the main Task.
* The Task in each subprocess reports the following to the main Task:
* Hyperparameters - Additional, different hyperparameters.
2021-05-18 22:31:01 +00:00
* Console - Text logged to the console as the Task in each subprocess executes.
2023-09-04 12:40:42 +00:00
* When the script runs, it creates an experiment named `Popen example` in the `examples` project.
2021-05-13 23:48:51 +00:00
## Hyperparameters
2021-12-14 13:12:30 +00:00
ClearML automatically logs the command line options defined with `argparse`. A parameter dictionary is logged by
2023-09-11 10:33:30 +00:00
connecting it to the Task using [`Task.connect()`](../../references/sdk/task.md#connect).
2021-05-13 23:48:51 +00:00
2021-12-14 13:12:30 +00:00
```python
additional_parameters = {
'stuff_' + str(randint(0, 100)): 'some stuff ' + str(randint(0, 100))
}
Task.current_task().connect(additional_parameters)
```
2021-05-13 23:48:51 +00:00
2023-01-12 10:49:55 +00:00
Command line options appear in **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **Args**.
2021-05-13 23:48:51 +00:00
![image](../../img/examples_subprocess_example_01.png)
Parameter dictionaries appear in **General**.
![image](../../img/examples_subprocess_example_01a.png)
2021-05-18 22:31:01 +00:00
## Console
2021-05-13 23:48:51 +00:00
2022-05-22 07:27:30 +00:00
Output to the console, including the text messages from the Task in each subprocess, appear in **CONSOLE**.
2021-05-13 23:48:51 +00:00
![image](../../img/examples_subprocess_example_02.png)