mirror of
https://github.com/clearml/clearml-docs
synced 2025-04-23 07:46:14 +00:00
Small fixes (#131)
This commit is contained in:
parent
6ae75beaa2
commit
ec304690b6
@ -37,7 +37,7 @@ A specific branch and commit ID, other than latest commit in master, to be execu
|
|||||||
`--branch <branch_name> --commit <commit_id>` flags.
|
`--branch <branch_name> --commit <commit_id>` flags.
|
||||||
If unspecified, `clearml-task` will use the latest commit from the master branch.
|
If unspecified, `clearml-task` will use the latest commit from the master branch.
|
||||||
|
|
||||||
### Command line options
|
### Command Line Options
|
||||||
|
|
||||||
<div className="tbl-cmd">
|
<div className="tbl-cmd">
|
||||||
|
|
||||||
|
@ -49,10 +49,10 @@ Task id=2f96ee95b05d4693b360d0fcbb26b727 sent for execution on queue default
|
|||||||
Execution log at: https://app.community.clear.ml/projects/552d5399112d47029c146d5248570295/experiments/2f96ee95b05d4693b360d0fcbb26b727/output/log
|
Execution log at: https://app.community.clear.ml/projects/552d5399112d47029c146d5248570295/experiments/2f96ee95b05d4693b360d0fcbb26b727/output/log
|
||||||
```
|
```
|
||||||
|
|
||||||
:::note
|
:::note Adding Requirements
|
||||||
**clearml-task** automatically finds the requirements.txt file in remote repositories.
|
`clearml-task` automatically finds the requirements.txt file in remote repositories.
|
||||||
If a remote repo does not have such a file, make sure to either add one with all the required Python packages,
|
If a remote repo does not have such a file, make sure to either add one with all the required Python packages,
|
||||||
or add the **`--packages '<package_name>`** flag to the command.
|
or add the `--packages "<package_name>"` flag to the command (for example: `--packages "tqdm>=2.1" "scikit-learn"`).
|
||||||
:::
|
:::
|
||||||
<br />
|
<br />
|
||||||
|
|
||||||
|
@ -83,7 +83,9 @@ dataset_project = "dataset_examples"
|
|||||||
|
|
||||||
from clearml import Dataset
|
from clearml import Dataset
|
||||||
|
|
||||||
dataset_path = Dataset.get(dataset_name=dataset_name, dataset_project=dataset_project).get_local_copy()
|
dataset_path = Dataset.get(
|
||||||
|
dataset_name=dataset_name,
|
||||||
|
dataset_project=dataset_project).get_local_copy()
|
||||||
|
|
||||||
trainset = datasets.CIFAR10(
|
trainset = datasets.CIFAR10(
|
||||||
root=dataset_path,
|
root=dataset_path,
|
||||||
|
@ -24,7 +24,9 @@ We first need to obtain a local copy of the CIFAR dataset.
|
|||||||
from clearml import StorageManager
|
from clearml import StorageManager
|
||||||
|
|
||||||
manager = StorageManager()
|
manager = StorageManager()
|
||||||
dataset_path = manager.get_local_copy(remote_url="https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz")
|
dataset_path = manager.get_local_copy(
|
||||||
|
remote_url="https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
This script downloads the data and `dataset_path` contains the path to the downloaded data.
|
This script downloads the data and `dataset_path` contains the path to the downloaded data.
|
||||||
|
@ -26,8 +26,12 @@ The example uploads a dictionary as an artifact in the main Task by calling the
|
|||||||
method on [`Task.current_task`](../../references/sdk/task.md#taskcurrent_task) (the main Task). The dictionary contains the [`dist.rank`](https://pytorch.org/docs/stable/distributed.html#torch.distributed.get_rank)
|
method on [`Task.current_task`](../../references/sdk/task.md#taskcurrent_task) (the main Task). The dictionary contains the [`dist.rank`](https://pytorch.org/docs/stable/distributed.html#torch.distributed.get_rank)
|
||||||
of the subprocess, making each unique.
|
of the subprocess, making each unique.
|
||||||
|
|
||||||
Task.current_task().upload_artifact(
|
```python
|
||||||
'temp {:02d}'.format(dist.get_rank()), artifact_object={'worker_rank': dist.get_rank()})
|
Task.current_task().upload_artifact(
|
||||||
|
'temp {:02d}'.format(dist.get_rank()),
|
||||||
|
artifact_object={'worker_rank': dist.get_rank()}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
All of these artifacts appear in the main Task under **ARTIFACTS** **>** **OTHER**.
|
All of these artifacts appear in the main Task under **ARTIFACTS** **>** **OTHER**.
|
||||||
|
|
||||||
@ -40,8 +44,14 @@ method on `Task.current_task().get_logger`, which is the logger for the main Tas
|
|||||||
with the same title (`loss`), but a different series name (containing the subprocess' `rank`), all loss scalar series are
|
with the same title (`loss`), but a different series name (containing the subprocess' `rank`), all loss scalar series are
|
||||||
logged together.
|
logged together.
|
||||||
|
|
||||||
Task.current_task().get_logger().report_scalar(
|
```python
|
||||||
'loss', 'worker {:02d}'.format(dist.get_rank()), value=loss.item(), iteration=i)
|
Task.current_task().get_logger().report_scalar(
|
||||||
|
'loss',
|
||||||
|
'worker {:02d}'.format(dist.get_rank()),
|
||||||
|
value=loss.item(),
|
||||||
|
iteration=i
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
The single scalar plot for loss appears in **RESULTS** **>** **SCALARS**.
|
The single scalar plot for loss appears in **RESULTS** **>** **SCALARS**.
|
||||||
|
|
||||||
@ -49,12 +59,14 @@ The single scalar plot for loss appears in **RESULTS** **>** **SCALARS**.
|
|||||||
|
|
||||||
## Hyperparameters
|
## Hyperparameters
|
||||||
|
|
||||||
**ClearML** automatically logs the argparse command line options. Since the [Task.connect](../../references/sdk/task#connect)
|
**ClearML** automatically logs the argparse command line options. Since the [`Task.connect`](../../references/sdk/task#connect)
|
||||||
method is called on `Task.current_task`, they are logged in the main Task. A different hyperparameter key is used in each
|
method is called on [`Task.current_task`](../../references/sdk/task.md#taskcurrent_task), they are logged in the main Task. A different hyperparameter key is used in each
|
||||||
subprocess, so they do not overwrite each other in the main Task.
|
subprocess, so they do not overwrite each other in the main Task.
|
||||||
|
|
||||||
param = {'worker_{}_stuff'.format(dist.get_rank()): 'some stuff ' + str(randint(0, 100))}
|
```python
|
||||||
Task.current_task().connect(param)
|
param = {'worker_{}_stuff'.format(dist.get_rank()): 'some stuff ' + str(randint(0, 100))}
|
||||||
|
Task.current_task().connect(param)
|
||||||
|
```
|
||||||
|
|
||||||
All the hyperparameters appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS**.
|
All the hyperparameters appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS**.
|
||||||
|
|
||||||
|
@ -14,11 +14,15 @@ which always returns the main Task.
|
|||||||
|
|
||||||
## Hyperparameters
|
## Hyperparameters
|
||||||
|
|
||||||
**ClearML** automatically logs the command line options defined with `argparse`. A parameter dictionary is logged by
|
ClearML automatically logs the command line options defined with `argparse`. A parameter dictionary is logged by
|
||||||
connecting it to the Task using a call to the [Task.connect](../../references/sdk/task#connect) method.
|
connecting it to the Task using a call to the [Task.connect](../../references/sdk/task#connect) method.
|
||||||
|
|
||||||
additional_parameters = {'stuff_' + str(randint(0, 100)): 'some stuff ' + str(randint(0, 100))}
|
```python
|
||||||
Task.current_task().connect(additional_parameters)
|
additional_parameters = {
|
||||||
|
'stuff_' + str(randint(0, 100)): 'some stuff ' + str(randint(0, 100))
|
||||||
|
}
|
||||||
|
Task.current_task().connect(additional_parameters)
|
||||||
|
```
|
||||||
|
|
||||||
Command line options appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **Args**.
|
Command line options appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **Args**.
|
||||||
|
|
||||||
|
@ -19,23 +19,20 @@ a shell script when a docker is started, but before an experiment is run.
|
|||||||
## Steps
|
## Steps
|
||||||
|
|
||||||
1. Open your ClearML configuration file for editing. Depending upon your operating system, it is:
|
1. Open your ClearML configuration file for editing. Depending upon your operating system, it is:
|
||||||
* Linux - ~/clearml.conf
|
* Linux - `~/clearml.conf`
|
||||||
* Mac - $HOME/clearml.conf
|
* Mac - `$HOME/clearml.conf`
|
||||||
* Windows - \\User\\<username\>\\clearml.conf
|
* Windows - `\User\<username>\clearml.conf`
|
||||||
|
|
||||||
When you open up the file, the first line should say: `# CLEARML-AGENT configuration file`
|
1. In the file, search for and go to, `extra_docker_shell_script:`, which is where we will be putting our extra script. If
|
||||||
|
it is commented out, make sure to uncomment the line. We will use the example script that is already there `["apt-get install -y bindfs", ]`.
|
||||||
|
|
||||||
1. In the file, search for and go to, "extra_docker_shell_script:", which is where we will be putting our extra script. If
|
|
||||||
it is commented out, make sure to uncomment the line. We will use the example script that is already there ["apt-get install -y bindfs", ].
|
|
||||||
|
|
||||||
1. Search for and go to "docker_force_pull" in the document, and make sure that it is set to "true", so that your docker image will
|
1. Search for and go to `docker_force_pull` in the document, and make sure that it is set to `true`, so that your docker
|
||||||
be updated.
|
image will be updated.
|
||||||
|
|
||||||
1. Run the `clearml-agent` in docker mode: `clearml-agent daemon --docker --queue default`. The agent will use the default
|
1. Run the `clearml-agent` in docker mode: `clearml-agent daemon --docker --queue default`. The agent will use the default
|
||||||
Cuda/Nvidia Docker Image.
|
Cuda/Nvidia Docker Image.
|
||||||
|
|
||||||
1. Enqueue any Clearml Task to the default queue, which the Agent is now listening to. The Agent pulls the Task, and then reproduces it,
|
1. Enqueue any ClearML Task to the `default` queue, which the Agent is now listening to. The Agent pulls the Task, and then reproduces it,
|
||||||
and now it will execute the `extra_docker_shell_script` that was put in the configuration file. Then the code will be
|
and now it will execute the `extra_docker_shell_script` that was put in the configuration file. Then the code will be
|
||||||
executed in the updated docker container. If we look at the console output in the web UI, the third entry should start
|
executed in the updated docker container. If we look at the console output in the web UI, the third entry should start
|
||||||
with `Executing: ['docker', 'run', '-t', '--gpus...'`, and towards the end of the entry, where the downloaded packages are
|
with `Executing: ['docker', 'run', '-t', '--gpus...'`, and towards the end of the entry, where the downloaded packages are
|
||||||
|
@ -18,9 +18,14 @@ Accuracy, learning rate, and training loss appear in **RESULTS** **>** **SCALARS
|
|||||||
is logged by connecting it to the Task using a call to the [Task.connect](../../../../../references/sdk/task.md#connect)
|
is logged by connecting it to the Task using a call to the [Task.connect](../../../../../references/sdk/task.md#connect)
|
||||||
method.
|
method.
|
||||||
|
|
||||||
configuration_dict = {'number_of_epochs': 6, 'batch_size': 16, 'ngrams': 2, 'base_lr': 1.0}
|
```python
|
||||||
configuration_dict = task.connect(configuration_dict) # enabling configuration override by clearml
|
configuration_dict = {
|
||||||
|
'number_of_epochs': 6, 'batch_size': 16, 'ngrams': 2, 'base_lr': 1.0
|
||||||
|
}
|
||||||
|
# enabling configuration override by clearml
|
||||||
|
configuration_dict = task.connect(configuration_dict)
|
||||||
|
```
|
||||||
|
|
||||||
Command line options appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **Args**.
|
Command line options appear in **CONFIGURATIONS** **>** **HYPER PARAMETERS** **>** **Args**.
|
||||||
|
|
||||||

|

|
||||||
|
@ -34,23 +34,14 @@ installed, it attempts to import `OptimizerBOHB`. If `clearml.automation.hpbands
|
|||||||
the `RandomSearch` for the search strategy.
|
the `RandomSearch` for the search strategy.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
aSearchStrategy = None
|
try:
|
||||||
|
from clearml.automation.optuna import OptimizerOptuna # noqa
|
||||||
if not aSearchStrategy:
|
aSearchStrategy = OptimizerOptuna
|
||||||
try:
|
except ImportError as ex:
|
||||||
from clearml.optuna import OptimizerOptuna
|
try:
|
||||||
aSearchStrategy = OptimizerOptuna
|
from clearml.automation.hpbandster import OptimizerBOHB # noqa
|
||||||
except ImportError as ex:
|
aSearchStrategy = OptimizerBOHB
|
||||||
pass
|
except ImportError as ex:
|
||||||
|
|
||||||
if not aSearchStrategy:
|
|
||||||
try:
|
|
||||||
from clearml.automation.hpbandster import OptimizerBOHB
|
|
||||||
aSearchStrategy = OptimizerBOHB
|
|
||||||
except ImportError as ex:
|
|
||||||
pass
|
|
||||||
|
|
||||||
if not aSearchStrategy:
|
|
||||||
logging.getLogger().warning(
|
logging.getLogger().warning(
|
||||||
'Apologies, it seems you do not have \'optuna\' or \'hpbandster\' installed, '
|
'Apologies, it seems you do not have \'optuna\' or \'hpbandster\' installed, '
|
||||||
'we will be using RandomSearch strategy instead')
|
'we will be using RandomSearch strategy instead')
|
||||||
@ -63,16 +54,16 @@ When the optimization starts, a callback is provided that returns the best perfo
|
|||||||
the `job_complete_callback` function returns the ID of `top_performance_job_id`.
|
the `job_complete_callback` function returns the ID of `top_performance_job_id`.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
def job_complete_callback(
|
def job_complete_callback(
|
||||||
job_id, # type: str
|
job_id, # type: str
|
||||||
objective_value, # type: float
|
objective_value, # type: float
|
||||||
objective_iteration, # type: int
|
objective_iteration, # type: int
|
||||||
job_parameters, # type: dict
|
job_parameters, # type: dict
|
||||||
top_performance_job_id # type: str
|
top_performance_job_id # type: str
|
||||||
):
|
):
|
||||||
print('Job completed!', job_id, objective_value, objective_iteration, job_parameters)
|
print('Job completed!', job_id, objective_value, objective_iteration, job_parameters)
|
||||||
if job_id == top_performance_job_id:
|
if job_id == top_performance_job_id:
|
||||||
print('WOOT WOOT we broke the record! Objective reached {}'.format(objective_value))
|
print('WOOT WOOT we broke the record! Objective reached {}'.format(objective_value))
|
||||||
```
|
```
|
||||||
|
|
||||||
## Initialize the Optimization Task
|
## Initialize the Optimization Task
|
||||||
@ -86,11 +77,13 @@ When the code runs, it creates an experiment named **Automatic Hyper-Parameter O
|
|||||||
the project **Hyper-Parameter Optimization**, which can be seen in the **ClearML Web UI**.
|
the project **Hyper-Parameter Optimization**, which can be seen in the **ClearML Web UI**.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# Connecting CLEARML
|
# Connecting CLEARML
|
||||||
task = Task.init(project_name='Hyper-Parameter Optimization',
|
task = Task.init(
|
||||||
task_name='Automatic Hyper-Parameter Optimization',
|
project_name='Hyper-Parameter Optimization',
|
||||||
task_type=Task.TaskTypes.optimizer,
|
task_name='Automatic Hyper-Parameter Optimization',
|
||||||
reuse_last_task_id=False)
|
task_type=Task.TaskTypes.optimizer,
|
||||||
|
reuse_last_task_id=False
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Set Up the Arguments
|
## Set Up the Arguments
|
||||||
@ -105,17 +98,17 @@ Since the arguments dictionary is connected to the Task, after the code runs onc
|
|||||||
to optimize a different experiment.
|
to optimize a different experiment.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# experiment template to optimize in the hyper-parameter optimization
|
# experiment template to optimize in the hyper-parameter optimization
|
||||||
args = {
|
args = {
|
||||||
'template_task_id': None,
|
'template_task_id': None,
|
||||||
'run_as_service': False,
|
'run_as_service': False,
|
||||||
}
|
}
|
||||||
args = task.connect(args)
|
args = task.connect(args)
|
||||||
|
|
||||||
# Get the template task experiment that we want to optimize
|
# Get the template task experiment that we want to optimize
|
||||||
if not args['template_task_id']:
|
if not args['template_task_id']:
|
||||||
args['template_task_id'] = Task.get_task(
|
args['template_task_id'] = Task.get_task(
|
||||||
project_name='examples', task_name='Keras HP optimization base').id
|
project_name='examples', task_name='Keras HP optimization base').id
|
||||||
```
|
```
|
||||||
|
|
||||||
## Creating the Optimizer Object
|
## Creating the Optimizer Object
|
||||||
@ -124,9 +117,9 @@ Initialize an [automation.HyperParameterOptimizer](../../../references/sdk/hpo_o
|
|||||||
object, setting the optimization parameters, beginning with the ID of the experiment to optimize.
|
object, setting the optimization parameters, beginning with the ID of the experiment to optimize.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
an_optimizer = HyperParameterOptimizer(
|
an_optimizer = HyperParameterOptimizer(
|
||||||
# This is the experiment we want to optimize
|
# This is the experiment we want to optimize
|
||||||
base_task_id=args['template_task_id'],
|
base_task_id=args['template_task_id'],
|
||||||
```
|
```
|
||||||
|
|
||||||
Set the hyperparameter ranges to sample, instantiating them as **ClearML** automation objects using [automation.UniformIntegerParameterRange](../../../references/sdk/hpo_parameters_uniformintegerparameterrange.md)
|
Set the hyperparameter ranges to sample, instantiating them as **ClearML** automation objects using [automation.UniformIntegerParameterRange](../../../references/sdk/hpo_parameters_uniformintegerparameterrange.md)
|
||||||
@ -190,24 +183,25 @@ The optimization can run as a service, if the `run_as_service` argument is set t
|
|||||||
running as a service, see [Services Mode](../../../clearml_agent.md#services-mode).
|
running as a service, see [Services Mode](../../../clearml_agent.md#services-mode).
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# if we are running as a service, just enqueue ourselves into the services queue and let it run the optimization
|
# if we are running as a service, just enqueue ourselves into the services queue and let it run the optimization
|
||||||
if args['run_as_service']:
|
if args['run_as_service']:
|
||||||
# if this code is executed by `clearml-agent` the function call does nothing.
|
# if this code is executed by `clearml-agent` the function call does nothing.
|
||||||
# if executed locally, the local process will be terminated, and a remote copy will be executed instead
|
# if executed locally, the local process will be terminated, and a remote copy will be executed instead
|
||||||
task.execute_remotely(queue_name='services', exit_process=True)
|
task.execute_remotely(queue_name='services', exit_process=True)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Optimize
|
## Optimize
|
||||||
|
|
||||||
The optimizer is ready. Set the report period and start it, providing the callback method to report the best performance.
|
The optimizer is ready. Set the report period and [start](../../../references/sdk/hpo_optimization_hyperparameteroptimizer.md#start)
|
||||||
|
it, providing the callback method to report the best performance.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# report every 12 seconds, this is way too often, but we are testing here J
|
# report every 12 seconds, this is way too often, but we are testing here J
|
||||||
an_optimizer.set_report_period(0.2)
|
an_optimizer.set_report_period(0.2)
|
||||||
# start the optimization process, callback function to be called every time an experiment is completed
|
# start the optimization process, callback function to be called every time an experiment is completed
|
||||||
# this function returns immediately
|
# this function returns immediately
|
||||||
an_optimizer.start(job_complete_callback=job_complete_callback)
|
an_optimizer.start(job_complete_callback=job_complete_callback)
|
||||||
# set the time limit for the optimization process (2 hours)
|
# set the time limit for the optimization process (2 hours)
|
||||||
```
|
```
|
||||||
|
|
||||||
Now that it is running:
|
Now that it is running:
|
||||||
@ -218,15 +212,15 @@ Now that it is running:
|
|||||||
1. Stop the optimizer.
|
1. Stop the optimizer.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# set the time limit for the optimization process (2 hours)
|
# set the time limit for the optimization process (2 hours)
|
||||||
an_optimizer.set_time_limit(in_minutes=90.0)
|
an_optimizer.set_time_limit(in_minutes=90.0)
|
||||||
# wait until process is done (notice we are controlling the optimization process in the background)
|
# wait until process is done (notice we are controlling the optimization process in the background)
|
||||||
an_optimizer.wait()
|
an_optimizer.wait()
|
||||||
# optimization is completed, print the top performing experiments id
|
# optimization is completed, print the top performing experiments id
|
||||||
top_exp = an_optimizer.get_top_experiments(top_k=3)
|
top_exp = an_optimizer.get_top_experiments(top_k=3)
|
||||||
print([t.id for t in top_exp])
|
print([t.id for t in top_exp])
|
||||||
# make sure background optimization stopped
|
# make sure background optimization stopped
|
||||||
an_optimizer.stop()
|
an_optimizer.stop()
|
||||||
|
|
||||||
print('We are done, good bye')
|
print('We are done, good bye')
|
||||||
```
|
```
|
||||||
|
@ -14,38 +14,40 @@ When the script runs, it creates an experiment named `3D plot reporting`, which
|
|||||||
To plot a series as a surface plot, use the [Logger.report_surface](../../references/sdk/logger.md#report_surface)
|
To plot a series as a surface plot, use the [Logger.report_surface](../../references/sdk/logger.md#report_surface)
|
||||||
method.
|
method.
|
||||||
|
|
||||||
# report 3d surface
|
```python
|
||||||
surface = np.random.randint(10, size=(10, 10))
|
# report 3d surface
|
||||||
Logger.current_logger().report_surface(
|
surface = np.random.randint(10, size=(10, 10))
|
||||||
"example_surface",
|
Logger.current_logger().report_surface(
|
||||||
"series1",
|
"example_surface",
|
||||||
iteration=iteration,
|
"series1",
|
||||||
matrix=surface,
|
iteration=iteration,
|
||||||
xaxis="title X",
|
matrix=surface,
|
||||||
yaxis="title Y",
|
xaxis="title X",
|
||||||
zaxis="title Z",
|
yaxis="title Y",
|
||||||
)
|
zaxis="title Z",
|
||||||
|
)
|
||||||
|
```
|
||||||
Visualize the reported surface plot in **RESULTS** **>** **PLOTS**.
|
Visualize the reported surface plot in **RESULTS** **>** **PLOTS**.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## 3D Scatter Plot
|
## 3D Scatter Plot
|
||||||
|
|
||||||
To plot a series as a 3-dimensional scatter plot, use the [Logger.report_scatter3d](../../references/sdk/logger.md#report_scatter3d)
|
To plot a series as a 3-dimensional scatter plot, use the [Logger.report_scatter3d](../../references/sdk/logger.md#report_scatter3d)
|
||||||
method.
|
method.
|
||||||
|
```python
|
||||||
# report 3d scatter plot
|
# report 3d scatter plot
|
||||||
scatter3d = np.random.randint(10, size=(10, 3))
|
scatter3d = np.random.randint(10, size=(10, 3))
|
||||||
Logger.current_logger().report_scatter3d(
|
Logger.current_logger().report_scatter3d(
|
||||||
"example_scatter_3d",
|
"example_scatter_3d",
|
||||||
"series_xyz",
|
"series_xyz",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter3d,
|
scatter=scatter3d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
zaxis="title z",
|
zaxis="title z",
|
||||||
)
|
)
|
||||||
|
```
|
||||||
|
|
||||||
Visualize the reported 3D scatter plot in **RESULTS** **>** **PLOTS**.
|
Visualize the reported 3D scatter plot in **RESULTS** **>** **PLOTS**.
|
||||||

|

|
||||||
|
@ -38,26 +38,30 @@ method. If the Pandas DataFrame changes, **ClearML** uploads the changes. The up
|
|||||||
|
|
||||||
For example:
|
For example:
|
||||||
|
|
||||||
df = pd.DataFrame(
|
```python
|
||||||
{
|
df = pd.DataFrame(
|
||||||
'num_legs': [2, 4, 8, 0],
|
{
|
||||||
'num_wings': [2, 0, 0, 0],
|
'num_legs': [2, 4, 8, 0],
|
||||||
'num_specimen_seen': [10, 2, 1, 8]
|
'num_wings': [2, 0, 0, 0],
|
||||||
},
|
'num_specimen_seen': [10, 2, 1, 8]
|
||||||
index=['falcon', 'dog', 'spider', 'fish']
|
},
|
||||||
)
|
index=['falcon', 'dog', 'spider', 'fish']
|
||||||
|
)
|
||||||
|
|
||||||
# Register Pandas object as artifact to watch
|
# Register Pandas object as artifact to watch
|
||||||
# (it will be monitored in the background and automatically synced and uploaded)
|
# (it will be monitored in the background and automatically synced and uploaded)
|
||||||
task.register_artifact('train', df, metadata={'counting': 'legs', 'max legs': 69}))
|
task.register_artifact('train', df, metadata={'counting': 'legs', 'max legs': 69}))
|
||||||
|
```
|
||||||
|
|
||||||
By changing the artifact, and calling the [Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
|
By changing the artifact, and calling the [Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
|
||||||
method to retrieve it, we can see that **ClearML** tracked the change.
|
method to retrieve it, we can see that **ClearML** tracked the change.
|
||||||
|
|
||||||
# change the artifact object
|
```python
|
||||||
df.sample(frac=0.5, replace=True, random_state=1)
|
# change the artifact object
|
||||||
# or access it from anywhere using the Task's get_registered_artifacts()
|
df.sample(frac=0.5, replace=True, random_state=1)
|
||||||
Task.current_task().get_registered_artifacts()['train'].sample(frac=0.5, replace=True, random_state=1)
|
# or access it from anywhere using the Task's get_registered_artifacts()
|
||||||
|
Task.current_task().get_registered_artifacts()['train'].sample(frac=0.5, replace=True, random_state=1)
|
||||||
|
```
|
||||||
|
|
||||||
## Artifacts Without Tracking
|
## Artifacts Without Tracking
|
||||||
|
|
||||||
@ -75,37 +79,52 @@ Artifacts without tracking include:
|
|||||||
* Wildcards (stored as a ZIP files)
|
* Wildcards (stored as a ZIP files)
|
||||||
|
|
||||||
### Pandas DataFrames
|
### Pandas DataFrames
|
||||||
|
```python
|
||||||
# add and upload pandas.DataFrame (onetime snapshot of the object)
|
# add and upload pandas.DataFrame (onetime snapshot of the object)
|
||||||
task.upload_artifact('Pandas', artifact_object=df)
|
task.upload_artifact('Pandas', artifact_object=df)
|
||||||
|
```
|
||||||
|
|
||||||
### Local Files
|
### Local Files
|
||||||
|
|
||||||
# add and upload local file artifact
|
```python
|
||||||
task.upload_artifact('local file', artifact_object=os.path.join('data_samples', 'dancing.jpg'))
|
# add and upload local file artifact
|
||||||
|
task.upload_artifact(
|
||||||
|
'local file',
|
||||||
|
artifact_object=os.path.join(
|
||||||
|
'data_samples',
|
||||||
|
'dancing.jpg'
|
||||||
|
)
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
### Dictionaries
|
### Dictionaries
|
||||||
|
```python
|
||||||
# add and upload dictionary stored as JSON)
|
# add and upload dictionary stored as JSON)
|
||||||
task.upload_artifact('dictionary', df.to_dict())
|
task.upload_artifact('dictionary', df.to_dict())
|
||||||
|
```
|
||||||
|
|
||||||
### Numpy Objects
|
### Numpy Objects
|
||||||
|
```python
|
||||||
# add and upload Numpy Object (stored as .npz file)
|
# add and upload Numpy Object (stored as .npz file)
|
||||||
task.upload_artifact('Numpy Eye', np.eye(100, 100))
|
task.upload_artifact('Numpy Eye', np.eye(100, 100))
|
||||||
|
```
|
||||||
|
|
||||||
### Image Files
|
### Image Files
|
||||||
|
```python
|
||||||
# add and upload Image (stored as .png file)
|
# add and upload Image (stored as .png file)
|
||||||
im = Image.open(os.path.join('data_samples', 'dancing.jpg'))
|
im = Image.open(os.path.join('data_samples', 'dancing.jpg'))
|
||||||
task.upload_artifact('pillow_image', im)
|
task.upload_artifact('pillow_image', im)
|
||||||
|
```
|
||||||
|
|
||||||
### Folders
|
### Folders
|
||||||
|
```python
|
||||||
# add and upload a folder, artifact_object should be the folder path
|
# add and upload a folder, artifact_object should be the folder path
|
||||||
task.upload_artifact('local folder', artifact_object=os.path.join('data_samples'))
|
task.upload_artifact('local folder', artifact_object=os.path.join('data_samples'))
|
||||||
|
```
|
||||||
|
|
||||||
### Wildcards
|
### Wildcards
|
||||||
|
```python
|
||||||
# add and upload a wildcard
|
# add and upload a wildcard
|
||||||
task.upload_artifact('wildcard jpegs', artifact_object=os.path.join('data_samples', '*.jpg'))
|
task.upload_artifact('wildcard jpegs', artifact_object=os.path.join('data_samples', '*.jpg'))
|
||||||
|
```
|
||||||
|
|
@ -23,8 +23,9 @@ Make a copy of `pytorch_mnist.py` in order to add explicit reporting to it.
|
|||||||
|
|
||||||
* In the local **ClearML** repository, `example` directory.
|
* In the local **ClearML** repository, `example` directory.
|
||||||
|
|
||||||
|
```bash
|
||||||
cp pytorch_mnist.py pytorch_mnist_tutorial.py
|
cp pytorch_mnist.py pytorch_mnist_tutorial.py
|
||||||
|
```
|
||||||
|
|
||||||
## Step 1: Setting an Output Destination for Model Checkpoints
|
## Step 1: Setting an Output Destination for Model Checkpoints
|
||||||
|
|
||||||
@ -42,17 +43,21 @@ In this tutorial, we specify a local folder destination.
|
|||||||
|
|
||||||
In `pytorch_mnist_tutorial.py`, change the code from:
|
In `pytorch_mnist_tutorial.py`, change the code from:
|
||||||
|
|
||||||
task = Task.init(project_name='examples', task_name='pytorch mnist train')
|
```python
|
||||||
|
task = Task.init(project_name='examples', task_name='pytorch mnist train')
|
||||||
|
```
|
||||||
|
|
||||||
to:
|
to:
|
||||||
|
|
||||||
model_snapshots_path = '/mnt/clearml'
|
```python
|
||||||
if not os.path.exists(model_snapshots_path):
|
model_snapshots_path = '/mnt/clearml'
|
||||||
os.makedirs(model_snapshots_path)
|
if not os.path.exists(model_snapshots_path):
|
||||||
|
os.makedirs(model_snapshots_path)
|
||||||
|
|
||||||
task = Task.init(project_name='examples',
|
task = Task.init(project_name='examples',
|
||||||
task_name='extending automagical ClearML example',
|
task_name='extending automagical ClearML example',
|
||||||
output_uri=model_snapshots_path)
|
output_uri=model_snapshots_path)
|
||||||
|
```
|
||||||
|
|
||||||
When the script runs, **ClearML** creates the following directory structure:
|
When the script runs, **ClearML** creates the following directory structure:
|
||||||
|
|
||||||
@ -94,83 +99,106 @@ package contains methods for explicit reporting of plots, log text, media, and t
|
|||||||
|
|
||||||
First, create a logger for the Task using the [Task.get_logger](../../references/sdk/task.md#get_logger)
|
First, create a logger for the Task using the [Task.get_logger](../../references/sdk/task.md#get_logger)
|
||||||
method.
|
method.
|
||||||
|
```python
|
||||||
logger = task.get_logger
|
logger = task.get_logger
|
||||||
|
```
|
||||||
|
|
||||||
### Plot Scalar Metrics
|
### Plot Scalar Metrics
|
||||||
|
|
||||||
Add scalar metrics using the [Logger.report_scalar](../../references/sdk/logger.md#report_scalar)
|
Add scalar metrics using the [Logger.report_scalar](../../references/sdk/logger.md#report_scalar)
|
||||||
method to report loss metrics.
|
method to report loss metrics.
|
||||||
|
|
||||||
def train(args, model, device, train_loader, optimizer, epoch):
|
```python
|
||||||
|
def train(args, model, device, train_loader, optimizer, epoch):
|
||||||
|
|
||||||
save_loss = []
|
save_loss = []
|
||||||
|
|
||||||
model.train()
|
model.train()
|
||||||
for batch_idx, (data, target) in enumerate(train_loader):
|
for batch_idx, (data, target) in enumerate(train_loader):
|
||||||
data, target = data.to(device), target.to(device)
|
data, target = data.to(device), target.to(device)
|
||||||
optimizer.zero_grad()
|
optimizer.zero_grad()
|
||||||
output = model(data)
|
output = model(data)
|
||||||
loss = F.nll_loss(output, target)
|
loss = F.nll_loss(output, target)
|
||||||
loss.backward()
|
loss.backward()
|
||||||
|
|
||||||
save_loss.append(loss)
|
save_loss.append(loss)
|
||||||
|
|
||||||
optimizer.step()
|
optimizer.step()
|
||||||
if batch_idx % args.log_interval == 0:
|
if batch_idx % args.log_interval == 0:
|
||||||
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
|
print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
|
||||||
epoch, batch_idx * len(data), len(train_loader.dataset),
|
epoch, batch_idx * len(data), len(train_loader.dataset),
|
||||||
100. * batch_idx / len(train_loader), loss.item()))
|
100. * batch_idx / len(train_loader), loss.item()))
|
||||||
# Add manual scalar reporting for loss metrics
|
# Add manual scalar reporting for loss metrics
|
||||||
logger.report_scalar(title='Scalar example {} - epoch'.format(epoch),
|
logger.report_scalar(title='Scalar example {} - epoch'.format(epoch),
|
||||||
series='Loss', value=loss.item(), iteration=batch_idx)
|
series='Loss', value=loss.item(), iteration=batch_idx)
|
||||||
|
```
|
||||||
|
|
||||||
### Plot Other (Not Scalar) Data
|
### Plot Other (Not Scalar) Data
|
||||||
|
|
||||||
The script contains a function named `test`, which determines loss and correct for the trained model. We add a histogram
|
The script contains a function named `test`, which determines loss and correct for the trained model. We add a histogram
|
||||||
and confusion matrix to log them.
|
and confusion matrix to log them.
|
||||||
|
|
||||||
def test(args, model, device, test_loader):
|
```python
|
||||||
|
def test(args, model, device, test_loader):
|
||||||
|
|
||||||
save_test_loss = []
|
save_test_loss = []
|
||||||
save_correct = []
|
save_correct = []
|
||||||
|
|
||||||
model.eval()
|
model.eval()
|
||||||
test_loss = 0
|
test_loss = 0
|
||||||
correct = 0
|
correct = 0
|
||||||
with torch.no_grad():
|
with torch.no_grad():
|
||||||
for data, target in test_loader:
|
for data, target in test_loader:
|
||||||
data, target = data.to(device), target.to(device)
|
data, target = data.to(device), target.to(device)
|
||||||
output = model(data)
|
output = model(data)
|
||||||
# sum up batch loss
|
# sum up batch loss
|
||||||
test_loss += F.nll_loss(output, target, reduction='sum').item()
|
test_loss += F.nll_loss(output, target, reduction='sum').item()
|
||||||
# get the index of the max log-probability
|
# get the index of the max log-probability
|
||||||
pred = output.argmax(dim=1, keepdim=True)
|
pred = output.argmax(dim=1, keepdim=True)
|
||||||
correct += pred.eq(target.view_as(pred)).sum().item()
|
correct += pred.eq(target.view_as(pred)).sum().item()
|
||||||
|
|
||||||
save_test_loss.append(test_loss)
|
save_test_loss.append(test_loss)
|
||||||
save_correct.append(correct)
|
save_correct.append(correct)
|
||||||
|
|
||||||
test_loss /= len(test_loader.dataset)
|
test_loss /= len(test_loader.dataset)
|
||||||
|
|
||||||
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
|
print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
|
||||||
test_loss, correct, len(test_loader.dataset),
|
test_loss, correct, len(test_loader.dataset),
|
||||||
100. * correct / len(test_loader.dataset)))
|
100. * correct / len(test_loader.dataset)))
|
||||||
|
|
||||||
logger.report_histogram(title='Histogram example', series='correct',
|
logger.report_histogram(
|
||||||
iteration=1, values=save_correct, xaxis='Test', yaxis='Correct')
|
title='Histogram example',
|
||||||
|
series='correct',
|
||||||
|
iteration=1,
|
||||||
|
values=save_correct,
|
||||||
|
xaxis='Test',
|
||||||
|
yaxis='Correct'
|
||||||
|
)
|
||||||
|
|
||||||
# Manually report test loss and correct as a confusion matrix
|
# Manually report test loss and correct as a confusion matrix
|
||||||
matrix = np.array([save_test_loss, save_correct])
|
matrix = np.array([save_test_loss, save_correct])
|
||||||
logger.report_confusion_matrix(title='Confusion matrix example',
|
logger.report_confusion_matrix(
|
||||||
series='Test loss / correct', matrix=matrix, iteration=1)
|
title='Confusion matrix example',
|
||||||
|
series='Test loss / correct',
|
||||||
|
matrix=matrix,
|
||||||
|
iteration=1
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
### Log Text
|
### Log Text
|
||||||
|
|
||||||
Extend **ClearML** by explicitly logging text, including errors, warnings, and debugging statements. We use the [Logger.report_text](../../references/sdk/logger.md#report_text)
|
Extend **ClearML** by explicitly logging text, including errors, warnings, and debugging statements. We use the [Logger.report_text](../../references/sdk/logger.md#report_text)
|
||||||
method and its argument `level` to report a debugging message.
|
method and its argument `level` to report a debugging message.
|
||||||
|
|
||||||
logger.report_text('The default output destination for model snapshots and artifacts is: {}'.format(model_snapshots_path ), level=logging.DEBUG)
|
```python
|
||||||
|
logger.report_text(
|
||||||
|
'The default output destination for model snapshots and artifacts is: {}'.format(
|
||||||
|
model_snapshots_path
|
||||||
|
),
|
||||||
|
level=logging.DEBUG
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
## Step 3: Registering Artifacts
|
## Step 3: Registering Artifacts
|
||||||
|
|
||||||
@ -182,16 +210,25 @@ Currently, **ClearML** supports Pandas DataFrames as registered artifacts.
|
|||||||
In the tutorial script, `test` function, we can assign the test loss and correct data to a Pandas DataFrame object and register
|
In the tutorial script, `test` function, we can assign the test loss and correct data to a Pandas DataFrame object and register
|
||||||
that Pandas DataFrame using the [Task.register_artifact](../../references/sdk/task.md#register_artifact) method.
|
that Pandas DataFrame using the [Task.register_artifact](../../references/sdk/task.md#register_artifact) method.
|
||||||
|
|
||||||
# Create the Pandas DataFrame
|
```python
|
||||||
test_loss_correct = {
|
# Create the Pandas DataFrame
|
||||||
'test lost': save_test_loss,
|
test_loss_correct = {
|
||||||
'correct': save_correct
|
'test lost': save_test_loss,
|
||||||
}
|
'correct': save_correct
|
||||||
df = pd.DataFrame(test_loss_correct, columns=['test lost','correct'])
|
}
|
||||||
|
df = pd.DataFrame(test_loss_correct, columns=['test lost','correct'])
|
||||||
|
|
||||||
# Register the test loss and correct as a Pandas DataFrame artifact
|
# Register the test loss and correct as a Pandas DataFrame artifact
|
||||||
task.register_artifact('Test_Loss_Correct', df, metadata={'metadata string': 'apple',
|
task.register_artifact(
|
||||||
'metadata int': 100, 'metadata dict': {'dict string': 'pear', 'dict int': 200}})
|
'Test_Loss_Correct',
|
||||||
|
df,
|
||||||
|
metadata={
|
||||||
|
'metadata string': 'apple',
|
||||||
|
'metadata int': 100,
|
||||||
|
'metadata dict': {'dict string': 'pear', 'dict int': 200}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
### Reference the Registered Artifact
|
### Reference the Registered Artifact
|
||||||
|
|
||||||
@ -201,9 +238,15 @@ In the tutorial script, we add [Task.current_task](../../references/sdk/task.md#
|
|||||||
[Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
|
[Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
|
||||||
methods to take a sample.
|
methods to take a sample.
|
||||||
|
|
||||||
# Once the artifact is registered, we can get it and work with it. Here, we sample it.
|
```python
|
||||||
sample = Task.current_task().get_registered_artifacts()['Test_Loss_Correct'].sample(frac=0.5,
|
# Once the artifact is registered, we can get it and work with it. Here, we sample it.
|
||||||
replace=True, random_state=1)
|
sample = Task.current_task().get_registered_artifacts()['Test_Loss_Correct'].sample(
|
||||||
|
frac=0.5,
|
||||||
|
replace=True,
|
||||||
|
random_state=1
|
||||||
|
)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
## Step 4: Uploading Artifacts
|
## Step 4: Uploading Artifacts
|
||||||
|
|
||||||
@ -220,10 +263,18 @@ Supported artifacts include:
|
|||||||
In the tutorial script, we upload the loss data as an artifact using the [Task.upload_artifact](../../references/sdk/task.md#upload_artifact)
|
In the tutorial script, we upload the loss data as an artifact using the [Task.upload_artifact](../../references/sdk/task.md#upload_artifact)
|
||||||
method with metadata specified in the `metadata` parameter.
|
method with metadata specified in the `metadata` parameter.
|
||||||
|
|
||||||
# Upload test loss as an artifact. Here, the artifact is numpy array
|
```python
|
||||||
task.upload_artifact('Predictions',artifact_object=np.array(save_test_loss),
|
# Upload test loss as an artifact. Here, the artifact is numpy array
|
||||||
metadata={'metadata string': 'banana', 'metadata integer': 300,
|
task.upload_artifact(
|
||||||
'metadata dictionary': {'dict string': 'orange', 'dict int': 400}})
|
'Predictions',
|
||||||
|
artifact_object=np.array(save_test_loss),
|
||||||
|
metadata={
|
||||||
|
'metadata string': 'banana',
|
||||||
|
'metadata integer': 300,
|
||||||
|
'metadata dictionary': {'dict string': 'orange', 'dict int': 400}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
## Additional Information
|
## Additional Information
|
||||||
|
|
||||||
|
@ -33,28 +33,41 @@ Report the following using the `Logger.report_media` parameter method `local_pat
|
|||||||
### Interactive HTML
|
### Interactive HTML
|
||||||
|
|
||||||
See the example script's [report_html_periodic_table](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L26) function, which reports a file created from Bokeh sample data.
|
See the example script's [report_html_periodic_table](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L26) function, which reports a file created from Bokeh sample data.
|
||||||
|
```python
|
||||||
Logger.current_logger().report_media("html", "periodic_html", iteration=iteration, local_path="periodic.html")
|
Logger.current_logger().report_media(
|
||||||
|
"html", "periodic_html", iteration=iteration, local_path="periodic.html"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
### Bokeh GroupBy HTML
|
### Bokeh GroupBy HTML
|
||||||
|
|
||||||
See the example script's [report_html_groupby](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L117) function, which reports a Pandas GroupBy with nested HTML, created from Bokeh sample data.
|
See the example script's [report_html_groupby](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L117) function, which reports a Pandas GroupBy with nested HTML, created from Bokeh sample data.
|
||||||
|
```python
|
||||||
|
Logger.current_logger().report_media(
|
||||||
|
"html",
|
||||||
|
"pandas_groupby_nested_html",
|
||||||
|
iteration=iteration,
|
||||||
|
local_path="bar_pandas_groupby_nested.html",
|
||||||
|
)
|
||||||
|
|
||||||
Logger.current_logger().report_media(
|
```
|
||||||
"html",
|
|
||||||
"pandas_groupby_nested_html",
|
|
||||||
iteration=iteration,
|
|
||||||
local_path="bar_pandas_groupby_nested.html",
|
|
||||||
)
|
|
||||||
|
|
||||||
### Bokeh Graph HTML
|
### Bokeh Graph HTML
|
||||||
|
|
||||||
See the example script's [report_html_graph](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L162) function, which reports a Bokeh plot created from Bokeh sample data.
|
See the example script's [report_html_graph](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L162) function, which reports a Bokeh plot created from Bokeh sample data.
|
||||||
|
|
||||||
Logger.current_logger().report_media("html", "Graph_html", iteration=iteration, local_path="graph.html")
|
```python
|
||||||
|
Logger.current_logger().report_media(
|
||||||
|
"html", "Graph_html", iteration=iteration, local_path="graph.html"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
### Bokeh Image HTML
|
### Bokeh Image HTML
|
||||||
|
|
||||||
See the example script's [report_html_image](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L195) function, which reports an image created from Bokeh sample data.
|
See the example script's [report_html_image](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L195) function, which reports an image created from Bokeh sample data.
|
||||||
|
|
||||||
Logger.current_logger().report_media("html", "Spectral_html", iteration=iteration, local_path="image.html")
|
```python
|
||||||
|
Logger.current_logger().report_media(
|
||||||
|
"html", "Spectral_html", iteration=iteration, local_path="image.html"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
@ -16,13 +16,17 @@ When the script runs, it creates an experiment named `hyper-parameters example`,
|
|||||||
## Argparse Command Line Options
|
## Argparse Command Line Options
|
||||||
|
|
||||||
If a code uses argparse and initializes a Task, **ClearML** automatically logs the argparse arguments.
|
If a code uses argparse and initializes a Task, **ClearML** automatically logs the argparse arguments.
|
||||||
|
|
||||||
parser = ArgumentParser()
|
```python
|
||||||
parser.add_argument('--argparser_int_value', help='integer value', type=int, default=1)
|
parser = ArgumentParser()
|
||||||
parser.add_argument('--argparser_disabled', action='store_true', default=False, help='disables something')
|
parser.add_argument('--argparser_int_value', help='integer value', type=int, default=1)
|
||||||
parser.add_argument('--argparser_str_value', help='string value', default='a string')
|
parser.add_argument(
|
||||||
|
'--argparser_disabled', action='store_true', default=False, help='disables something'
|
||||||
|
)
|
||||||
|
parser.add_argument('--argparser_str_value', help='string value', default='a string')
|
||||||
|
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
```
|
||||||
|
|
||||||
Command line options appears in **HYPER PARAMETERS** **>** **Args**.
|
Command line options appears in **HYPER PARAMETERS** **>** **Args**.
|
||||||
|
|
||||||
@ -32,14 +36,17 @@ Command line options appears in **HYPER PARAMETERS** **>** **Args**.
|
|||||||
|
|
||||||
**ClearML** automatically logs TensorFlow Definitions, whether they are defined before or after the Task is initialized.
|
**ClearML** automatically logs TensorFlow Definitions, whether they are defined before or after the Task is initialized.
|
||||||
|
|
||||||
flags.DEFINE_string('echo', None, 'Text to echo.')
|
```python
|
||||||
flags.DEFINE_string('another_str', 'My string', 'A string', module_name='test')
|
flags.DEFINE_string('echo', None, 'Text to echo.')
|
||||||
|
flags.DEFINE_string('another_str', 'My string', 'A string', module_name='test')
|
||||||
|
|
||||||
task = Task.init(project_name='examples', task_name='hyper-parameters example')
|
task = Task.init(project_name='examples', task_name='hyper-parameters example')
|
||||||
|
|
||||||
flags.DEFINE_integer('echo3', 3, 'Text to echo.')
|
flags.DEFINE_integer('echo3', 3, 'Text to echo.')
|
||||||
|
|
||||||
flags.DEFINE_string('echo5', '5', 'Text to echo.', module_name='test')
|
flags.DEFINE_string('echo5', '5', 'Text to echo.', module_name='test')
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
TensorFlow Definitions appear in **HYPER PARAMETERS** **>** **TF_DEFINE**.
|
TensorFlow Definitions appear in **HYPER PARAMETERS** **>** **TF_DEFINE**.
|
||||||
|
|
||||||
@ -50,22 +57,25 @@ TensorFlow Definitions appear in **HYPER PARAMETERS** **>** **TF_DEFINE**.
|
|||||||
Connect a parameter dictionary to a Task by calling the [Task.connect](../../references/sdk/task.md#connect)
|
Connect a parameter dictionary to a Task by calling the [Task.connect](../../references/sdk/task.md#connect)
|
||||||
method, and **ClearML** logs the parameters. **ClearML** also tracks changes to the parameters.
|
method, and **ClearML** logs the parameters. **ClearML** also tracks changes to the parameters.
|
||||||
|
|
||||||
parameters = {
|
```python
|
||||||
'list': [1, 2, 3],
|
parameters = {
|
||||||
'dict': {'a': 1, 'b': 2},
|
'list': [1, 2, 3],
|
||||||
'tuple': (1, 2, 3),
|
'dict': {'a': 1, 'b': 2},
|
||||||
'int': 3,
|
'tuple': (1, 2, 3),
|
||||||
'float': 2.2,
|
'int': 3,
|
||||||
'string': 'my string',
|
'float': 2.2,
|
||||||
}
|
'string': 'my string',
|
||||||
|
}
|
||||||
|
|
||||||
parameters = task.connect(parameters)
|
parameters = task.connect(parameters)
|
||||||
|
|
||||||
# adding new parameter after connect (will be logged as well)
|
# adding new parameter after connect (will be logged as well)
|
||||||
parameters['new_param'] = 'this is new'
|
parameters['new_param'] = 'this is new'
|
||||||
|
|
||||||
# changing the value of a parameter (new value will be stored instead of previous one)
|
# changing the value of a parameter (new value will be stored instead of previous one)
|
||||||
parameters['float'] = '9.9'
|
parameters['float'] = '9.9'
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
Parameters from dictionaries connected to Tasks appear in **HYPER PARAMETERS** **>** **General**.
|
Parameters from dictionaries connected to Tasks appear in **HYPER PARAMETERS** **>** **General**.
|
||||||
|
|
||||||
|
@ -20,27 +20,39 @@ When the script runs, it creates an experiment named `image reporting`, which is
|
|||||||
Report images using several formats by calling the [Logger.report_image](../../references/sdk/logger.md#report_image)
|
Report images using several formats by calling the [Logger.report_image](../../references/sdk/logger.md#report_image)
|
||||||
method:
|
method:
|
||||||
|
|
||||||
# report image as float image
|
```python
|
||||||
m = np.eye(256, 256, dtype=np.float)
|
# report image as float image
|
||||||
Logger.current_logger().report_image("image", "image float", iteration=iteration, image=m)
|
m = np.eye(256, 256, dtype=np.float)
|
||||||
|
Logger.current_logger().report_image("image", "image float", iteration=iteration, image=m)
|
||||||
|
|
||||||
# report image as uint8
|
# report image as uint8
|
||||||
m = np.eye(256, 256, dtype=np.uint8) * 255
|
m = np.eye(256, 256, dtype=np.uint8) * 255
|
||||||
Logger.current_logger().report_image("image", "image uint8", iteration=iteration, image=m)
|
Logger.current_logger().report_image("image", "image uint8", iteration=iteration, image=m)
|
||||||
|
|
||||||
# report image as uint8 RGB
|
# report image as uint8 RGB
|
||||||
m = np.concatenate((np.atleast_3d(m), np.zeros((256, 256, 2), dtype=np.uint8)), axis=2)
|
m = np.concatenate((np.atleast_3d(m), np.zeros((256, 256, 2), dtype=np.uint8)), axis=2)
|
||||||
Logger.current_logger().report_image("image", "image color red", iteration=iteration, image=m)
|
Logger.current_logger().report_image(
|
||||||
|
"image",
|
||||||
|
"image color red",
|
||||||
|
iteration=iteration,
|
||||||
|
image=m
|
||||||
|
)
|
||||||
|
|
||||||
# report PIL Image object
|
# report PIL Image object
|
||||||
image_open = Image.open(os.path.join("data_samples", "picasso.jpg"))
|
image_open = Image.open(os.path.join("data_samples", "picasso.jpg"))
|
||||||
Logger.current_logger().report_image("image", "image PIL", iteration=iteration, image=image_open)
|
Logger.current_logger().report_image(
|
||||||
|
"image",
|
||||||
|
"image PIL",
|
||||||
|
iteration=iteration,
|
||||||
|
image=image_open
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
**ClearML** reports these images as debug samples in the **ClearML Web UI** **>** experiment details **>** **RESULTS** tab
|
**ClearML** reports these images as debug samples in the **ClearML Web UI** **>** experiment details **>** **RESULTS** tab
|
||||||
**>** **DEBUG SAMPLES** sub-tab.
|
**>** **DEBUG SAMPLES** sub-tab.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Double click a thumbnail and the image viewer opens.
|
Double click a thumbnail, and the image viewer opens.
|
||||||
|
|
||||||

|

|
@ -24,15 +24,19 @@ project.
|
|||||||
Report by calling the [Logger.report_media](../../references/sdk/logger.md#report_media)
|
Report by calling the [Logger.report_media](../../references/sdk/logger.md#report_media)
|
||||||
method using the `url` parameter.
|
method using the `url` parameter.
|
||||||
|
|
||||||
# report video, an already uploaded video media (url)
|
```python
|
||||||
Logger.current_logger().report_media(
|
# report video, an already uploaded video media (url)
|
||||||
'video', 'big bunny', iteration=1,
|
Logger.current_logger().report_media(
|
||||||
url='https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4')
|
'video', 'big bunny', iteration=1,
|
||||||
|
url='https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/720/Big_Buck_Bunny_720_10s_1MB.mp4'
|
||||||
|
)
|
||||||
|
|
||||||
# report audio, report an already uploaded audio media (url)
|
# report audio, report an already uploaded audio media (url)
|
||||||
Logger.current_logger().report_media(
|
Logger.current_logger().report_media(
|
||||||
'audio', 'pink panther', iteration=1,
|
'audio', 'pink panther', iteration=1,
|
||||||
url='https://www2.cs.uic.edu/~i101/SoundFiles/PinkPanther30.wav')
|
url='https://www2.cs.uic.edu/~i101/SoundFiles/PinkPanther30.wav'
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
The reported audio can be viewed in the **DEBUG SAMPLES** sub-tab. Double click a thumbnail, and the audio player opens.
|
The reported audio can be viewed in the **DEBUG SAMPLES** sub-tab. Double click a thumbnail, and the audio player opens.
|
||||||
|
|
||||||
@ -43,10 +47,13 @@ The reported audio can be viewed in the **DEBUG SAMPLES** sub-tab. Double click
|
|||||||
|
|
||||||
Use the `local_path` parameter.
|
Use the `local_path` parameter.
|
||||||
|
|
||||||
# report audio, report local media audio file
|
```python
|
||||||
Logger.current_logger().report_media(
|
# report audio, report local media audio file
|
||||||
'audio', 'tada', iteration=1,
|
Logger.current_logger().report_media(
|
||||||
local_path=os.path.join('data_samples', 'sample.mp3'))
|
'audio', 'tada', iteration=1,
|
||||||
|
local_path=os.path.join('data_samples', 'sample.mp3')
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
The reported video can be viewed in the **DEBUG SAMPLES** sub-tab. Double click a thumbnail, and the video player opens.
|
The reported video can be viewed in the **DEBUG SAMPLES** sub-tab. Double click a thumbnail, and the video player opens.
|
||||||
|
|
||||||
|
@ -17,11 +17,12 @@ Connect a configuration file to a Task by calling the [Task.connect_configuratio
|
|||||||
method with the file location and the configuration object's name as arguments. In this example, we connect a JSON file and a YAML file
|
method with the file location and the configuration object's name as arguments. In this example, we connect a JSON file and a YAML file
|
||||||
to a Task.
|
to a Task.
|
||||||
|
|
||||||
config_file_json = 'data_samples/sample.json'
|
```python
|
||||||
task.connect_configuration(name="json file", configuration=config_file_json)
|
config_file_json = 'data_samples/sample.json'
|
||||||
...
|
task.connect_configuration(name="json file", configuration=config_file_json)
|
||||||
config_file_yaml = 'data_samples/config_yaml.yaml'
|
config_file_yaml = 'data_samples/config_yaml.yaml'
|
||||||
task.connect_configuration(configuration=config_file_yaml, name="yaml file")
|
task.connect_configuration(configuration=config_file_yaml, name="yaml file")
|
||||||
|
```
|
||||||
|
|
||||||
The configuration is logged to the ClearML Task and can be viewed in the **ClearML Web UI** experiment details **>** **CONFIGURATION** tab **>** **CONFIGURATION OBJECTS**
|
The configuration is logged to the ClearML Task and can be viewed in the **ClearML Web UI** experiment details **>** **CONFIGURATION** tab **>** **CONFIGURATION OBJECTS**
|
||||||
section. The contents of the JSON file will appear in the **json file** object, and the contents of the YAML file will appear
|
section. The contents of the JSON file will appear in the **json file** object, and the contents of the YAML file will appear
|
||||||
@ -34,17 +35,21 @@ in the **yaml file** object, as specified in the `name` parameter of the `connec
|
|||||||
Connect a configuration dictionary to a Task by creating a dictionary, and then calling the [Task.connect_configuration](../../references/sdk/task.md#connect_configuration)
|
Connect a configuration dictionary to a Task by creating a dictionary, and then calling the [Task.connect_configuration](../../references/sdk/task.md#connect_configuration)
|
||||||
method with the dictionary and the object name as arguments. After the configuration is connected, **ClearML** tracks changes to it.
|
method with the dictionary and the object name as arguments. After the configuration is connected, **ClearML** tracks changes to it.
|
||||||
|
|
||||||
model_config_dict = {
|
```python
|
||||||
'CHANGE ME': 13.37,
|
model_config_dict = {
|
||||||
'dict': {'sub_value': 'string', 'sub_integer': 11},
|
'CHANGE ME': 13.37,
|
||||||
'list_of_ints': [1, 2, 3, 4],
|
'dict': {'sub_value': 'string', 'sub_integer': 11},
|
||||||
}
|
'list_of_ints': [1, 2, 3, 4],
|
||||||
model_config_dict = task.connect_configuration(name='dictionary', configuration=model_config_dict)
|
}
|
||||||
|
model_config_dict = task.connect_configuration(
|
||||||
# Update the dictionary after connecting it, and the changes will be tracked as well.
|
name='dictionary',
|
||||||
model_config_dict['new value'] = 10
|
configuration=model_config_dict
|
||||||
model_config_dict['CHANGE ME'] *= model_config_dict['new value']
|
)
|
||||||
|
|
||||||
|
# Update the dictionary after connecting it, and the changes will be tracked as well.
|
||||||
|
model_config_dict['new value'] = 10
|
||||||
|
model_config_dict['CHANGE ME'] *= model_config_dict['new value']
|
||||||
|
```
|
||||||
The configurations are connected to the ClearML Task and can be viewed in the **ClearML Web UI** **>** experiment details **>** **CONFIGURATION** tab **>**
|
The configurations are connected to the ClearML Task and can be viewed in the **ClearML Web UI** **>** experiment details **>** **CONFIGURATION** tab **>**
|
||||||
**CONFIGURATION OBJECTS** area **>** **dictionary** object.
|
**CONFIGURATION OBJECTS** area **>** **dictionary** object.
|
||||||
|
|
||||||
@ -55,13 +60,16 @@ The configurations are connected to the ClearML Task and can be viewed in the **
|
|||||||
Connect a label enumeration dictionary by creating the dictionary, and then calling the [Task.connect_label_enumeration](../../references/sdk/task.md#connect_label_enumeration)
|
Connect a label enumeration dictionary by creating the dictionary, and then calling the [Task.connect_label_enumeration](../../references/sdk/task.md#connect_label_enumeration)
|
||||||
method with the dictionary as an argument.
|
method with the dictionary as an argument.
|
||||||
|
|
||||||
# store the label enumeration of the training model
|
```python
|
||||||
labels = {'background': 0, 'cat': 1, 'dog': 2}
|
# store the label enumeration of the training model
|
||||||
task.connect_label_enumeration(labels)
|
labels = {'background': 0, 'cat': 1, 'dog': 2}
|
||||||
|
task.connect_label_enumeration(labels)
|
||||||
|
```
|
||||||
|
|
||||||
Log a local model file.
|
Log a local model file.
|
||||||
|
```python
|
||||||
OutputModel().update_weights('my_best_model.bin')
|
OutputModel().update_weights('my_best_model.bin')
|
||||||
|
```
|
||||||
|
|
||||||
The model which is stored contains the model configuration and the label enumeration.
|
The model which is stored contains the model configuration and the label enumeration.
|
||||||
|
|
||||||
|
@ -14,17 +14,24 @@ When the script runs, it creates an experiment named `pandas table reporting`, w
|
|||||||
Report Pandas DataFrames by calling the [Logger.report_table](../../references/sdk/logger.md#report_table)
|
Report Pandas DataFrames by calling the [Logger.report_table](../../references/sdk/logger.md#report_table)
|
||||||
method, and providing the DataFrame in the `table_plot` parameter.
|
method, and providing the DataFrame in the `table_plot` parameter.
|
||||||
|
|
||||||
# Report table - DataFrame with index
|
```python
|
||||||
df = pd.DataFrame(
|
# Report table - DataFrame with index
|
||||||
{
|
df = pd.DataFrame(
|
||||||
"num_legs": [2, 4, 8, 0],
|
{
|
||||||
"num_wings": [2, 0, 0, 0],
|
"num_legs": [2, 4, 8, 0],
|
||||||
"num_specimen_seen": [10, 2, 1, 8],
|
"num_wings": [2, 0, 0, 0],
|
||||||
},
|
"num_specimen_seen": [10, 2, 1, 8],
|
||||||
index=["falcon", "dog", "spider", "fish"],
|
},
|
||||||
)
|
index=["falcon", "dog", "spider", "fish"],
|
||||||
df.index.name = "id"
|
)
|
||||||
Logger.current_logger().report_table("table pd", "PD with index", iteration=iteration, table_plot=df)
|
df.index.name = "id"
|
||||||
|
Logger.current_logger().report_table(
|
||||||
|
"table pd",
|
||||||
|
"PD with index",
|
||||||
|
iteration=iteration,
|
||||||
|
table_plot=df
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@ -32,8 +39,15 @@ method, and providing the DataFrame in the `table_plot` parameter.
|
|||||||
|
|
||||||
Report CSV files by providing the URL location of the CSV file in the `url` parameter. For a local CSV file, use the `csv` parameter.
|
Report CSV files by providing the URL location of the CSV file in the `url` parameter. For a local CSV file, use the `csv` parameter.
|
||||||
|
|
||||||
# Report table - CSV from path
|
```python
|
||||||
csv_url = "https://raw.githubusercontent.com/plotly/datasets/master/Mining-BTC-180.csv"
|
# Report table - CSV from path
|
||||||
Logger.current_logger().report_table("table csv", "remote csv", iteration=iteration, url=csv_url)
|
csv_url = "https://raw.githubusercontent.com/plotly/datasets/master/Mining-BTC-180.csv"
|
||||||
|
Logger.current_logger().report_table(
|
||||||
|
"table csv",
|
||||||
|
"remote csv",
|
||||||
|
iteration=iteration,
|
||||||
|
url=csv_url
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||

|

|
@ -11,14 +11,25 @@ Plotly figure, using the `figure` parameter.
|
|||||||
In this example, the Plotly figure is created using `plotly.express.scatter` (see [Scatter Plots in Python](https://plotly.com/python/line-and-scatter/)
|
In this example, the Plotly figure is created using `plotly.express.scatter` (see [Scatter Plots in Python](https://plotly.com/python/line-and-scatter/)
|
||||||
in the Plotly documentation):
|
in the Plotly documentation):
|
||||||
|
|
||||||
# Iris dataset
|
```python
|
||||||
df = px.data.iris()
|
# Iris dataset
|
||||||
|
df = px.data.iris()
|
||||||
|
|
||||||
# create complex plotly figure
|
# create complex plotly figure
|
||||||
fig = px.scatter(df, x="sepal_width", y="sepal_length", color="species", marginal_y="rug", marginal_x="histogram")
|
fig = px.scatter(
|
||||||
|
df,
|
||||||
|
x="sepal_width",
|
||||||
|
y="sepal_length",
|
||||||
|
color="species",
|
||||||
|
marginal_y="rug",
|
||||||
|
marginal_x="histogram"
|
||||||
|
)
|
||||||
|
|
||||||
# report the plotly figure
|
# report the plotly figure
|
||||||
task.get_logger().report_plotly(title="iris", series="sepal", iteration=0, figure=fig)
|
task.get_logger().report_plotly(
|
||||||
|
title="iris", series="sepal", iteration=0, figure=fig
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
When the script runs, it creates an experiment named `plotly reporting`, which is associated with the examples project.
|
When the script runs, it creates an experiment named `plotly reporting`, which is associated with the examples project.
|
||||||
|
|
||||||
|
@ -12,14 +12,24 @@ To reports scalars, call the [Logger.report_scalar](../../references/sdk/logger.
|
|||||||
method. To report more than one series on the same plot, use the same `title` argument. For different plots, use different
|
method. To report more than one series on the same plot, use the same `title` argument. For different plots, use different
|
||||||
`title` arguments.
|
`title` arguments.
|
||||||
|
|
||||||
# report two scalar series on the same graph
|
```python
|
||||||
for i in range(100):
|
# report two scalar series on the same graph
|
||||||
Logger.current_logger().report_scalar("unified graph", "series A", iteration=i, value=1./(i+1))
|
for i in range(100):
|
||||||
Logger.current_logger().report_scalar("unified graph", "series B", iteration=i, value=10./(i+1))
|
Logger.current_logger().report_scalar(
|
||||||
|
"unified graph", "series A", iteration=i, value=1./(i+1)
|
||||||
|
)
|
||||||
|
Logger.current_logger().report_scalar(
|
||||||
|
"unified graph", "series B", iteration=i, value=10./(i+1)
|
||||||
|
)
|
||||||
|
|
||||||
# report two scalar series on two different graphs
|
# report two scalar series on two different graphs
|
||||||
for i in range(100):
|
for i in range(100):
|
||||||
Logger.current_logger().report_scalar("graph A", "series A", iteration=i, value=1./(i+1))
|
Logger.current_logger().report_scalar(
|
||||||
Logger.current_logger().report_scalar("graph B", "series B", iteration=i, value=10./(i+1))
|
"graph A", "series A", iteration=i, value=1./(i+1)
|
||||||
|
)
|
||||||
|
Logger.current_logger().report_scalar(
|
||||||
|
"graph B", "series B", iteration=i, value=10./(i+1)
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||

|

|
@ -19,37 +19,39 @@ method. To report more than one series on the same plot, use same the `title` ar
|
|||||||
`title` arguments. Specify the type of histogram with the `mode` parameter. The `mode` values are `group` (the default),
|
`title` arguments. Specify the type of histogram with the `mode` parameter. The `mode` values are `group` (the default),
|
||||||
`stack`, and `relative`.
|
`stack`, and `relative`.
|
||||||
|
|
||||||
# report a single histogram
|
```python
|
||||||
histogram = np.random.randint(10, size=10)
|
# report a single histogram
|
||||||
Logger.current_logger().report_histogram(
|
histogram = np.random.randint(10, size=10)
|
||||||
"single_histogram",
|
Logger.current_logger().report_histogram(
|
||||||
"random histogram",
|
"single_histogram",
|
||||||
iteration=iteration,
|
"random histogram",
|
||||||
values=histogram,
|
iteration=iteration,
|
||||||
xaxis="title x",
|
values=histogram,
|
||||||
yaxis="title y",
|
xaxis="title x",
|
||||||
)
|
yaxis="title y",
|
||||||
|
)
|
||||||
|
|
||||||
# report two histograms on the same graph (plot)
|
# report two histograms on the same graph (plot)
|
||||||
histogram1 = np.random.randint(13, size=10)
|
histogram1 = np.random.randint(13, size=10)
|
||||||
histogram2 = histogram * 0.75
|
histogram2 = histogram * 0.75
|
||||||
Logger.current_logger().report_histogram(
|
Logger.current_logger().report_histogram(
|
||||||
"two_histogram",
|
"two_histogram",
|
||||||
"series 1",
|
"series 1",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
values=histogram1,
|
values=histogram1,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
)
|
)
|
||||||
|
|
||||||
Logger.current_logger().report_histogram(
|
Logger.current_logger().report_histogram(
|
||||||
"two_histogram",
|
"two_histogram",
|
||||||
"series 2",
|
"series 2",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
values=histogram2,
|
values=histogram2,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
)
|
)
|
||||||
|
```
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@ -60,69 +62,75 @@ method. To report more than one series on the same plot, use same the `title` ar
|
|||||||
Report confusion matrices by calling the [Logger.report_matrix](../../references/sdk/logger.md#report_matrix)
|
Report confusion matrices by calling the [Logger.report_matrix](../../references/sdk/logger.md#report_matrix)
|
||||||
method.
|
method.
|
||||||
|
|
||||||
# report confusion matrix
|
```python
|
||||||
confusion = np.random.randint(10, size=(10, 10))
|
# report confusion matrix
|
||||||
Logger.current_logger().report_matrix(
|
confusion = np.random.randint(10, size=(10, 10))
|
||||||
"example_confusion",
|
Logger.current_logger().report_matrix(
|
||||||
"ignored",
|
"example_confusion",
|
||||||
iteration=iteration,
|
"ignored",
|
||||||
matrix=confusion,
|
iteration=iteration,
|
||||||
xaxis="title X",
|
matrix=confusion,
|
||||||
yaxis="title Y",
|
xaxis="title X",
|
||||||
)
|
yaxis="title Y",
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
# report confusion matrix with 0,0 is at the top left
|
```python
|
||||||
Logger.current_logger().report_matrix(
|
# report confusion matrix with 0,0 is at the top left
|
||||||
"example_confusion_0_0_at_top",
|
Logger.current_logger().report_matrix(
|
||||||
"ignored",
|
"example_confusion_0_0_at_top",
|
||||||
iteration=iteration,
|
"ignored",
|
||||||
matrix=confusion,
|
iteration=iteration,
|
||||||
xaxis="title X",
|
matrix=confusion,
|
||||||
yaxis="title Y",
|
xaxis="title X",
|
||||||
yaxis_reversed=True,
|
yaxis="title Y",
|
||||||
)
|
yaxis_reversed=True,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
## 2D Scatter Plots
|
## 2D Scatter Plots
|
||||||
|
|
||||||
Report 2D scatter plots by calling the [Logger.report_scatter2d](../../references/sdk/logger.md#report_scatter2d)
|
Report 2D scatter plots by calling the [Logger.report_scatter2d](../../references/sdk/logger.md#report_scatter2d)
|
||||||
method. Use the `mode` parameter to plot data points with lines (by default), markers, or both lines and markers.
|
method. Use the `mode` parameter to plot data points with lines (by default), markers, or both lines and markers.
|
||||||
|
|
||||||
scatter2d = np.hstack(
|
```python
|
||||||
(np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1)))
|
scatter2d = np.hstack(
|
||||||
)
|
(np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1)))
|
||||||
|
)
|
||||||
|
|
||||||
# report 2d scatter plot with lines
|
# report 2d scatter plot with lines
|
||||||
Logger.current_logger().report_scatter2d(
|
Logger.current_logger().report_scatter2d(
|
||||||
"example_scatter",
|
"example_scatter",
|
||||||
"series_xy",
|
"series_xy",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter2d,
|
scatter=scatter2d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
)
|
)
|
||||||
|
|
||||||
# report 2d scatter plot with markers
|
# report 2d scatter plot with markers
|
||||||
Logger.current_logger().report_scatter2d(
|
Logger.current_logger().report_scatter2d(
|
||||||
"example_scatter",
|
"example_scatter",
|
||||||
"series_markers",
|
"series_markers",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter2d,
|
scatter=scatter2d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
mode='markers'
|
mode='markers'
|
||||||
)
|
)
|
||||||
|
|
||||||
# report 2d scatter plot with lines and markers
|
# report 2d scatter plot with lines and markers
|
||||||
Logger.current_logger().report_scatter2d(
|
Logger.current_logger().report_scatter2d(
|
||||||
"example_scatter",
|
"example_scatter",
|
||||||
"series_lines+markers",
|
"series_lines+markers",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter2d,
|
scatter=scatter2d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
mode='lines+markers'
|
mode='lines+markers'
|
||||||
)
|
)
|
||||||
|
```
|
||||||
|
|
||||||

|

|
@ -342,11 +342,19 @@ one ROI labeled with both `Car` and `largely_occluded` will be input.
|
|||||||
```python
|
```python
|
||||||
myDataView = DataView(iteration_order=IterationOrder.random, iteration_infinite=True)
|
myDataView = DataView(iteration_order=IterationOrder.random, iteration_infinite=True)
|
||||||
|
|
||||||
myDataView.add_query(dataset_name='myDataset', version_name='training',
|
myDataView.add_query(
|
||||||
roi_query='Car', weight = 1)
|
dataset_name='myDataset',
|
||||||
|
version_name='training',
|
||||||
|
roi_query='Car',
|
||||||
|
weight = 1
|
||||||
|
)
|
||||||
|
|
||||||
myDataView.add_query(dataset_name='myDataset', version_name='training',
|
myDataView.add_query(
|
||||||
roi_query='label.keyword:\"Car\" AND label.keyword:\"largely_occluded\"', weight = 5)
|
dataset_name='myDataset',
|
||||||
|
version_name='training',
|
||||||
|
roi_query='label.keyword:\"Car\" AND label.keyword:\"largely_occluded\"',
|
||||||
|
weight = 5
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Mapping ROI Labels
|
### Mapping ROI Labels
|
||||||
|
@ -251,6 +251,7 @@ mask value as a list with the RGB values in the `mask_rgb` parameter, and a list
|
|||||||
frame = SingleFrame(
|
frame = SingleFrame(
|
||||||
source='/home/user/woof_meow.jpg',
|
source='/home/user/woof_meow.jpg',
|
||||||
preview_uri='https://storage.googleapis.com/kaggle-competitions/kaggle/3362/media/woof_meow.jpg',
|
preview_uri='https://storage.googleapis.com/kaggle-competitions/kaggle/3362/media/woof_meow.jpg',
|
||||||
|
)
|
||||||
|
|
||||||
frame.add_annotation(mask_rgb=[0, 0, 0], labels=['cat'])
|
frame.add_annotation(mask_rgb=[0, 0, 0], labels=['cat'])
|
||||||
```
|
```
|
||||||
|
@ -35,7 +35,7 @@ When archiving an experiment:
|
|||||||
|
|
||||||
* Experiments or models table - Right click the experiment or model **>** **Restore**.
|
* Experiments or models table - Right click the experiment or model **>** **Restore**.
|
||||||
* Info panel or full screen details view - Click <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Bars menu" className="icon size-sm space-sm" />
|
* Info panel or full screen details view - Click <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Bars menu" className="icon size-sm space-sm" />
|
||||||
(menu) **>** **Restore from archive**.
|
(menu) **>** **Restore from Archive**.
|
||||||
|
|
||||||
* Restore multiple experiments or models from the:
|
* Restore multiple experiments or models from the:
|
||||||
|
|
||||||
|
@ -126,7 +126,7 @@ Visualize the comparison of scalars, which includes metrics and monitored resour
|
|||||||
|
|
||||||
1. Click the **SCALARS** tab.
|
1. Click the **SCALARS** tab.
|
||||||
1. In the dropdown menu (upper right of the left sidebar), choose either:
|
1. In the dropdown menu (upper right of the left sidebar), choose either:
|
||||||
* **Last values** (the final or most recent value)
|
* **Last Values** (the final or most recent value)
|
||||||
* **Min Values** (the minimal values)
|
* **Min Values** (the minimal values)
|
||||||
* **Max Values** (the maximal values)
|
* **Max Values** (the maximal values)
|
||||||
1. Sort by variant.
|
1. Sort by variant.
|
||||||
|
Loading…
Reference in New Issue
Block a user