mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-22 17:56:07 +00:00
Small edits (#790)
This commit is contained in:
parent
1c45124714
commit
34affd55fb
@ -33,8 +33,8 @@ of the optimization results in table and graph forms.
|
|||||||
|`--params-override`|Additional parameters of the base task to override for this parameter search. Use the following JSON format for each parameter: `{"name": "param_name", "value": <new_value>}`. Windows users, see [JSON format note](#json_note).|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
|`--params-override`|Additional parameters of the base task to override for this parameter search. Use the following JSON format for each parameter: `{"name": "param_name", "value": <new_value>}`. Windows users, see [JSON format note](#json_note).|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
||||||
|`--objective-metric-title`| Objective metric title to maximize/minimize (e.g. 'validation').|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
|`--objective-metric-title`| Objective metric title to maximize/minimize (e.g. 'validation').|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
||||||
|`--objective-metric-series`| Objective metric series to maximize/minimize (e.g. 'loss').|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
|`--objective-metric-series`| Objective metric series to maximize/minimize (e.g. 'loss').|<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
||||||
|`--objective-metric-sign`| Optimization target, whether to maximize or minimize the value of the objective metric specified. Possible values: "min", "max", "min_global", "max_global". See more information [here](#optimization-objective). |<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
|`--objective-metric-sign`| Optimization target, whether to maximize or minimize the value of the objective metric specified. Possible values: "min", "max", "min_global", "max_global". For more information, see [Optimization Objective](#optimization-objective). |<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
||||||
|`--optimizer-class`|The optimizer to use. Possible values are: OptimizerOptuna (default), OptimizerBOHB, GridSearch, RandomSearch. See more information [here](../fundamentals/hpo.md#supported-optimizers). |<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
|`--optimizer-class`|The optimizer to use. Possible values are: OptimizerOptuna (default), OptimizerBOHB, GridSearch, RandomSearch. For more information, see [Supported Optimizers](../fundamentals/hpo.md#supported-optimizers). |<img src="/docs/latest/icons/ico-optional-no.svg" alt="No" className="icon size-md center-md" />|
|
||||||
|`--optimization-time-limit`|The maximum time (minutes) for the optimization to run. The default is `None`, indicating no time limit.|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
|`--optimization-time-limit`|The maximum time (minutes) for the optimization to run. The default is `None`, indicating no time limit.|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
||||||
|`--compute-time-limit`|The maximum compute time in minutes that experiment can consume. If this time limit is exceeded, all jobs are aborted.|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
|`--compute-time-limit`|The maximum compute time in minutes that experiment can consume. If this time limit is exceeded, all jobs are aborted.|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
||||||
|`--pool-period-min`|The time between two consecutive polls (minutes).|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
|`--pool-period-min`|The time between two consecutive polls (minutes).|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|
||||||
|
@ -42,7 +42,7 @@ Datasets** section.
|
|||||||
|
|
||||||
Attach informative metrics or debug samples to the Dataset itself. Use [`Dataset.get_logger()`](../references/sdk/dataset.md#get_logger)
|
Attach informative metrics or debug samples to the Dataset itself. Use [`Dataset.get_logger()`](../references/sdk/dataset.md#get_logger)
|
||||||
to access the dataset's logger object, then add any additional information to the dataset, using the methods
|
to access the dataset's logger object, then add any additional information to the dataset, using the methods
|
||||||
available with a [logger](../references/sdk/logger.md) object.
|
available with a [`Logger`](../references/sdk/logger.md) object.
|
||||||
|
|
||||||
You can add some dataset summaries (like [table reporting](../references/sdk/logger.md#report_table)) to create a preview
|
You can add some dataset summaries (like [table reporting](../references/sdk/logger.md#report_table)) to create a preview
|
||||||
of the data stored for better visibility, or attach any statistics generated by the data ingestion process.
|
of the data stored for better visibility, or attach any statistics generated by the data ingestion process.
|
||||||
|
@ -226,7 +226,7 @@ dataset.remove_files(dataset_path="*.csv", recursive=True)
|
|||||||
|
|
||||||
Add informative metrics, plots, or media to the Dataset. Use [`Dataset.get_logger()`](../references/sdk/dataset.md#get_logger)
|
Add informative metrics, plots, or media to the Dataset. Use [`Dataset.get_logger()`](../references/sdk/dataset.md#get_logger)
|
||||||
to access the dataset's logger object, then add any additional information to the dataset, using the methods
|
to access the dataset's logger object, then add any additional information to the dataset, using the methods
|
||||||
available with a [logger](../references/sdk/logger.md) object.
|
available with a [`Logger`](../references/sdk/logger.md) object.
|
||||||
|
|
||||||
You can add some dataset summaries (like [table reporting](../references/sdk/logger.md#report_table)) to create a preview
|
You can add some dataset summaries (like [table reporting](../references/sdk/logger.md#report_table)) to create a preview
|
||||||
of the data stored for better visibility, or attach any statistics generated by the data ingestion process.
|
of the data stored for better visibility, or attach any statistics generated by the data ingestion process.
|
||||||
@ -261,8 +261,8 @@ Use the `output_url` parameter to specify storage target, such as S3 / GS / Azur
|
|||||||
* Google Cloud Storage: `gs://bucket-name/folder`
|
* Google Cloud Storage: `gs://bucket-name/folder`
|
||||||
* Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
|
* Azure Storage: `azure://<account name>.blob.core.windows.net/path/to/file`
|
||||||
|
|
||||||
By default, the dataset uploads to ClearML's file server. This target storage overrides the `output_uri` value of the
|
By default, the dataset uploads to ClearML's file server. This target storage overrides the `output_uri` value of
|
||||||
[`Dataset.create`](#creating-datasets) method.
|
[`Dataset.create()`](#creating-datasets).
|
||||||
|
|
||||||
ClearML supports parallel uploading of datasets. Use the `max_workers` parameter to specify the number of threads to use
|
ClearML supports parallel uploading of datasets. Use the `max_workers` parameter to specify the number of threads to use
|
||||||
when uploading the dataset. By default, it's the number of your machine's logical cores.
|
when uploading the dataset. By default, it's the number of your machine's logical cores.
|
||||||
@ -288,7 +288,7 @@ The folder changes will be reflected in a new dataset version. This method saves
|
|||||||
update (add / remove) files in a dataset.
|
update (add / remove) files in a dataset.
|
||||||
|
|
||||||
## Deleting Datasets
|
## Deleting Datasets
|
||||||
Delete a dataset using [`Dataset.delete()`](../references/sdk/dataset.md#datasetdelete) method. Input any of the
|
Delete a dataset using the [`Dataset.delete()`](../references/sdk/dataset.md#datasetdelete) class method. Input any of the
|
||||||
attributes of the dataset(s) you want to delete, including ID, project name, version, and/or dataset name. Multiple
|
attributes of the dataset(s) you want to delete, including ID, project name, version, and/or dataset name. Multiple
|
||||||
datasets matching the query will raise an exception, unless you pass `entire_dataset=True` and `force=True`. In this
|
datasets matching the query will raise an exception, unless you pass `entire_dataset=True` and `force=True`. In this
|
||||||
case, all matching datasets will be deleted.
|
case, all matching datasets will be deleted.
|
||||||
|
@ -41,10 +41,10 @@ task = Task.init(
|
|||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
Once a task is created, the task object can be accessed from anywhere in the code by calling [`Task.current_task`](../references/sdk/task.md#taskcurrent_task).
|
Once a task is created, the task object can be accessed from anywhere in the code by calling [`Task.current_task()`](../references/sdk/task.md#taskcurrent_task).
|
||||||
|
|
||||||
If multiple tasks need to be created in the same process (for example, for logging multiple manual runs),
|
If multiple tasks need to be created in the same process (for example, for logging multiple manual runs),
|
||||||
make sure to close a task, before initializing a new one. To close a task simply call [`Task.close`](../references/sdk/task.md#close)
|
make sure to close a task, before initializing a new one. To close a task simply call [`Task.close()`](../references/sdk/task.md#close)
|
||||||
(see example [here](../guides/advanced/multiple_tasks_single_process.md)).
|
(see example [here](../guides/advanced/multiple_tasks_single_process.md)).
|
||||||
|
|
||||||
When initializing a task, its project needs to be specified. If the project entered does not exist, it will be created on-the-fly.
|
When initializing a task, its project needs to be specified. If the project entered does not exist, it will be created on-the-fly.
|
||||||
@ -875,7 +875,7 @@ me = Person('Erik', 5)
|
|||||||
|
|
||||||
params_dictionary = {'epochs': 3, 'lr': 0.4}
|
params_dictionary = {'epochs': 3, 'lr': 0.4}
|
||||||
|
|
||||||
task = Task.init(project_name='examples',task_name='argparser')
|
task = Task.init(project_name='examples',task_name='python objects')
|
||||||
|
|
||||||
task.connect(me)
|
task.connect(me)
|
||||||
task.connect(params_dictionary)
|
task.connect(params_dictionary)
|
||||||
|
@ -78,8 +78,8 @@ and the model inference itself will be executed on the Triton Engine container.
|
|||||||
|
|
||||||
## Advanced Setup - S3/GS/Azure Access (Optional)
|
## Advanced Setup - S3/GS/Azure Access (Optional)
|
||||||
To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage,
|
To add access credentials and allow the inference containers to download models from your S3/GS/Azure object-storage,
|
||||||
add the respective environment variables to your env files (example.env). See further details on configuring the storage
|
add the respective environment variables to your env files (example.env). For further details, see
|
||||||
access [here](../integrations/storage.md#configuring-storage).
|
[Configuring Storage](../integrations/storage.md#configuring-storage).
|
||||||
|
|
||||||
```
|
```
|
||||||
AWS_ACCESS_KEY_ID
|
AWS_ACCESS_KEY_ID
|
||||||
|
12
docs/faq.md
12
docs/faq.md
@ -375,7 +375,7 @@ An experiment's name is a user-controlled property, which can be accessed via th
|
|||||||
|
|
||||||
For example, to distinguish between different experiments, you can append the task ID to the task name:
|
For example, to distinguish between different experiments, you can append the task ID to the task name:
|
||||||
```python
|
```python
|
||||||
task = Task.init('examples', 'train')
|
task = Task.init(project_name='examples', task_name='train')
|
||||||
task.name += ' {}'.format(task.id)
|
task.name += ' {}'.format(task.id)
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -625,8 +625,8 @@ For example, the following two scatter2D series are reported on the same plot, b
|
|||||||
```python
|
```python
|
||||||
scatter2d_1 = np.hstack((np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1))))
|
scatter2d_1 = np.hstack((np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1))))
|
||||||
logger.report_scatter2d(
|
logger.report_scatter2d(
|
||||||
"example_scatter",
|
title="example_scatter",
|
||||||
"series_1",
|
series="series_1",
|
||||||
iteration=1,
|
iteration=1,
|
||||||
scatter=scatter2d_1,
|
scatter=scatter2d_1,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
@ -635,8 +635,8 @@ logger.report_scatter2d(
|
|||||||
|
|
||||||
scatter2d_2 = np.hstack((np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1))))
|
scatter2d_2 = np.hstack((np.atleast_2d(np.arange(0, 10)).T, np.random.randint(10, size=(10, 1))))
|
||||||
logger.report_scatter2d(
|
logger.report_scatter2d(
|
||||||
"example_scatter",
|
title="example_scatter",
|
||||||
"series_2",
|
series="series_2",
|
||||||
iteration=1,
|
iteration=1,
|
||||||
scatter=scatter2d_2,
|
scatter=scatter2d_2,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
@ -783,7 +783,7 @@ Yes! You can run ClearML in Jupyter Notebooks using either of the following:
|
|||||||
1. You can now use ClearML in your notebook!
|
1. You can now use ClearML in your notebook!
|
||||||
```python
|
```python
|
||||||
# create a task and start training
|
# create a task and start training
|
||||||
task = Task.init('jupyter project', 'my notebook')
|
task = Task.init(project_name='jupyter project', task_name='my notebook')
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
@ -74,14 +74,14 @@ When connecting objects to ClearML, users can directly access and modify an obje
|
|||||||
pair in a parameter dictionary).
|
pair in a parameter dictionary).
|
||||||
|
|
||||||
#### Connecting Configuration Objects
|
#### Connecting Configuration Objects
|
||||||
Configuration objects are dictionaries or configuration files connected to the task using the
|
Configuration objects are dictionaries or configuration files connected to the task using
|
||||||
[`Task.connect_configuration`](../references/sdk/task.md#connect_configuration) method. With this method, configuration
|
[`Task.connect_configuration()`](../references/sdk/task.md#connect_configuration). With this method, configuration
|
||||||
objects are saved as blobs i.e. ClearML is not aware of their internal structure.
|
objects are saved as blobs i.e. ClearML is not aware of their internal structure.
|
||||||
|
|
||||||
#### Setting and Updating Parameters
|
#### Setting and Updating Parameters
|
||||||
ClearML provides methods to set and update task parameters manually. Use the [`Task.set_parameters`](../references/sdk/task.md#set_parameters)
|
ClearML provides methods to set and update task parameters manually. Use [`Task.set_parameters()`](../references/sdk/task.md#set_parameters)
|
||||||
method to define parameters manually. To update the parameters in an experiment, use the [`Task.set_parameters_as_dict`](../references/sdk/task.md#set_parameters_as_dict)
|
to define parameters manually. To update the parameters in an experiment, use [`Task.set_parameters_as_dict()`](../references/sdk/task.md#set_parameters_as_dict).
|
||||||
method. The `set_parameters_as_dict` method updates parameters while the `set_parameters` method overrides the parameters.
|
The `set_parameters_as_dict` method updates parameters while the `set_parameters` method overrides the parameters.
|
||||||
|
|
||||||
ClearML does not automatically track changes to explicitly set parameters.
|
ClearML does not automatically track changes to explicitly set parameters.
|
||||||
|
|
||||||
|
@ -37,7 +37,10 @@ In the example script's `train` function, the following code explicitly reports
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
Logger.current_logger().report_scalar(
|
Logger.current_logger().report_scalar(
|
||||||
"train", "loss", iteration=(epoch * len(train_loader) + batch_idx), value=loss.item()
|
title="train",
|
||||||
|
series="loss",
|
||||||
|
iteration=(epoch * len(train_loader) + batch_idx),
|
||||||
|
value=loss.item()
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -45,10 +48,10 @@ In the script's `test` function, the code explicitly reports `loss` and `accurac
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
Logger.current_logger().report_scalar(
|
Logger.current_logger().report_scalar(
|
||||||
"test", "loss", iteration=epoch, value=test_loss
|
title="test", series="loss", iteration=epoch, value=test_loss
|
||||||
)
|
)
|
||||||
Logger.current_logger().report_scalar(
|
Logger.current_logger().report_scalar(
|
||||||
"test", "accuracy", iteration=epoch, value=(correct / len(test_loader.dataset))
|
title="test", series="accuracy", iteration=epoch, value=(correct / len(test_loader.dataset))
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -114,7 +114,7 @@ does the following:
|
|||||||
1. Store the data as an artifact named `dataset` using [`Task.upload_artifact()`](../../references/sdk/task.md#upload_artifact):
|
1. Store the data as an artifact named `dataset` using [`Task.upload_artifact()`](../../references/sdk/task.md#upload_artifact):
|
||||||
```python
|
```python
|
||||||
# add and upload local file containing our toy dataset
|
# add and upload local file containing our toy dataset
|
||||||
task.upload_artifact('dataset', artifact_object=local_iris_pkl)
|
task.upload_artifact(name='dataset', artifact_object=local_iris_pkl)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Step 2 - Processing the Data
|
## Step 2 - Processing the Data
|
||||||
@ -146,10 +146,10 @@ does the following:
|
|||||||
1. Generate testing and training sets from the data and store them as artifacts.
|
1. Generate testing and training sets from the data and store them as artifacts.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
task.upload_artifact('X_train', X_train)
|
task.upload_artifact(name='X_train', artifact_object=X_train)
|
||||||
task.upload_artifact('X_test', X_test)
|
task.upload_artifact(name='X_test', artifact_object=X_test)
|
||||||
task.upload_artifact('y_train', y_train)
|
task.upload_artifact(name='y_train', artifact_object=y_train)
|
||||||
task.upload_artifact('y_test', y_test)
|
task.upload_artifact(name='y_test', artifact_object=y_test)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Step 3 - Training the Network
|
## Step 3 - Training the Network
|
||||||
|
@ -11,15 +11,14 @@ ClearML reports these plots in the experiment's **PLOTS** tab.
|
|||||||
|
|
||||||
## Surface Plot
|
## Surface Plot
|
||||||
|
|
||||||
To plot a series as a surface plot, use the [Logger.report_surface](../../references/sdk/logger.md#report_surface)
|
To plot a series as a surface plot, use [`Logger.report_surface()`](../../references/sdk/logger.md#report_surface):
|
||||||
method.
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# report 3d surface
|
# report 3d surface
|
||||||
surface = np.random.randint(10, size=(10, 10))
|
surface = np.random.randint(10, size=(10, 10))
|
||||||
Logger.current_logger().report_surface(
|
Logger.current_logger().report_surface(
|
||||||
"example_surface",
|
title="example_surface",
|
||||||
"series1",
|
series="series1",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
matrix=surface,
|
matrix=surface,
|
||||||
xaxis="title X",
|
xaxis="title X",
|
||||||
@ -33,14 +32,14 @@ View the reported surface plot in **PLOTS**.
|
|||||||
|
|
||||||
## 3D Scatter Plot
|
## 3D Scatter Plot
|
||||||
|
|
||||||
To plot a series as a 3D scatter plot, use the [Logger.report_scatter3d](../../references/sdk/logger.md#report_scatter3d)
|
To plot a series as a 3D scatter plot, use [`Logger.report_scatter3d()`](../../references/sdk/logger.md#report_scatter3d):
|
||||||
method.
|
|
||||||
```python
|
```python
|
||||||
# report 3d scatter plot
|
# report 3d scatter plot
|
||||||
scatter3d = np.random.randint(10, size=(10, 3))
|
scatter3d = np.random.randint(10, size=(10, 3))
|
||||||
Logger.current_logger().report_scatter3d(
|
Logger.current_logger().report_scatter3d(
|
||||||
"example_scatter_3d",
|
title="example_scatter_3d",
|
||||||
"series_xyz",
|
series="series_xyz",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter3d,
|
scatter=scatter3d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
|
@ -18,7 +18,7 @@ Configure ClearML for uploading artifacts to any of the supported types of stora
|
|||||||
S3 buckets, Google Cloud Storage, and Azure Storage ([debug sample storage](../../references/sdk/logger.md#set_default_upload_destination)
|
S3 buckets, Google Cloud Storage, and Azure Storage ([debug sample storage](../../references/sdk/logger.md#set_default_upload_destination)
|
||||||
is different). Configure ClearML in any of the following ways:
|
is different). Configure ClearML in any of the following ways:
|
||||||
|
|
||||||
* In the configuration file, set [default_output_uri](../../configs/clearml_conf.md#config_default_output_uri).
|
* In the configuration file, set [`default_output_uri`](../../configs/clearml_conf.md#config_default_output_uri).
|
||||||
* In code, when [initializing a Task](../../references/sdk/task.md#taskinit), use the `output_uri` parameter.
|
* In code, when [initializing a Task](../../references/sdk/task.md#taskinit), use the `output_uri` parameter.
|
||||||
* In the **ClearML Web UI**, when [modifying an experiment](../../webapp/webapp_exp_tuning.md#output-destination).
|
* In the **ClearML Web UI**, when [modifying an experiment](../../webapp/webapp_exp_tuning.md#output-destination).
|
||||||
|
|
||||||
@ -26,12 +26,12 @@ When the script runs, it creates an experiment named `artifacts example` in the
|
|||||||
|
|
||||||
ClearML reports artifacts in the **ClearML Web UI** **>** experiment details **>** **ARTIFACTS** tab.
|
ClearML reports artifacts in the **ClearML Web UI** **>** experiment details **>** **ARTIFACTS** tab.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## Dynamically Tracked Artifacts
|
## Dynamically Tracked Artifacts
|
||||||
|
|
||||||
Currently, ClearML supports uploading and dynamically tracking Pandas DataFrames. Use the [Task.register_artifact](../../references/sdk/task.md#register_artifact)
|
ClearML supports uploading and dynamically tracking Pandas DataFrames. Use [`Task.register_artifact()`](../../references/sdk/task.md#register_artifact)
|
||||||
method. If the Pandas DataFrame changes, ClearML uploads the changes. The updated artifact is associated with the experiment.
|
to add a DataFrame to a task. If the DataFrame is modified, ClearML will automatically update the changes.
|
||||||
|
|
||||||
For example:
|
For example:
|
||||||
|
|
||||||
@ -47,11 +47,15 @@ df = pd.DataFrame(
|
|||||||
|
|
||||||
# Register Pandas object as artifact to watch
|
# Register Pandas object as artifact to watch
|
||||||
# (it will be monitored in the background and automatically synced and uploaded)
|
# (it will be monitored in the background and automatically synced and uploaded)
|
||||||
task.register_artifact('train', df, metadata={'counting': 'legs', 'max legs': 69}))
|
task.register_artifact(
|
||||||
|
name='train',
|
||||||
|
artifact=df,
|
||||||
|
metadata={'counting': 'legs', 'max legs': 68}
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
By changing the artifact, and calling the [Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
|
By modifying the artifact, and calling [`Task.get_registered_artifacts()`](../../references/sdk/task.md#get_registered_artifacts)
|
||||||
method to retrieve it, you can see that ClearML tracked the change.
|
to retrieve it, you can see ClearML tracking the changes:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# change the artifact object
|
# change the artifact object
|
||||||
@ -78,7 +82,7 @@ Artifacts without tracking include:
|
|||||||
### Pandas DataFrames
|
### Pandas DataFrames
|
||||||
```python
|
```python
|
||||||
# add and upload pandas.DataFrame (onetime snapshot of the object)
|
# add and upload pandas.DataFrame (onetime snapshot of the object)
|
||||||
task.upload_artifact('Pandas', artifact_object=df)
|
task.upload_artifact(name='Pandas', artifact_object=df)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Local Files
|
### Local Files
|
||||||
@ -86,7 +90,7 @@ task.upload_artifact('Pandas', artifact_object=df)
|
|||||||
```python
|
```python
|
||||||
# add and upload local file artifact
|
# add and upload local file artifact
|
||||||
task.upload_artifact(
|
task.upload_artifact(
|
||||||
'local file',
|
name='local file',
|
||||||
artifact_object=os.path.join(
|
artifact_object=os.path.join(
|
||||||
'data_samples',
|
'data_samples',
|
||||||
'dancing.jpg'
|
'dancing.jpg'
|
||||||
@ -97,31 +101,31 @@ task.upload_artifact(
|
|||||||
### Dictionaries
|
### Dictionaries
|
||||||
```python
|
```python
|
||||||
# add and upload dictionary stored as JSON
|
# add and upload dictionary stored as JSON
|
||||||
task.upload_artifact('dictionary', df.to_dict())
|
task.upload_artifact(name='dictionary', artifact_object=df.to_dict())
|
||||||
```
|
```
|
||||||
|
|
||||||
### Numpy Objects
|
### Numpy Objects
|
||||||
```python
|
```python
|
||||||
# add and upload Numpy Object (stored as .npz file)
|
# add and upload Numpy Object (stored as .npz file)
|
||||||
task.upload_artifact('Numpy Eye', np.eye(100, 100))
|
task.upload_artifact(name='Numpy Eye', artifact_object=np.eye(100, 100))
|
||||||
```
|
```
|
||||||
|
|
||||||
### Image Files
|
### Image Files
|
||||||
```python
|
```python
|
||||||
# add and upload Image (stored as .png file)
|
# add and upload Image (stored as .png file)
|
||||||
im = Image.open(os.path.join('data_samples', 'dancing.jpg'))
|
im = Image.open(os.path.join('data_samples', 'dancing.jpg'))
|
||||||
task.upload_artifact('pillow_image', im)
|
task.upload_artifact(name='pillow_image', artifact_object=im)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Folders
|
### Folders
|
||||||
```python
|
```python
|
||||||
# add and upload a folder, artifact_object should be the folder path
|
# add and upload a folder, artifact_object should be the folder path
|
||||||
task.upload_artifact('local folder', artifact_object=os.path.join('data_samples'))
|
task.upload_artifact(name='local folder', artifact_object=os.path.join('data_samples'))
|
||||||
```
|
```
|
||||||
|
|
||||||
### Wildcards
|
### Wildcards
|
||||||
```python
|
```python
|
||||||
# add and upload a wildcard
|
# add and upload a wildcard
|
||||||
task.upload_artifact('wildcard jpegs', artifact_object=os.path.join('data_samples', '*.jpg'))
|
task.upload_artifact(name='wildcard jpegs', artifact_object=os.path.join('data_samples', '*.jpg'))
|
||||||
```
|
```
|
||||||
|
|
@ -53,8 +53,8 @@ scatter2d = np.hstack(
|
|||||||
)
|
)
|
||||||
# report 2d scatter plot with markers
|
# report 2d scatter plot with markers
|
||||||
logger.report_scatter2d(
|
logger.report_scatter2d(
|
||||||
"example_scatter",
|
title="example_scatter",
|
||||||
"series_lines+markers",
|
series="series_lines+markers",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter2d,
|
scatter=scatter2d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
@ -73,8 +73,8 @@ To plot a series as a 3D scatter plot, use [`Logger.report_scatter3d()`](../../r
|
|||||||
# report 3d scatter plot
|
# report 3d scatter plot
|
||||||
scatter3d = np.random.randint(10, size=(10, 3))
|
scatter3d = np.random.randint(10, size=(10, 3))
|
||||||
logger.report_scatter3d(
|
logger.report_scatter3d(
|
||||||
"example_scatter_3d",
|
title="example_scatter_3d",
|
||||||
"series_xyz",
|
series="series_xyz",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
scatter=scatter3d,
|
scatter=scatter3d,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
@ -91,8 +91,8 @@ To plot a series as a surface plot, use [`Logger.report_surface()`](../../refere
|
|||||||
# report 3d surface
|
# report 3d surface
|
||||||
surface = np.random.randint(10, size=(10, 10))
|
surface = np.random.randint(10, size=(10, 10))
|
||||||
logger.report_surface(
|
logger.report_surface(
|
||||||
"example_surface",
|
title="example_surface",
|
||||||
"series1",
|
series="series1",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
matrix=surface,
|
matrix=surface,
|
||||||
xaxis="title X",
|
xaxis="title X",
|
||||||
@ -111,8 +111,8 @@ Report confusion matrices by calling [`Logger.report_confusion_matrix()`](../../
|
|||||||
# report confusion matrix
|
# report confusion matrix
|
||||||
confusion = np.random.randint(10, size=(10, 10))
|
confusion = np.random.randint(10, size=(10, 10))
|
||||||
logger.report_confusion_matrix(
|
logger.report_confusion_matrix(
|
||||||
"example_confusion",
|
title="example_confusion",
|
||||||
"ignored",
|
series="ignored",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
matrix=confusion,
|
matrix=confusion,
|
||||||
xaxis="title X",
|
xaxis="title X",
|
||||||
@ -131,8 +131,8 @@ To report more than one series on the same plot, use the same `title` argument.
|
|||||||
# report a single histogram
|
# report a single histogram
|
||||||
histogram = np.random.randint(10, size=10)
|
histogram = np.random.randint(10, size=10)
|
||||||
logger.report_histogram(
|
logger.report_histogram(
|
||||||
"single_histogram",
|
title="single_histogram",
|
||||||
"random histogram",
|
series="random histogram",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
values=histogram,
|
values=histogram,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
@ -147,16 +147,16 @@ logger.report_histogram(
|
|||||||
histogram1 = np.random.randint(13, size=10)
|
histogram1 = np.random.randint(13, size=10)
|
||||||
histogram2 = histogram * 0.75
|
histogram2 = histogram * 0.75
|
||||||
logger.report_histogram(
|
logger.report_histogram(
|
||||||
"two_histogram",
|
title="two_histogram",
|
||||||
"series 1",
|
series="series 1",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
values=histogram1,
|
values=histogram1,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
yaxis="title y",
|
yaxis="title y",
|
||||||
)
|
)
|
||||||
logger.report_histogram(
|
logger.report_histogram(
|
||||||
"two_histogram",
|
title="two_histogram",
|
||||||
"series 2",
|
series="series 2",
|
||||||
iteration=iteration,
|
iteration=iteration,
|
||||||
values=histogram2,
|
values=histogram2,
|
||||||
xaxis="title x",
|
xaxis="title x",
|
||||||
@ -185,7 +185,7 @@ image_local_copy = StorageManager.get_local_copy(
|
|||||||
### Audio
|
### Audio
|
||||||
|
|
||||||
```python
|
```python
|
||||||
logger.report_media('audio', 'pink panther', iteration=1, local_path=audio_local_copy)
|
logger.report_media(title='audio', series='pink panther', iteration=1, local_path=audio_local_copy)
|
||||||
```
|
```
|
||||||
|
|
||||||

|

|
||||||
@ -194,8 +194,8 @@ logger.report_media('audio', 'pink panther', iteration=1, local_path=audio_local
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
logger.report_media(
|
logger.report_media(
|
||||||
"html",
|
title="html",
|
||||||
"url_html",
|
series="url_html",
|
||||||
iteration=1,
|
iteration=1,
|
||||||
url="https://clear.ml/docs/latest/docs/index.html"
|
url="https://clear.ml/docs/latest/docs/index.html"
|
||||||
)
|
)
|
||||||
@ -206,7 +206,12 @@ logger.report_media(
|
|||||||
### Images
|
### Images
|
||||||
|
|
||||||
```python
|
```python
|
||||||
logger.report_image("image", "image from url", iteration=100, local_path=image_local_copy)
|
logger.report_image(
|
||||||
|
title="image",
|
||||||
|
series="image from url",
|
||||||
|
iteration=100,
|
||||||
|
local_path=image_local_copy
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||

|

|
||||||
@ -214,7 +219,12 @@ logger.report_image("image", "image from url", iteration=100, local_path=image_l
|
|||||||
### Video
|
### Video
|
||||||
|
|
||||||
```python
|
```python
|
||||||
logger.report_media('video', 'big bunny', iteration=1, local_path=video_local_copy)
|
logger.report_media(
|
||||||
|
title='video',
|
||||||
|
series='big bunny',
|
||||||
|
iteration=1,
|
||||||
|
local_path=video_local_copy
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||

|

|
||||||
|
@ -50,7 +50,9 @@ Use the `local_path` parameter.
|
|||||||
```python
|
```python
|
||||||
# report audio, report local media audio file
|
# report audio, report local media audio file
|
||||||
Logger.current_logger().report_media(
|
Logger.current_logger().report_media(
|
||||||
'audio', 'tada', iteration=1,
|
title='audio',
|
||||||
|
series='tada',
|
||||||
|
iteration=1,
|
||||||
local_path=os.path.join('data_samples', 'sample.mp3')
|
local_path=os.path.join('data_samples', 'sample.mp3')
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
@ -18,11 +18,17 @@ from clearml import Task
|
|||||||
task = Task.init(task_name="<task_name>", project_name="<project_name>")
|
task = Task.init(task_name="<task_name>", project_name="<project_name>")
|
||||||
```
|
```
|
||||||
|
|
||||||
This will create a ClearML Task that captures your script's information, including Git details, uncommitted code, python
|
This will create a ClearML Task that captures:
|
||||||
environment, your matplotlib visualizations, and more. View these matplotlib plots and images in the [WebApp](../webapp/webapp_exp_track_visual.md),
|
* Git details
|
||||||
|
* Source code and uncommitted changes
|
||||||
|
* Installed packages
|
||||||
|
* Matplotlib visualizations
|
||||||
|
* And more
|
||||||
|
|
||||||
|
View captured Matplotlib plots and images in the [WebApp](../webapp/webapp_exp_track_visual.md),
|
||||||
in the experiment's **Plots** and **Debug Samples** tabs respectively.
|
in the experiment's **Plots** and **Debug Samples** tabs respectively.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## Automatic Logging Control
|
## Automatic Logging Control
|
||||||
By default, when ClearML is integrated into your script, it captures all of your matplotlib visualizations.
|
By default, when ClearML is integrated into your script, it captures all of your matplotlib visualizations.
|
||||||
|
@ -220,7 +220,7 @@ Click <img src="/docs/latest/icons/ico-settings.svg" alt="Setting Gear" classNam
|
|||||||
metrics to view.
|
metrics to view.
|
||||||
|
|
||||||
In the **Scalars** view, click <img src="/docs/latest/icons/ico-tune.svg" alt="Tune" className="icon size-md" /> to access
|
In the **Scalars** view, click <img src="/docs/latest/icons/ico-tune.svg" alt="Tune" className="icon size-md" /> to access
|
||||||
scalar plot tools (see [here](webapp_exp_track_visual.md#scalar-plot-tools)).
|
[scalar plot tools](webapp_exp_track_visual.md#scalar-plot-tools).
|
||||||
|
|
||||||
|
|
||||||
## Creating an Experiment Leaderboard
|
## Creating an Experiment Leaderboard
|
||||||
|
Loading…
Reference in New Issue
Block a user