mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-03 02:32:49 +00:00
Rewrite fundamentals sections (#252)
This commit is contained in:
pollfly
GitHub
parent
a9b5bd9697
commit
1a46cbdb93
@ -1,29 +0,0 @@
|
||||
---
|
||||
title: ClearML SDK
|
||||
---
|
||||
|
||||
The **ClearML Python Package** supports the [automatic logging](fundamentals/logger.md#automatic-reporting)
|
||||
that documents the experiment for you, and an extensive set of powerful features and functionality you can use to improve experimentation, other workflows, and get more out of ClearML.
|
||||
|
||||
The **ClearML Python Package** collects data from scripts including the Git repository (branch, commit ID, and uncommitted changes), working directory and entry point, hyperparameters, initial weights model, model snapshots (checkpoints), output model, other artifacts, metrics, logs, other reported data (from libraries and visualization toolkits), and debug samples.
|
||||
|
||||
In conjunction with the **ClearML Hosted Service** (or self-hosted **ClearML Server**) and **ClearML Agent**, the **ClearML Python Package** allows you and your teammates to collaborate programmatically and using the **ClearML Web UI**.
|
||||
|
||||
## Modules
|
||||
|
||||
* [Task](references/sdk/task.md) - The `task` module contains the `task.Task` class which is the code template for all `Task` features and functionality, including collecting data from scripts, storing that data in a `Task` object, automatic bindings with frameworks (TensorFlow/TensorBoard, PyTorch, Keras, Fastai, scikit-learn), libraries (Pandas, Plotly, AutoKeras), and visualization tools (Matplotlib, Seaborn), and a robust set of methods for Task execution, cloning, connecting parameter dictionaries, configurations, models, working with storage, and more.
|
||||
|
||||
* [Logger](references/sdk/logger.md) - The `logger` module contains the `logger.Logger` class which is the ClearML console log and metric statistics interface, and contains methods for explicit reporting, setting an upload destination in storage for debug samples, logger cache control, and TensorBoard support in addition to ClearML automatic TensorBoard logging.
|
||||
|
||||
* [Model](references/sdk/model_model.md) - The `model` module contains three classes: `model.Model` which represents an existing model in ClearML that can be loaded and connected to a Task, `model.InputModel` which represents an existing model that you can load into ClearML, and `model.OutputModel` which represents the experiment output model that is always connected to the Task.
|
||||
|
||||
* [Automation](references/sdk/automation_controller_pipelinecontroller.md) - The `automation` module contains classes supporting hyperparameter optimization, including Optuna, HpBandSter, grid searching, random searching, you own customized search strategies, and resource budgeting for searches; the AWS autoscaler; pipeline controllers; and Task monitoring.
|
||||
|
||||
* [StorageManager](references/sdk/storage.md) - The `storage` module contains the `storage.manager.StorageManager` class which provides support for downloading and uploading from storage, including folders, S3, Google Cloud Storage, Azure Storage, and http(s).
|
||||
|
||||
* [Dataset](references/sdk/dataset.md) - The `dataset` module contains classes that helps manage Dataset. Users can create, modify and delete datasets as well as retrieve them for use in their code
|
||||
|
||||
## Examples
|
||||
|
||||
ClearML example scripts in the [examples folder](https://github.com/allegroai/clearml/tree/master/examples) of the `clearml` GitHub repository. They are pre-loaded in the **ClearML Hosted Service**, and can be viewed, cloned, and edited in the **ClearML Web UI**, `ClearML Examples` project. The examples are each explained in this documentation's [examples section](guides/main.md).
|
||||
|
||||
113
docs/clearml_sdk/clearml_sdk.md
Normal file
113
docs/clearml_sdk/clearml_sdk.md
Normal file
@ -0,0 +1,113 @@
|
||||
---
|
||||
title: Overview
|
||||
---
|
||||
|
||||
The ClearML Python Package supports the [automatic logging](../fundamentals/logger.md#automatic-reporting) that documents
|
||||
experiments for you, and an extensive set of powerful features and functionality you can use to improve experimentation
|
||||
and other workflows.
|
||||
|
||||
:::tip Installation
|
||||
For installation instructions, see [Getting Started](../getting_started/ds/ds_first_steps.md#install-clearml).
|
||||
:::
|
||||
|
||||
The ClearML Python Package collects the scripts’ entire execution information, including:
|
||||
* Git repository (branch, commit ID, and uncommitted changes)
|
||||
* Working directory and entry point
|
||||
* Hyperparameters
|
||||
* Initial weights model, model snapshots (checkpoints), output model
|
||||
* Artifacts, metrics, logs, other reported data (from libraries and visualization toolkits), and debug samples.
|
||||
|
||||
In conjunction with the ClearML Hosted Service (or self-hosted [ClearML Server](../deploying_clearml/clearml_server.md))
|
||||
and [ClearML Agent](../clearml_agent.md), the ClearML Python Package allows you and your teammates to collaborate
|
||||
programmatically and use the [ClearML Web UI](../webapp/webapp_overview.md).
|
||||
|
||||
## Classes and Modules
|
||||
|
||||
### Task
|
||||
The `Task` class is the code template for all Task features and functionality including:
|
||||
* Data collection and storage from scripts
|
||||
* Automatic bindings with frameworks, libraries, and visualization tools
|
||||
* A robust set of methods for Task execution (cloning, connecting parameter dictionaries, configurations, and models)
|
||||
* and more!
|
||||
|
||||
See an [overview](task_sdk.md) of `Task`'s pythonic methods or the [Task SDK reference page](task_sdk.md).
|
||||
|
||||
### Model
|
||||
The `model` module contains three classes that provide support for working with models in ClearML:
|
||||
* `Model` - represents an existing model in ClearML that can be loaded and connected to a Task
|
||||
* `InputModel` - represents an existing model that you can load into ClearML
|
||||
* `OutputModel` - represents the experiment output model that is always connected to the Task
|
||||
|
||||
See an [overview](../clearml_sdk/model_sdk.md) of the Model classes' pythonic methods, or the SDK reference pages for [`Model`](../references/sdk/model_model.md),
|
||||
[`InputModel`](../references/sdk/model_inputmodel.md), and [`OutputModel`](../references/sdk/model_outputmodel.md).
|
||||
|
||||
### Logger
|
||||
The `Logger` class is the ClearML console log and metric statistics interface. The class contains methods for:
|
||||
* Explicit reporting
|
||||
* Setting an upload destination for debug sample storage
|
||||
* Controlling ClearML's logging of TensorBoard and Matplotlib outputs
|
||||
|
||||
See the [Logger SDK reference page](../references/sdk/logger.md).
|
||||
|
||||
|
||||
### Hyperparameter Optimization
|
||||
|
||||
ClearML's `optimization` module includes classes that support hyperparameter optimization:
|
||||
* [HyperParameterOptimizer](../references/sdk/automation_controller_pipelinecontroller.md) - Hyperparameter search
|
||||
controller class
|
||||
* Optimization search strategy classes including [Optuna](../references/sdk/hpo_optuna_optuna_optimizeroptuna.md), [HpBandSter](../references/sdk/hpo_hpbandster_bandster_optimizerbohb.md),
|
||||
[GridSearch](../references/sdk/hpo_optimization_gridsearch.md), [RandomSearch](../references/sdk/hpo_optimization_randomsearch.md),
|
||||
and a base [SearchStrategy](https://github.com/allegroai/clearml/blob/master/clearml/automation/optimization.py#L310)
|
||||
that can be customized
|
||||
|
||||
See the [HyperParameterOptimizer SDK reference page](../references/sdk/hpo_optimization_hyperparameteroptimizer.md).
|
||||
|
||||
### Pipeline
|
||||
|
||||
ClearML's `automation` module includes classes that support creating pipelines:
|
||||
* [PipelineController](../pipelines/pipelines_sdk_tasks.md) - A pythonic interface for
|
||||
defining and configuring a pipeline controller and its steps. The controller and steps can be functions in your
|
||||
python code, or existing ClearML [tasks](../fundamentals/task.md).
|
||||
* [PipelineDecorator](../pipelines/pipelines_sdk_function_decorators.md) - A set
|
||||
of Python decorators which transform your functions into the pipeline controller and steps.
|
||||
|
||||
### Dataset
|
||||
The `Dataset` class supports creating, modifying, and managing datasets,
|
||||
as well as retrieving them for use in code.
|
||||
|
||||
See [ClearML Data](../clearml_data/clearml_data.md) or the [Dataset SDK reference page](../references/sdk/dataset.md).
|
||||
|
||||
|
||||
### StorageManager
|
||||
The `StorageManager` class provides support for downloading and uploading from storage,
|
||||
including local folders, S3, Google Cloud Storage, Azure Storage, and http(s).
|
||||
|
||||
See the [StorageManager SDK reference page](../references/sdk/storage.md).
|
||||
|
||||
### ClearmlJob
|
||||
Use the ClearmlJob to create and manage jobs based on existing tasks. The class supports changing a job's parameters,
|
||||
configurations, and other execution details.
|
||||
|
||||
See reference page [here](../references/sdk/automation_job_clearmljob.md).
|
||||
|
||||
### AutoScaler
|
||||
The `AutoScaler` class facilitates implementing resource budgeting. See class methods [here](https://github.com/allegroai/clearml/blob/master/clearml/automation/auto_scaler.py).
|
||||
ClearML also provides a class specifically for AWS autoscaling. See [code](https://github.com/allegroai/clearml/blob/master/clearml/automation/aws_auto_scaler.py#L22)
|
||||
and [example script](https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py).
|
||||
|
||||
### TaskScheduler
|
||||
The `TaskScheduler` class supports methods for scheduling periodic execution (like cron jobs). See the [code](https://github.com/allegroai/clearml/blob/master/clearml/automation/scheduler.py#L481)
|
||||
and [example](https://github.com/allegroai/clearml/blob/master/examples/scheduler/cron_example.py).
|
||||
|
||||
### TriggerScheduler
|
||||
The `TriggerScheduler` class facilitates triggering task execution in the case that specific events occur in the system
|
||||
(e.g. model publication, dataset creation, task failure). See [code](https://github.com/allegroai/clearml/blob/master/clearml/automation/trigger.py#L148)
|
||||
and [usage example](https://github.com/allegroai/clearml/blob/master/examples/scheduler/trigger_example.py).
|
||||
|
||||
## Examples
|
||||
|
||||
The `clearml` GitHub repository includes an [examples folder](https://github.com/allegroai/clearml/tree/master/examples)
|
||||
with example scripts demonstrating how to use the various functionalities of the ClearML SDK.
|
||||
|
||||
These examples are pre-loaded in the [ClearML Hosted Service](https://app.clear.ml), and can be viewed, cloned,
|
||||
and edited in the ClearML Web UI's `ClearML Examples` project. The examples are each explained in the [examples section](../guides/main.md).
|
||||
133
docs/clearml_sdk/model_sdk.md
Normal file
133
docs/clearml_sdk/model_sdk.md
Normal file
@ -0,0 +1,133 @@
|
||||
---
|
||||
title: Model
|
||||
---
|
||||
|
||||
The following page provides an overview of the basic Pythonic interface to ClearML Models.
|
||||
|
||||
ClearML provides the following classes to work with models:
|
||||
* `Model` - represents an experiment's output model (training results). An OutputModel is always connected to a [task](../fundamentals/task.md).
|
||||
* `InputModel` - represents an existing ClearML model to be used in an experiment.
|
||||
* `OutputModel` - represents a ClearML model, regardless of any task connection.
|
||||
|
||||
## Output Models
|
||||
|
||||
### Manually Logging Models
|
||||
|
||||
To manually log a model, create an instance of OutputModel class.
|
||||
|
||||
```python
|
||||
from clearml import OutputModel, Task
|
||||
|
||||
# Instantiate a Task
|
||||
task = Task.init(project_name="myProject", task_name="myTask")
|
||||
|
||||
# Create an output model for the PyTorch framework
|
||||
output_model = OutputModel(task=task, framework="PyTorch")
|
||||
```
|
||||
|
||||
You can set the destination the model will be uploaded to and its label enumeration using the
|
||||
[`OutputModel.set_upload_destination`](../references/sdk/model_outputmodel.md#set_upload_destination) and
|
||||
[`OutputModel.update_labels`](../references/sdk/model_outputmodel.md#update_labels) methods respectively.
|
||||
|
||||
```python
|
||||
# Set the URI of the storage destination for uploaded model weight files
|
||||
output_model.set_upload_destination(uri=models_upload_destination)
|
||||
|
||||
# Set the label numeration
|
||||
output_model.update_labels({'background': 0, 'label': 255})
|
||||
```
|
||||
|
||||
### Updating Models
|
||||
ClearML doesn’t automatically log the snapshots of manually logged models. To update an experiment’s model use the
|
||||
[OutputModel.update_weights](../references/sdk/model_outputmodel.md#update_weights) method.
|
||||
|
||||
```python
|
||||
# If validation shows this network is the best so far, update the output model
|
||||
if val_log['iou'] > best_iou:
|
||||
output_model.update_weights(weights_filename='models/model.pth')
|
||||
```
|
||||
|
||||
* Specify either the path of a local weights file to upload (`weights_filename`), or the network location of a remote
|
||||
weights file (`registered_uri`).
|
||||
* Use the `upload_uri` argument to explicitly specify an upload destination for the weights file.
|
||||
* Model metadata
|
||||
* `update_comment` - update the model's description
|
||||
* `iteration` - input the iteration number
|
||||
|
||||
Alternatively, update a model through its associated task, using the [`Task.update_output_model`](../references/sdk/task.md#update_output_model)
|
||||
method.
|
||||
|
||||
## Input Models
|
||||
|
||||
### Using Registered Models
|
||||
|
||||
To use a ClearML model as an input model, create an InputModel object and [connect](../references/sdk/task.md#connect)
|
||||
it to a task.
|
||||
|
||||
```python
|
||||
# Create an input model using the ClearML ID of a model already registered in the ClearML platform
|
||||
input_model = InputModel(model_id="fd8b402e874549d6944eebd49e37eb7b")
|
||||
|
||||
# Connect the input model to the task
|
||||
task.connect(input_model)
|
||||
```
|
||||
|
||||
### Importing Models
|
||||
|
||||
To import an existing model, use the [`InputModel.import_model`](../references/sdk/model_outputmodel.md#inputmodelimport_model)
|
||||
class method and specify the `weights_url` - the URL for the imported model. If the URL already exists in the ClearML
|
||||
server, it is reused. Otherwise, a new model is registered.
|
||||
|
||||
Then [connect](../references/sdk/task.md#connect) the model to a task.
|
||||
|
||||
```python
|
||||
# Instantiate a Task
|
||||
task = Task.init(project_name="examples", task_name="example task")
|
||||
|
||||
input_model = InputModel.import_model(
|
||||
# Name for model in ClearML
|
||||
name='Input Model with Network Design',
|
||||
# Import the model using a URL
|
||||
weights_url='https://s3/models/model.pth',
|
||||
# Set label enumeration values
|
||||
label_enumeration={'person' : 1, 'car' : 2, 'truck' : 3, 'bus' : 4,
|
||||
'motorcycle' : 5, 'bicycle' : 6, 'ignore': -1},
|
||||
framework='PyTorch'
|
||||
)
|
||||
|
||||
# Connect the input model to the task
|
||||
task.connect(input_model)
|
||||
```
|
||||
|
||||
## Querying Models
|
||||
Retrieve a list of model objects by querying the system by model names, projects, tags, and more, using the
|
||||
[`Model.query_models`](../references/sdk/model_model.md#modelquery_models) and / or
|
||||
the [`InputModel.query_models`](../references/sdk/model_inputmodel.md#inputmodelquery_models) class methods. These
|
||||
methods return a list of model objects that match the queries. The list is ordered according to the models’ last update
|
||||
time.
|
||||
|
||||
When you query models by tags, use the `-` prefix in order to filter out models with that tag.
|
||||
|
||||
```python
|
||||
model_list = Model.query_models(
|
||||
# Only models from `examples` project
|
||||
project_name='examples',
|
||||
# Only models with input name
|
||||
model_name=None,
|
||||
# Only models with `demo` tag but without `TF` tag
|
||||
tags=['demo', '-TF'],
|
||||
# If `True`, only published models
|
||||
only_published=False,
|
||||
# If `True`, include archived models
|
||||
include_archived=True,
|
||||
# Maximum number of models returned
|
||||
max_results=5
|
||||
)
|
||||
```
|
||||
|
||||
## SDK Reference
|
||||
|
||||
For information about all model methods, see the following SDK reference pages:
|
||||
* [Model](../references/sdk/model_model.md)
|
||||
* [InputModel](../references/sdk/model_inputmodel.md)
|
||||
* [OutputModel](../references/sdk/model_outputmodel.md)
|
||||
589
docs/clearml_sdk/task_sdk.md
Normal file
589
docs/clearml_sdk/task_sdk.md
Normal file
@ -0,0 +1,589 @@
|
||||
---
|
||||
title: Task
|
||||
---
|
||||
|
||||
The following page provides an overview of the basic Pythonic interface to ClearML Tasks.
|
||||
|
||||
## Task Creation
|
||||
|
||||
[`Task.init`](../references/sdk/task.md) is the main method used to create tasks in ClearML. It will create a task, and
|
||||
populate it with:
|
||||
* A link to the running git repository (including commit ID and local uncommitted changes)
|
||||
* Python packages used (i.e. directly imported Python packages, and the versions available on the machine)
|
||||
* Argparse arguments (default and specific to the current execution)
|
||||
* Reports to Tensorboard & Matplotlib and model checkpoints.
|
||||
|
||||
:::note
|
||||
ClearML object (e.g. task, project) names are required to be at least 3 characters long
|
||||
:::
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
|
||||
|
||||
task = Task.init(
|
||||
project_name='example', # project name of at least 3 characters
|
||||
task_name='task template', # task name of at least 3 characters
|
||||
task_type=None,
|
||||
tags=None,
|
||||
reuse_last_task_id=True,
|
||||
continue_last_task=False,
|
||||
output_uri=None,
|
||||
auto_connect_arg_parser=True,
|
||||
auto_connect_frameworks=True,
|
||||
auto_resource_monitoring=True,
|
||||
auto_connect_streams=True,
|
||||
)
|
||||
```
|
||||
|
||||
Once a task is created, the task object can be accessed from anywhere in the code by calling [`Task.current_task`](../references/sdk/task.md#taskcurrent_task).
|
||||
|
||||
If multiple tasks need to be created in the same process (for example, for logging multiple manual runs),
|
||||
make sure to close a task, before initializing a new one. To close a task simply call [`Task.close`](../references/sdk/task.md#close)
|
||||
(see example [here](../guides/advanced/multiple_tasks_single_process.md)).
|
||||
|
||||
When initializing a task, its project needs to be specified. If the project entered does not exist, it will be created on-the-fly.
|
||||
Projects can be divided into sub-projects, just like folders are broken into sub-folders.
|
||||
|
||||
For example:
|
||||
```python
|
||||
Task.init(project_name='main_project/sub_project', task_name='test')
|
||||
```
|
||||
|
||||
Nesting projects works on multiple levels. For example: `project_name=main_project/sub_project/sub_sub_project`
|
||||
|
||||
### Automatic Logging
|
||||
After invoking `Task.init` in a script, ClearML starts its automagical logging, which includes the following elements:
|
||||
* **Hyperparameters** - ClearML logs the following types of hyperparameters:
|
||||
* Command Line Parsing - ClearML captures any command line parameters passed when invoking code that uses standard python packages, including:
|
||||
* [click](https://click.palletsprojects.com) (see code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/click/click_multi_cmd.py)).
|
||||
* argparse (see argparse logging example [here](../guides/reporting/hyper_parameters.md).)
|
||||
* [Python Fire](https://github.com/google/python-fire) - see code examples [here](https://github.com/allegroai/clearml/tree/master/examples/frameworks/fire).
|
||||
* [LightningCLI](https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_cli.html) - see code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/jsonargparse/pytorch_lightning_cli.py).
|
||||
* TensorFlow Definitions (`absl-py`)
|
||||
* [Hydra](https://github.com/facebookresearch/hydra) - the Omegaconf which holds all the configuration files, as well as overridden values.
|
||||
* **Models** - ClearML automatically logs and updates the models and all snapshot paths saved with the following frameworks:
|
||||
* Tensorflow (see [code example](../guides/frameworks/tensorflow/tensorflow_mnist.md))
|
||||
* Keras (see [code example](../guides/frameworks/keras/keras_tensorboard.md))
|
||||
* Pytorch (see [code example](../guides/frameworks/pytorch/pytorch_mnist.md))
|
||||
* scikit-learn (only using joblib) (see [code example](../guides/frameworks/scikit-learn/sklearn_joblib_example.md))
|
||||
* XGBoost (only using joblib) (see [code example](../guides/frameworks/xgboost/xgboost_sample.md))
|
||||
* FastAI (see [code example](../guides/frameworks/fastai/fastai_with_tensorboard.md))
|
||||
* MegEngine (see [code example](../guides/frameworks/megengine/megengine_mnist.md))
|
||||
* CatBoost (see [code example](../guides/frameworks/catboost/catboost.md))
|
||||
|
||||
* **Metrics, scalars, plots, debug images** reported through supported frameworks, including:
|
||||
* Matplotlib (see [code example](../guides/frameworks/matplotlib/matplotlib_example.md))
|
||||
* Tensorboard (see [code example](../guides/frameworks/pytorch/pytorch_tensorboardx.md))
|
||||
* TensorboardX (see [code example](../guides/frameworks/tensorboardx/tensorboardx.md))
|
||||
|
||||
* **Execution details** including:
|
||||
* Git information
|
||||
* Uncommitted code modifications - In cases where no git repository is detected (e.g. when a single python script is
|
||||
executed outside a git repository, or when running from a Jupyter Notebook), ClearML logs the contents
|
||||
of the executed script
|
||||
* Python environment
|
||||
* Execution [configuration](../webapp/webapp_exp_track_visual.md#configuration)
|
||||
|
||||
To control a task's framework logging, use the `auto_connect_framworks` parameter of the [`Task.init`](../references/sdk/task.md#taskinit)
|
||||
method. Turn off all automatic logging by setting the parameter to `False`. For finer grained control of logged frameworks,
|
||||
input a dictionary, with framework-boolean pairs.
|
||||
|
||||
For example:
|
||||
```python
|
||||
auto_connect_frameworks={
|
||||
'matplotlib': True, 'tensorflow': False, 'tensorboard': False, 'pytorch': True,
|
||||
'xgboost': False, 'scikit': True, 'fastai': True, 'lightgbm': False,
|
||||
'hydra': True, 'detect_repository': True, 'tfdefines': True, 'joblib': True,
|
||||
'megengine': True, 'jsonargparse': True, 'catboost': True
|
||||
}
|
||||
```
|
||||
|
||||
### Task Reuse
|
||||
Every `Task.init` call will create a new task for the current execution.
|
||||
In order to mitigate the clutter that a multitude of debugging tasks might create, a task will be reused if:
|
||||
* The last time it was executed (on this machine) was under 72 hours ago (configurable, see
|
||||
`sdk.development.task_reuse_time_window_in_hours` in the [`sdk.development` section](../configs/clearml_conf.md#sdkdevelopment) of
|
||||
the ClearML configuration reference)
|
||||
* The previous task execution did not have any artifacts / models
|
||||
|
||||
It's possible to always create a new task by passing `reuse_last_task_id=False`.
|
||||
|
||||
See full `Task.init` reference [here](../references/sdk/task.md#taskinit).
|
||||
|
||||
### Empty Task Creation
|
||||
|
||||
A task can also be created without the need to execute the code itself.
|
||||
Unlike the runtime detections, all the environment and configuration details need to be provided explicitly.
|
||||
|
||||
For example:
|
||||
```python
|
||||
task = Task.create(
|
||||
project_name='example',
|
||||
task_name='task template',
|
||||
repo='https://github.com/allegroai/clearml.git',
|
||||
branch='master',
|
||||
script='examples/reporting/html_reporting.py',
|
||||
working_directory='.',
|
||||
docker=None,
|
||||
)
|
||||
```
|
||||
|
||||
See full `Task.create` reference [here](../references/sdk/task.md#taskcreate).
|
||||
|
||||
### Tracking Task Progress
|
||||
Track a task’s progress by setting the task progress property using the [`Task.set_progress`](../references/sdk/task.md#set_progress) method.
|
||||
Set a task’s progress to a numeric value between 0 - 100. Access the task’s current progress, using the
|
||||
[`Task.get_progress`](../references/sdk/task.md#get_progress) method.
|
||||
|
||||
```python
|
||||
task = Task.init(project_name="examples", task_name="Track experiment progress")
|
||||
task.set_progress(0)
|
||||
# task doing stuff
|
||||
task.set_progress(50)
|
||||
print(task.get_progress())
|
||||
# task doing more stuff
|
||||
task.set_progress(100)
|
||||
```
|
||||
|
||||
While the task is running, the WebApp will show the task’s progress indication in the experiment table, next to the
|
||||
task’s status. If a task failed or was aborted, you can view how much progress it had made.
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
Additionally, you can view a task’s progress in its [INFO](../webapp/webapp_exp_track_visual.md#general-information) tab
|
||||
in the WebApp.
|
||||
|
||||
|
||||
## Accessing Tasks
|
||||
A task can be identified by its project and name, and by a unique identifier (UUID string). The name and project of
|
||||
a task can be changed after an experiment has been executed, but its ID can't be changed.
|
||||
|
||||
Programmatically, task objects can be retrieved by querying the system based on either the task ID or a project and name
|
||||
combination using the [`Task.get_task`](../references/sdk/task.md#taskget_task) class method. If a project / name
|
||||
combination is used, and multiple tasks have the exact same name, the function will return the *last modified task*.
|
||||
|
||||
For example:
|
||||
* Accessing a task object with a task ID:
|
||||
|
||||
```python
|
||||
a_task = Task.get_task(task_id='123456deadbeef')
|
||||
```
|
||||
* Accessing a task with a project and name:
|
||||
|
||||
```python
|
||||
a_task = Task.get_task(project_name='examples', task_name='artifacts')
|
||||
```
|
||||
|
||||
Once a task object is obtained, it's possible to query the state of the task, reported scalars, etc.
|
||||
The task's outputs, such as artifacts and models, can also be retrieved.
|
||||
|
||||
## Querying / Searching Tasks
|
||||
|
||||
Searching and filtering tasks can be done via the [web UI](../webapp/webapp_overview.md) and programmatically.
|
||||
Input search parameters into the [`Task.get_tasks`](../references/sdk/task.md#taskget_tasks) method, which returns a
|
||||
list of task objects that match the search.
|
||||
|
||||
For example:
|
||||
```python
|
||||
task_list = Task.get_tasks(
|
||||
task_ids=None, # type Optional[Sequence[str]]
|
||||
project_name=None, # Optional[str]
|
||||
task_name=None, # Optional[str]
|
||||
task_filter=None # Optional[Dict]
|
||||
)
|
||||
```
|
||||
|
||||
It's possible to also filter tasks by passing filtering rules to `task_filter`.
|
||||
|
||||
For example:
|
||||
```python
|
||||
task_filter={
|
||||
# only tasks with tag `included_tag` and without tag `excluded_tag`
|
||||
'tags': ['included_tag', '-excluded_tag'],
|
||||
# filter out archived tasks
|
||||
'system_tags': ['-archived'],
|
||||
# only completed & published tasks
|
||||
'status': ['completed', 'published'],
|
||||
# only training type tasks
|
||||
'type': ['training'],
|
||||
# match text in task comment or task name
|
||||
'search_text': 'reg_exp_text'
|
||||
}
|
||||
```
|
||||
|
||||
## Cloning & Executing Tasks
|
||||
|
||||
Once a task object is created, it can be copied (cloned). [`Task.clone`](../references/sdk/task.md#taskclone) returns
|
||||
a copy of the original task (`source_task`). By default, the cloned task is added to the same project as the original,
|
||||
and it's called "Clone Of ORIGINAL_NAME", but the name / project / comment (description) of the cloned task can be directly overridden.
|
||||
|
||||
```python
|
||||
task = Task.init(project_name='examples', task_name='original task',)
|
||||
cloned = Task.clone(
|
||||
source_task=task, # type: Optional[Union[Task, str]]
|
||||
# override default name
|
||||
name='newly created task', # type: Optional[str]
|
||||
comment=None, # type: Optional[str]
|
||||
# insert cloned task into a different project
|
||||
project=None, # type: Optional[str]
|
||||
)
|
||||
```
|
||||
|
||||
A newly cloned task has a [draft](../fundamentals/task.md#task-states) status, so it's modifiable.
|
||||
|
||||
Once a task is modified, launch it by pushing it into an execution queue with the [Task.enqueue](../references/sdk/task.md#taskenqueue)
|
||||
class method. Then a [ClearML Agent](../clearml_agent.md) assigned to the queue will pull the task from the queue and execute
|
||||
it.
|
||||
|
||||
```python
|
||||
Task.enqueue(
|
||||
task=task, # type: Union[Task, str]
|
||||
queue_name='default', # type: Optional[str]
|
||||
queue_id=None # type: Optional[str]
|
||||
)
|
||||
```
|
||||
|
||||
See enqueue [example](https://github.com/allegroai/clearml/blob/master/examples/automation/programmatic_orchestration.py).
|
||||
|
||||
## Advanced Flows
|
||||
|
||||
### Remote Execution
|
||||
|
||||
A compelling workflow is:
|
||||
1. Run code on a development machine for a few iterations, or just set up the environment.
|
||||
1. Move the execution to a beefier remote machine for the actual training.
|
||||
|
||||
Use the [Task.execute_remotely](../references/sdk/task.md#execute_remotely) method to implement this workflow. This method
|
||||
stops the current manual execution, and then re-runs it on a remote machine.
|
||||
|
||||
For example:
|
||||
```python
|
||||
task.execute_remotely(
|
||||
queue_name='default', # type: Optional[str]
|
||||
clone=False, # type: bool
|
||||
exit_process=True # type: bool
|
||||
)
|
||||
```
|
||||
|
||||
Once the method is called on the machine, it stops the local process and enqueues the current task into the `default`
|
||||
queue. From there, an agent can pull and launch it.
|
||||
|
||||
See the [Remote Execution](../guides/advanced/execute_remotely.md) example.
|
||||
|
||||
#### Remote Function Execution
|
||||
A specific function can also be launched on a remote machine with the [`Task.create_function_task`](../references/sdk/task.md#create_function_task)
|
||||
method.
|
||||
|
||||
For example:
|
||||
```python
|
||||
def run_me_remotely(some_argument):
|
||||
print(some_argument)
|
||||
|
||||
a_func_task = task.create_function_task(
|
||||
func=run_me_remotely, # type: Callable
|
||||
func_name='func_id_run_me_remotely', # type:Optional[str]
|
||||
task_name='a func task', # type:Optional[str]
|
||||
# everything below will be passed directly to our function as arguments
|
||||
some_argument=123
|
||||
)
|
||||
```
|
||||
Arguments passed to the function will be automatically logged under the `Function` section in the Hyperparameters tab.
|
||||
Like any other arguments, they can be changed from the UI or programmatically.
|
||||
|
||||
:::note Function Task Creation
|
||||
Function tasks must be created from within a regular task, created by calling `Task.init`
|
||||
:::
|
||||
|
||||
### Offline Mode
|
||||
|
||||
You can work with tasks in Offline Mode, in which all the data and logs that the Task captures are stored in a local
|
||||
folder, which can later be uploaded to the [ClearML Server](../deploying_clearml/clearml_server.md).
|
||||
|
||||
Before initializing a Task, use the [Task.set_offline](../references/sdk/task.md#taskset_offline) class method and set
|
||||
the `offline_mode` argument to `True`. The method returns the Task ID and a path to the session folder.
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
# Use the set_offline class method before initializing a Task
|
||||
Task.set_offline(offline_mode=True)
|
||||
# Initialize a Task
|
||||
task = Task.init(project_name="examples", task_name="my_task")
|
||||
|
||||
# Rest of code is executed. All data is logged locally and not onto the server
|
||||
```
|
||||
|
||||
All the information captured by the Task is saved locally. Once the task script finishes execution, it's zipped.
|
||||
|
||||
Upload the execution data that the Task captured offline to the ClearML Server using one of the following:
|
||||
* [`clearml-task`](../apps/clearml_task.md) CLI
|
||||
```bash
|
||||
clearml-task --import-offline-session "path/to/session/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip"
|
||||
```
|
||||
Pass the path to the zip folder containing the session with the `--import-offline-session` parameter
|
||||
|
||||
* [Task.import_offline_session](../references/sdk/task.md#taskimport_offline_session) class method
|
||||
```python
|
||||
from clearml import Task
|
||||
Task.import_offline_session(session_folder_zip="path/to/session/.clearml/cache/offline/b786845decb14eecadf2be24affc7418.zip")
|
||||
```
|
||||
|
||||
In the `session_folder_zip` argument, insert the path to the zip folder containing the session.
|
||||
|
||||
Both options will upload the Task's full execution details and outputs and return a link to the Task's results page on
|
||||
the ClearML Server.
|
||||
|
||||
## Artifacts
|
||||
Artifacts are the output files created by a task. ClearML uploads and logs these products so they can later be easily
|
||||
accessed, modified, and used.
|
||||
|
||||
### Logging Artifacts
|
||||
To log an artifact in a task, use the [`upload_artifact`](../references/sdk/task.md#upload_artifact) method.
|
||||
|
||||
For example:
|
||||
* Upload a local file containing the preprocessing results of the data:
|
||||
```python
|
||||
task.upload_artifact(name='data', artifact_object='/path/to/preprocess_data.csv')
|
||||
```
|
||||
|
||||
* Upload an entire folder with all its content by passing the folder, which will be zipped and uploaded as a single
|
||||
zip file:
|
||||
```python
|
||||
task.upload_artifact(name='folder', artifact_object='/path/to/folder')
|
||||
```
|
||||
|
||||
* Serialize and upload a Python object. ClearML automatically chooses the file format based on the object’s type, or you
|
||||
can explicitly specify the format as follows:
|
||||
* dict - `.json` (default), `.yaml`
|
||||
* pandas.DataFrame - `.csv.gz` (default), `.parquet`, `.feather`, `.pickle`
|
||||
* numpy.ndarray - `.npz` (default), `.csv.gz`
|
||||
* PIL.Image - Any PIL-supported extensions (default `.png`)
|
||||
|
||||
For example:
|
||||
```python
|
||||
person_dict = {'name': 'Erik', 'age': 30}
|
||||
|
||||
# upload as JSON artifact
|
||||
task.upload_artifact(name='person dictionary json', artifact_object=person_dict)
|
||||
|
||||
# upload as YAML artifact
|
||||
task.upload_artifact(
|
||||
name='person dictionary yaml',
|
||||
artifact_object=person_dict,
|
||||
extension_name="yaml"
|
||||
)
|
||||
```
|
||||
|
||||
See more details in the [Artifacts Reporting example](../guides/reporting/artifacts.md) and in the [SDK reference](../references/sdk/task.md#upload_artifact).
|
||||
|
||||
### Using Artifacts
|
||||
A task's artifacts are accessed through the task’s *artifact* property which lists the artifacts’ locations.
|
||||
|
||||
The artifacts can subsequently be retrieved from their respective locations by using:
|
||||
* `get_local_copy()`- Downloads the artifact and caches it for later use, returning the path to the cached copy.
|
||||
* `get()` - Returns a Python object constructed from the downloaded artifact file.
|
||||
|
||||
The code below demonstrates how to access a file artifact using the previously generated preprocessed data:
|
||||
```python
|
||||
# get instance of task that created artifact, using task ID
|
||||
preprocess_task = Task.get_task(task_id='the_preprocessing_task_id')
|
||||
# access artifact
|
||||
local_csv = preprocess_task.artifacts['data'].get_local_copy()
|
||||
```
|
||||
|
||||
See more details in the [Using Artifacts example](https://github.com/allegroai/clearml/blob/master/examples/reporting/using_artifacts_example.py).
|
||||
|
||||
## Models
|
||||
The following is an overview of working with models through a `Task` object. It is also possible to work directly with model
|
||||
objects (see [Models (SDK)](model_sdk.md)).
|
||||
|
||||
### Logging Models Manually
|
||||
|
||||
To manually log a model in a task, create an instance of the [OutputModel](../references/sdk/model_outputmodel.md) class.
|
||||
An OutputModel object is always registered as an output model of the task it is constructed from.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
from clearml import OutputModel, Task
|
||||
|
||||
# Instantiate a Task
|
||||
task = Task.init(project_name="myProject", task_name="myTask")
|
||||
|
||||
# Instantiate an OutputModel with a task object argument
|
||||
output_model = OutputModel(task=task, framework="PyTorch")
|
||||
```
|
||||
|
||||
### Updating Models Manually
|
||||
|
||||
The snapshots of manually uploaded models aren't automatically captured. To update a task's model, use the
|
||||
[Task.update_output_model](../references/sdk/task.md#update_output_model) method:
|
||||
|
||||
```python
|
||||
task.update_output_model(model_path='path/to/model')
|
||||
```
|
||||
|
||||
It's possible to modify the following parameters:
|
||||
* Model location
|
||||
* Model name
|
||||
* Model description
|
||||
* Iteration number
|
||||
* Model tags
|
||||
|
||||
Models can also be manually updated independently, without any task. See [OutputModel.update_weights](../references/sdk/model_outputmodel.md#update_weights).
|
||||
|
||||
### Using Models
|
||||
|
||||
Accessing a task’s previously trained model is quite similar to accessing task artifacts. A task's models are accessed
|
||||
through the task’s models property which lists the input models and output model snapshots’ locations.
|
||||
|
||||
The models can subsequently be retrieved from their respective locations by using `get_local_copy()` which downloads the
|
||||
model and caches it for later use, returning the path to the cached copy (if using Tensorflow, the snapshots are stored
|
||||
in a folder, so the `local_weights_path` will point to a folder containing the requested snapshot).
|
||||
|
||||
```python
|
||||
prev_task = Task.get_task(task_id='the_training_task')
|
||||
last_snapshot = prev_task.models['output'][-1]
|
||||
local_weights_path = last_snapshot.get_local_copy()
|
||||
```
|
||||
|
||||
Notice that if one of the frameworks loads an existing weights file, the running task will automatically update its
|
||||
"Input Model", pointing directly to the original training task's model. This makes it easy to get the full lineage of
|
||||
every trained and used model in our system!
|
||||
|
||||
Models loaded by the ML framework appear under the "Input Models" section, under the Artifacts tab in the ClearML UI.
|
||||
|
||||
### Setting Upload Destination
|
||||
|
||||
ClearML automatically captures the storage location of Models created by frameworks such as TF, Pytorch, and scikit-learn.
|
||||
By default, it stores the local path they are saved at.
|
||||
|
||||
To automatically store all created models by a specific experiment, modify the `Task.init` function as such:
|
||||
|
||||
```python
|
||||
task = Task.init(project_name='examples', task_name='storing model', output_uri='s3://my_models/')
|
||||
```
|
||||
|
||||
To automatically store all models created by any experiment at a specific location, edit the `clearml.conf` (see
|
||||
[ClearML Configuration Reference](../configs/clearml_conf.md#sdkdevelopment)) and set `sdk.developmenmt.default_output_uri`
|
||||
to the desired storage (see [Storage](../integrations/storage.md)). This is especially helpful when
|
||||
using [clearml-agent](../clearml_agent.md) to execute code.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Manual Hyperparameter Logging
|
||||
|
||||
#### Setting Parameters
|
||||
|
||||
To define parameters manually use the [`Task.set_parameters`](../references/sdk/task.md#set_parameters) method to specify
|
||||
name-value pairs in a parameter dictionary.
|
||||
|
||||
Parameters can be designated into sections: specify a parameter’s section by prefixing its name, delimited with a slash
|
||||
(i.e. `section_name/parameter_name’:value`). `General` is the default section.
|
||||
|
||||
Call the [`set_parameter`](../references/sdk/task.md#set_parameter) method to set a single parameter.
|
||||
|
||||
```python
|
||||
task = Task.init(project_name='examples', task_name='parameters')
|
||||
|
||||
# override parameters with provided dictionary
|
||||
task.set_parameters({'Args/epochs':7, 'lr': 0.5})
|
||||
|
||||
# setting a single parameter
|
||||
task.set_parameter(name='decay',value=0.001)
|
||||
```
|
||||
|
||||
:::warning Overwriting Parameters
|
||||
The `set_parameters` method replaces any existing hyperparameters in the task.
|
||||
:::
|
||||
|
||||
#### Adding Parameters
|
||||
To update the parameters in a task, use the [`Task.set_parameters_as_dict`](../references/sdk/task.md#set_parameters_as_dict)
|
||||
method. Arguments and values are input as a dictionary. Like in `set_parameters` above, the parameter's section can
|
||||
be specified.
|
||||
|
||||
```python
|
||||
task = Task.task_get(task_id='123456789')
|
||||
|
||||
# add parameters
|
||||
task.set_parameters_as_dict({'my_args/lr':0.3, 'epochs':10})
|
||||
```
|
||||
|
||||
### Accessing Parameters
|
||||
|
||||
To access all task parameters, use the [`Task.get_parameters`](../references/sdk/task.md#get_parameters) method. This
|
||||
method returns a flattened dictionary of the `'section/parameter': 'value'` pairs.
|
||||
|
||||
```python
|
||||
task = Task.get_task(project_name='examples', task_name='parameters')
|
||||
|
||||
# will print a flattened dictionary of the 'section/parameter': 'value' pairs
|
||||
print(task.get_parameters())
|
||||
```
|
||||
|
||||
Access a specific parameter with the [`Task.get_parameter`](../references/sdk/task.md#get_parameter) method specifying
|
||||
the parameter name.
|
||||
|
||||
### Tracking Python Objects
|
||||
|
||||
ClearML can track Python objects (such as dictionaries and custom classes) as they evolve in your code, and log them to
|
||||
your task’s configuration using the [`Task.connect`](../references/sdk/task.md#connect) method. Once objects are connected
|
||||
to a task, ClearML automatically logs all object elements (e.g. class members, dictionary key-values pairs).
|
||||
|
||||
```python
|
||||
class person:
|
||||
def __init__(self, name, age):
|
||||
self.name = name
|
||||
self.age = age
|
||||
|
||||
me = person('Erik', 5)
|
||||
|
||||
params_dictionary = {'epochs': 3, 'lr': 0.4}
|
||||
|
||||
task = Task.init(project_name='examples',task_name='argparser')
|
||||
|
||||
task.connect(me)
|
||||
task.connect(params_dictionary)
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Configuration Objects
|
||||
|
||||
Configuration objects more elaborate than a key-value dictionary (such as nested dictionaries or configuration files),
|
||||
can be logged to a task using the [`Task.connect_configuration`](../references/sdk/task.md#connect_configuration) method.
|
||||
This method saves configuration objects as blobs (i.e. ClearML is not aware of their internal structure).
|
||||
|
||||
```python
|
||||
# connect a configuration dictionary
|
||||
model_config_dict = {
|
||||
'value': 13.37, 'dict': {'sub_value': 'string'}, 'list_of_ints': [1, 2, 3, 4],
|
||||
}
|
||||
model_config_dict = task.connect_configuration(name='dictionary', configuration=model_config_dict)
|
||||
|
||||
# connect a configuration file
|
||||
config_file_yaml = task.connect_configuration(name="yaml file", configuration='path/to/configuration/file.yaml', )
|
||||
```
|
||||
|
||||
|
||||
|
||||
### User Properties
|
||||
A task’s user properties do not impact task execution so you can add / modify the properties at any stage. Add user
|
||||
properties to a task with the [Task.set_user_properties](../references/sdk/task.md#set_user_properties) method.
|
||||
|
||||
```python
|
||||
task.set_user_properties(
|
||||
{"name": "backbone", "description": "network type", "value": "great"}
|
||||
)
|
||||
```
|
||||
|
||||
The above example sets the "backbone" property in a task.
|
||||
|
||||
|
||||
|
||||
## SDK Reference
|
||||
For detailed information, see the complete [Task SDK reference page](../references/sdk/task.md)
|
||||
@ -302,7 +302,7 @@ parameters_dict = Task.current_task().connect(parameters_dict)
|
||||
|
||||
**I noticed that all of my experiments appear as "Training" Are there other options?** <a id="other-experiment-types"></a>
|
||||
|
||||
Yes! When creating experiments and calling [Task.init](fundamentals/task.md#usage),
|
||||
Yes! When creating experiments and calling [Task.init](references/sdk/task.md#taskinit),
|
||||
you can provide an experiment type. ClearML supports [multiple experiment types](fundamentals/task.md#task-types). For example:
|
||||
|
||||
```python
|
||||
|
||||
@ -1,193 +1,77 @@
|
||||
---
|
||||
title: Artifacts & Models
|
||||
title: Models
|
||||
---
|
||||
|
||||
ClearML allows easy storage of experiments' output products as **artifacts** that can later be accessed easily
|
||||
and used, through the web UI or programmatically.
|
||||
ClearML supports tracking, updating, and visualizing models.
|
||||
|
||||
A few examples of artifacts are:
|
||||
* Model snapshot / weights file
|
||||
* Data preprocessing
|
||||
* Feature representation of data
|
||||
* and more!
|
||||
Models are stored in ClearML as experiment artifacts, but unlike other artifacts that are dependent on their creating
|
||||
task, models are independent entities with their own unique ID. Models can be accessed directly with a model object or
|
||||
indirectly via their creating task. This property makes Models a standalone entry that can be used as an artifactory
|
||||
interface.
|
||||
|
||||
## Artifacts
|
||||
### Logging Artifacts
|
||||
To log any type of artifact to a Task, use the [`upload_artifact`](../references/sdk/task.md#upload_artifact) method.
|
||||
For example:
|
||||
## Automatically Logging Models
|
||||
|
||||
* Upload a local file containing the preprocessing results of the data.
|
||||
```python
|
||||
task.upload_artifact(name='data', artifact_object='/path/to/preprocess_data.csv')
|
||||
```
|
||||
* Upload an entire folder with all its content by passing the folder, which will be zipped and uploaded as a single
|
||||
zip file:
|
||||
```python
|
||||
task.upload_artifact(name='folder', artifact_object='/path/to/folder/')
|
||||
```
|
||||
* Serialize and upload a Python object. ClearML automatically chooses the file format based on the object’s type, or you
|
||||
can explicitly specify the format as follows:
|
||||
* dict - `.json` (default), `.yaml`
|
||||
* pandas.DataFrame - `.csv.gz` (default), `.parquet`, `.feather`, `.pickle`
|
||||
* numpy.ndarray - `.npz` (default), `.csv.gz`
|
||||
* PIL.Image - Any PIL-supported extensions (default `.png`)
|
||||
Once integrated into code, ClearML automatically logs and tracks models and any snapshots created by the following
|
||||
frameworks:
|
||||
- Tensorflow (see [code example](../guides/frameworks/tensorflow/tensorflow_mnist.md))
|
||||
- Keras (see [code example](../guides/frameworks/keras/keras_tensorboard.md))
|
||||
- Pytorch (see [code example](../guides/frameworks/pytorch/pytorch_mnist.md))
|
||||
- scikit-learn (only using joblib) (see [code example](../guides/frameworks/scikit-learn/sklearn_joblib_example.md))
|
||||
- XGBoost (only using joblib) (see [code example](../guides/frameworks/xgboost/xgboost_sample.md))
|
||||
- FastAI (see [code example](../guides/frameworks/fastai/fastai_with_tensorboard.md))
|
||||
- MegEngine (see [code example](../guides/frameworks/megengine/megengine_mnist.md))
|
||||
- CatBoost (see [code example](../guides/frameworks/catboost/catboost.md))
|
||||
|
||||
When a supported framework loads a weights file, the running task will be automatically updated, with its input model
|
||||
pointing directly to the original training task's model.
|
||||
|
||||
## Manually Logging Models
|
||||
|
||||
### Output Models
|
||||
|
||||
ClearML stores training results as output models. The `OutputModel` object is instantiated with a task object as an
|
||||
argument (see [`task`](../references/sdk/model_outputmodel.md) parameter), so it's automatically registered as the Task’s
|
||||
output model. Since OutputModel objects are connected to tasks, the models are traceable in experiments.
|
||||
|
||||
Output models are read-write so weights can be updated throughout training. Additionally, users can specify a model's
|
||||
network design and label enumeration. Once an output model is registered, it can be used as the input model for another
|
||||
experiment.
|
||||
|
||||
The snapshots of manually uploaded models aren't automatically captured, but ClearML provides methods to update them
|
||||
through a `Task` or `OutputModel` object.
|
||||
|
||||
### Input Models
|
||||
|
||||
ClearML provides flexibility for explicitly connecting input models and experimentation, including:
|
||||
|
||||
* Importing pre-trained models from external sources such as Amazon AWS, GIT repositories, PyTorch, and TensorFlow.
|
||||
* Using standalone models already registered in ClearML by previously run experiments.
|
||||
* Defining your own input models in scripts
|
||||
|
||||
For example:
|
||||
```python
|
||||
person_dict = {'name': 'Erik', 'age': 30}
|
||||
|
||||
# upload as JSON artifact
|
||||
task.upload_artifact(name='person dictionary json', artifact_object=person_dict)
|
||||
|
||||
# upload as YAML artifact
|
||||
task.upload_artifact(
|
||||
name='person dictionary yaml',
|
||||
artifact_object=person_dict,
|
||||
extension_name="yaml"
|
||||
)
|
||||
```
|
||||
## Setting Upload Destination
|
||||
|
||||
See more details in the artifacts [example](../guides/reporting/artifacts.md).
|
||||
* ClearML automatically captures the storage path of Models created by supported frameworks. By default, it stores the
|
||||
local path they are saved to.
|
||||
* Upload destinations can be specified explicitly on a per OutputModel or per experiment basis. Alternatively, the upload
|
||||
destination of all OutputModels can be specified in the ClearML [configuration file](../configs/clearml_conf.md).
|
||||
|
||||
### Using Artifacts
|
||||
To access a Task's artifact in order to use it:
|
||||
1. Get the Task that created the artifact (see more details on [querying](task.md#querying--searching-tasks)
|
||||
Tasks).
|
||||
## WebApp Interface
|
||||
|
||||
1. Retrieve all the Task's artifacts with the *artifact* property, which is essentially a dictionary,
|
||||
where the key is the artifact name, and the value is the artifact itself.
|
||||
1. Access a specific artifact using one of the following methods:
|
||||
- Access files by calling `get_local_copy()`, which caches the files for later use and returns a path to the cached
|
||||
file
|
||||
- Access object artifacts by using the `get()` method that returns the Python object.
|
||||
|
||||
The code below demonstrates how to access a file artifact using the previously generated preprocessed data:
|
||||
```python
|
||||
# get instance of Task that created artifact, using Task ID
|
||||
preprocess_task = Task.get_task(task_id='the_preprocessing_task_id')
|
||||
# access artifact
|
||||
local_csv = preprocess_task.artifacts['data'].get_local_copy()
|
||||
```
|
||||
In the ClearML's web UI, model information can be located through a project's Model Table or through the model's creating
|
||||
task.
|
||||
|
||||
See more details in the using artifacts [example](../guides/reporting/using_artifacts.md).
|
||||
Models associated with a task appear in the task's **ARTIFACTS** tab. To see further model details, including design,
|
||||
label enumeration, and general information, click the model name, which is a hyperlink to the
|
||||
[model's detail page](../webapp/webapp_model_viewing.md).
|
||||
|
||||
### List of Supported Artifacts
|
||||
Models can also be accessed through their associated project's [Model Table](../webapp/webapp_model_table.md), where all
|
||||
the models associated with a project are listed.
|
||||
|
||||
- Numpy array (as npz file)
|
||||
- Pandas dataframe
|
||||
- PIL (converted to jpg)
|
||||
- Files and folders
|
||||
- Python objects (pickled)
|
||||
|
||||
|
||||
## Models
|
||||
Models are a special kind of artifact and, unlike regular artifacts, which can only be accessed with the creating Task's ID,
|
||||
Models are entities with their own unique ID that can be accessed directly or via the creating task.
|
||||
## SDK Interface
|
||||
|
||||
This property makes Models a standalone entry that can be used as an artifactory interface.
|
||||
See [the Models SDK interface](../clearml_sdk/model_sdk.md) for an overview for using the most basic Pythonic methods of the model
|
||||
classes. See a detailed list of all available methods in the [Model](../references/sdk/model_model.md), [OutputModel](../references/sdk/model_outputmodel.md), and [InputModel](../references/sdk/model_inputmodel.md)
|
||||
reference pages.
|
||||
|
||||
### Automatic Model Logging
|
||||
|
||||
When models are saved using certain frameworks (for instance, by calling the `torch.save()` method), ClearML automatically
|
||||
logs the models and all snapshot paths.
|
||||
|
||||
|
||||
|
||||
See automatic model logging examples:
|
||||
* [TF](../guides/frameworks/tensorflow/tensorflow_mnist.md)
|
||||
* [PyTorch](../guides/frameworks/pytorch/pytorch_mnist.md)
|
||||
* [Keras](../guides/frameworks/keras/keras_tensorboard.md)
|
||||
* [Scikit-Learn](../guides/frameworks/scikit-learn/sklearn_joblib_example.md)
|
||||
* [XGBoost](../guides/frameworks/xgboost/xgboost_sample.md)
|
||||
* [FastAI](../guides/frameworks/fastai/fastai_with_tensorboard.md)
|
||||
|
||||
|
||||
### Manual Model Logging
|
||||
|
||||
To manually log a model, create an instance of OutputModel class:
|
||||
```python
|
||||
from clearml import OutputModel, Task
|
||||
|
||||
# Instantiate a Task
|
||||
task = Task.init(project_name="myProject", task_name="myTask")
|
||||
|
||||
# Instantiate an OutputModel, with a Task object argument
|
||||
output_model = OutputModel(task=task, framework="PyTorch")
|
||||
```
|
||||
|
||||
The OutputModel object is always connected to a Task object as it's instantiated with a Task object as an argument.
|
||||
It is, therefore, automatically registered as the Task’s output model.
|
||||
|
||||
The snapshots of manually uploaded models aren't automatically captured, but there are two methods
|
||||
to update an output model.
|
||||
|
||||
#### Updating Via Task Object
|
||||
Using the [Task.update_output_model](../references/sdk/task.md#update_output_model) method:
|
||||
|
||||
```python
|
||||
task.update_output_model(model_path='path/to/model')
|
||||
```
|
||||
It's possible to modify the following parameters:
|
||||
* Weights file / folder - Uploads the files specified with the `model_path`.
|
||||
If a remote storage is provided (S3 / GS / Https etc...), it saves the URL.
|
||||
* Model Metadata - Model name, description, iteration number of model, and tags.
|
||||
|
||||
#### Updating Via Model Object
|
||||
Using the [OutputModel.update_weights](../references/sdk/model_outputmodel.md#update_weights) method:
|
||||
|
||||
```python
|
||||
output_model.update_weights()
|
||||
```
|
||||
* Specify either the name of a locally stored weights file to upload (`weights_filename`), or the URI of a storage destination
|
||||
for model weight upload (`registered_uri`).
|
||||
* Model Metadata - Model description and iteration number.
|
||||
|
||||
See [Model Configuration](../guides/reporting/model_config.md) example.
|
||||
|
||||
### Using Models
|
||||
|
||||
Loading a previously trained model is quite similar to loading artifacts.
|
||||
|
||||
```python
|
||||
prev_task = Task.get_task(task_id='the_training_task')
|
||||
last_snapshot = prev_task.models['output'][-1]
|
||||
local_weights_path = last_snapshot.get_local_copy()
|
||||
```
|
||||
1. Get the instance of the Task that created the original weights files
|
||||
2. Query the Task on its output models (a list of snapshots)
|
||||
3. Get the latest snapshot (if using Tensorflow, the snapshots are stored in a folder, so the `local_weights_path` will point to a folder containing the requested snapshot).
|
||||
|
||||
Notice that if one of the frameworks will load the weights file, the running Task will automatically update, with
|
||||
"Input Model" pointing directly to the original training Task's model. With this feature, it's easy to get a full genealogy
|
||||
of every trained and used model in our system!
|
||||
|
||||
Loading framework models appear under the "Input Models" section, under the Artifacts tab in the ClearML UI.
|
||||
|
||||
### Setting Upload Destination
|
||||
|
||||
ClearML automatically captures the storage path of Models created by frameworks such as TF, Pytorch, and scikit-learn. By default,
|
||||
it stores the local loading path they are saved to.
|
||||
|
||||
To automatically store all created models by a specific experiment, modify the `Task.init()` function as such:
|
||||
```python
|
||||
task = Task.init(project_name='examples', task_name='storing model', output_uri='s3://my_models/')
|
||||
```
|
||||
|
||||
To automatically store all created models from all experiments in a certain storage medium, edit the `clearml.conf` (see
|
||||
[ClearML Configuration Reference](../configs/clearml_conf.md#sdkdevelopment)) and set `sdk.developmenmt.default_output_uri` to the desired
|
||||
storage (see [Storage](../integrations/storage.md)).
|
||||
This is especially helpful when using [clearml-agent](../clearml_agent.md) to execute code.
|
||||
|
||||
### List of Supported Frameworks
|
||||
|
||||
- TensorFlow
|
||||
- Keras
|
||||
- PyTorch
|
||||
- PyTorch Ignite
|
||||
- PyTorch Lightning
|
||||
- scikit-learn (only using joblib)
|
||||
- XGBoost (only using joblib)
|
||||
- AutoKeras
|
||||
- FastAI
|
||||
- LightGBM
|
||||
- MegEngine
|
||||
- CatBoost
|
||||
@ -2,217 +2,117 @@
|
||||
title: Hyperparameters
|
||||
---
|
||||
|
||||
Hyperparameters are the configuration options given for a script.
|
||||
ClearML logs hyperparameters used in experiments from multiple different sources.
|
||||
Hyperparameters are a script's configuration options. Since hyperparameters can have substantial impact on
|
||||
model performance, it is crucial to efficiently track and manage them.
|
||||
|
||||
In ClearML, parameters are split into 3 sections:
|
||||
- User Properties - Modifiable section that can be edited post execution.
|
||||
- Hyperparameters - Individual parameters for configuration.
|
||||
- Configuration Objects - Usually configuration files (Json / YAML) or python objects.
|
||||
|
||||
These sections are further broken down into sub-sections (General / Args / TF_Define) for convenience.
|
||||
ClearML supports tracking and managing hyperparameters in each experiment and provides a dedicated [hyperparameter
|
||||
optimization module](hpo.md). With ClearML's logging and tracking capabilities, experiments can be reproduced, and their
|
||||
hyperparameters and results can be saved and compared, which is key to understanding model behavior.
|
||||
|
||||
|
||||
ClearML lets you easily try out different hyperparameter values without changing your original code. ClearML’s [execution
|
||||
agent](../clearml_agent.md) will override the original values with any new ones you specify through the web UI (see
|
||||
[Configuration](../webapp/webapp_exp_tuning.md#configuration) in the Tuning Experiments page). It's also possible to
|
||||
programmatically set experiment parameters.
|
||||
|
||||
## Command Line Parsing
|
||||
ClearML captures any command line parameters passed when invoking code that uses standard python packages such as
|
||||
argparse, [click](https://click.palletsprojects.com), or [Python Fire](https://github.com/google/python-fire). This happens automatically with no additional code required beyond
|
||||
initializing ClearML.
|
||||
## Tracking Hyperparameters
|
||||
Hyperparameters can be added from anywhere in your code, and ClearML provides multiple ways to obtain them! ClearML logs
|
||||
and tracks hyperparameters of various types, supporting automatic logging and explicit reporting.
|
||||
|
||||
### Argparse Example
|
||||
### Automatic Logging
|
||||
Once a ClearML Task has been [initialized](../references/sdk/task.md#taskinit) in a script, ClearML automatically captures and tracks
|
||||
the following types of parameters:
|
||||
* Command line parsing - command line parameters passed when invoking code that uses standard python packages, including:
|
||||
* [click](https://click.palletsprojects.com) - see code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/click/click_multi_cmd.py).
|
||||
* [argparse](https://docs.python.org/3/library/argparse.html) - see code example [here](../guides/frameworks/pytorch/pytorch_tensorboardx.md).
|
||||
* [Python Fire](https://github.com/google/python-fire) - see code examples [here](https://github.com/allegroai/clearml/tree/master/examples/frameworks/fire).
|
||||
* [LightningCLI](https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_cli.html) - see code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/jsonargparse/pytorch_lightning_cli.py).
|
||||
* TensorFlow Definitions (`absl-py`). See examples of ClearML's automatic logging of TF Defines:
|
||||
* [TensorFlow MNIST](../guides/frameworks/tensorflow/tensorflow_mnist.md)
|
||||
* [TensorBoard PR Curve](../guides/frameworks/tensorflow/tensorboard_pr_curve.md)
|
||||
* [Hydra](https://github.com/facebookresearch/hydra) - ClearML logs the `Omegaconf` which holds all the configuration files,
|
||||
as well as values overridden during runtime. See code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/hydra/hydra_example.py).
|
||||
|
||||
:::tip Disabling Automatic Logging
|
||||
Automatic logging can be disabled. See this [FAQ](../faq.md#controlling_logging).
|
||||
:::
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
import argparse
|
||||
### Environment Variables
|
||||
|
||||
parser = argparse.ArgumentParser(description="Script Argparser")
|
||||
parser.add_argument("-lr", default=0.001, help="Initial learning rate")
|
||||
parser.add_argument("-epochs", default= 10, help="Total number of epochs")
|
||||
args = parser.parse_args()
|
||||
|
||||
|
||||
task = Task.init(project_name="examples",task_name="argparser logging")
|
||||
```
|
||||
|
||||
See another argparse logging example [here](../guides/reporting/hyper_parameters.md).
|
||||
|
||||
### Click Example
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
import click
|
||||
|
||||
task = Task.init(project_name='examples', task_name='click single command')
|
||||
|
||||
@click.command()
|
||||
@click.option('--count', default=1, help='Number of greetings.')
|
||||
@click.option('--name', prompt='Your name',
|
||||
help='The person to greet.')
|
||||
def hello(count, name):
|
||||
for x in range(count):
|
||||
click.echo("Hello {}!".format(name))
|
||||
|
||||
hello()
|
||||
```
|
||||
|
||||
See another code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/click/click_multi_cmd.py).
|
||||
|
||||
## Connecting Objects
|
||||
|
||||
Users can directly connect objects, such as dictionaries or even custom classes, to Tasks.
|
||||
All class members will be automatically fetched and logged by ClearML.
|
||||
|
||||
* Connecting a class:
|
||||
```python
|
||||
class person:
|
||||
def __init__(self, name, age):
|
||||
self.name = name
|
||||
self.age = age
|
||||
|
||||
|
||||
me = person('Erik',5)
|
||||
|
||||
task = Task.init(project_name='examples',task_name='argparser')
|
||||
|
||||
task.connect(me)
|
||||
```
|
||||
See connecting configuration objects example [here](../guides/reporting/hyper_parameters.md).
|
||||
|
||||
|
||||
* Connecting a dictionary:
|
||||
```python
|
||||
task = Task.init(project_name='examples', task_name='dictionary logging')
|
||||
|
||||
params_dictionary = {'epochs': 3, 'lr': 0.4}
|
||||
|
||||
task.connect(params_dictionary)
|
||||
```
|
||||
|
||||
## User Properties
|
||||
|
||||
User properties are an editable key / value store, which enables adding information to an experiment,
|
||||
making it easier to search / filter. User properties are like parameters that can also be added after a Task's execution, which
|
||||
can also be added to an [experiment table](../webapp/webapp_exp_table.md) (i.e. customize columns).
|
||||
|
||||
For example:
|
||||
```python
|
||||
task.set_user_properties({"name": "backbone",
|
||||
"description": "network type",
|
||||
"value": "great"})
|
||||
```
|
||||
|
||||
The above example will connect to the Task a user property named "backbone", with the description "network type", and
|
||||
the value of "great".
|
||||
|
||||
## Environment Variables
|
||||
|
||||
:::important
|
||||
:::important Experiment Reproducibility
|
||||
Relying on environment variables makes an experiment not fully reproducible, since ClearML Agent can't reproduce them at
|
||||
runtime.
|
||||
:::
|
||||
|
||||
Environment variables can be logged by modifying the [clearml.conf](../configs/clearml_conf.md) file. Modify the *log_os_environments*
|
||||
Environment variables can be logged by modifying the [clearml.conf](../configs/clearml_conf.md) file. Modify the `log_os_environments`
|
||||
parameter specifying parameters to log.
|
||||
|
||||
`log_os_environments: ["AWS_*", "CUDA_VERSION"]`
|
||||
```editorconfig
|
||||
log_os_environments: ["AWS_*", "CUDA_VERSION"]`
|
||||
```
|
||||
|
||||
It's also possible to specify environment variables using the CLEARML_LOG_ENVIRONMENT variable.
|
||||
It's also possible to specify environment variables using the `CLEARML_LOG_ENVIRONMENT` variable.
|
||||
|
||||
:::note
|
||||
The CLEARML_LOG_ENVIRONMENT always overrides the clearml.conf file.
|
||||
:::note Overriding clearml.conf
|
||||
The `CLEARML_LOG_ENVIRONMENT` always overrides the `clearml.conf` file.
|
||||
:::
|
||||
|
||||
## TF Defines
|
||||
When a script that has integrated ClearML is executed, the environment variables listed in `clearml.conf` or specified by
|
||||
the `CLEARML_LOG_ENVIRONMENT` variable are logged by ClearML.
|
||||
|
||||
ClearML automatically captures TensorFlow definitions, which are used as configuration files for Tensorflow.
|
||||
### Explicit Logging
|
||||
|
||||
See examples of ClearML's automatic logging of TF Defines:
|
||||
* [TensorFlow MNIST](../guides/frameworks/tensorflow/tensorflow_mnist.md)
|
||||
* [TensorBoard PR Curve](../guides/frameworks/tensorflow/tensorboard_pr_curve.md)
|
||||
To augment its automatic logging, ClearML supports explicitly logging parameters. ClearML provides methods to directly
|
||||
connect Python objects and configuration objects, as well as manually set and update task parameters.
|
||||
|
||||
## Hydra
|
||||
#### Connecting Python Objects
|
||||
Users can directly connect Python objects, such as dictionaries and custom classes, to tasks, using the
|
||||
[Task.connect](../references/sdk/task.md#connect) method. Once objects are connected to a task, all object elements
|
||||
(e.g. class members, dictionary key-values pairs) are automatically logged by ClearML. Additionally, ClearML tracks these
|
||||
values as they change through your code.
|
||||
|
||||
[Hydra](https://github.com/facebookresearch/hydra) is a module developed by FaceBook AI Research to manage experiments'
|
||||
parameters. Hydra offers the best of both worlds, managing configurations with files while making parameters overridable at runtime.
|
||||
When connecting objects to ClearML, users can directly access and modify an object's elements (e.g. a specific key-value
|
||||
pair in a parameter dictionary).
|
||||
|
||||
ClearML logs the Omegaconf which holds all the configuration files, as well as overridden values.
|
||||
#### Connecting Configuration Objects
|
||||
Configuration objects are dictionaries or configuration files connected to the task using the
|
||||
[Task.connect_configuration](../references/sdk/task.md#connect_configuration) method. With this method, configuration
|
||||
objects are saved as blobs i.e. ClearML is not aware of their internal structure.
|
||||
|
||||
Check out the [example code](https://github.com/allegroai/clearml/blob/master/examples/frameworks/hydra/hydra_example.py),
|
||||
which demonstrates the creation of a configuration object to which configuration values can be added and overridden using the
|
||||
command line.
|
||||
#### Setting and Updating Parameters
|
||||
ClearML provides methods to set and update task parameters manually. Use the [Task.set_parameters](../references/sdk/task.md#set_parameters)
|
||||
method to define parameters manually. To update the parameters in an experiment, use the [Task.set_parameters_as_dict](../references/sdk/task.md#set_parameters_as_dict)
|
||||
method. The `set_parameters_as_dict` method updates parameters while the `set_parameters` method overrides the parameters.
|
||||
|
||||
## Configuration Objects
|
||||
ClearML does not automatically track changes to explicitly set parameters.
|
||||
|
||||
Configuration objects are dictionaries or configuration files connected to the Task. Unlike Hyperparameters, these are saved as a whole and not
|
||||
divided into individual parameters.
|
||||
### User Properties
|
||||
User properties do not impact tasks execution and so can be modified at any stage. They offer the convenience of setting
|
||||
helpful values which then be displayed in the [experiment table](../webapp/webapp_exp_table.md) (i.e. customize columns),
|
||||
making it easier to search / filter experiments. Add user properties to an experiment with the
|
||||
[Task.set_user_properties](../references/sdk/task.md#set_user_properties) method.
|
||||
|
||||
To connect a configuration dictionary:
|
||||
```python
|
||||
model_config_dict = {
|
||||
'value': 13.37,
|
||||
'dict': {'sub_value': 'string', 'sub_integer': 11},
|
||||
'list_of_ints': [1, 2, 3, 4],
|
||||
}
|
||||
model_config_dict = task.connect_configuration(name='dictionary', configuration=model_config_dict)
|
||||
```
|
||||
### Accessing Parameters
|
||||
|
||||
ClearML provides methods to directly access a task’s logged parameters.
|
||||
|
||||
To connect a configuration file:
|
||||
```python
|
||||
config_file_yaml = task.connect_configuration(name="yaml file", configuration='path/to/configuration/file.yaml', )
|
||||
```
|
||||
To get all of a task's parameters and properties (hyperparameters, configuration objects, and user properties), use the
|
||||
[Task.get_parameters](../references/sdk/task.md#get_parameters) method, which will return a dictionary with the parameters,
|
||||
including their sub-sections (see [WebApp sections](#webapp-access) below).
|
||||
|
||||
Configuration objects can be split into categories in the Configuration section.
|
||||
The "name" argument, is the name of the section that the object will go into. If a section name is not specified, the default section is *General*.
|
||||
## WebApp Interface
|
||||
|
||||
|
||||
Configurations can be viewed in web UI experiment pages, in the **CONFIGURATIONS** panel.
|
||||
|
||||
## Manual Parameter Access
|
||||
The configuration panel is split into three sections according to type:
|
||||
- **User Properties** - Modifiable section that can be edited post-execution.
|
||||
- **Hyperparameters** - Individual parameters for configuration
|
||||
- **Configuration Objects** - Usually configuration files (Json / YAML) or Python objects.
|
||||
|
||||
These sections are further broken down into sub-sections based on how the parameters were logged (General / Args / TF_Define / Environment).
|
||||
|
||||
### Manual Parameter Input
|
||||
|
||||
|
||||
In addition to connecting a dictionary or a class to log hyperparameters, users can also use the `set_parameters` method
|
||||
to define parameters manually. Parameters are inputted as dictionaries.
|
||||
## SDK Interface
|
||||
|
||||
Additionally, parameters can be categorized, and each category appears in its own section in the hyperparameter tab of the web UI.
|
||||
Specify a section by putting its name before the parameter, for example `'Args/epochs': 'value'` - 'epochs' will go into the
|
||||
'Args' section. If a section isn't specified, the parameter will go into the *General* section by default.
|
||||
|
||||
Calling the `set_parameter` method will set a single parameter.
|
||||
|
||||
```python
|
||||
task = Task.init(project_name='examples', task_name='parameters')
|
||||
|
||||
# override parameters with provided dictionary
|
||||
task.set_parameters({'Args/epochs':7, 'lr': 0.5})
|
||||
|
||||
# setting a single parameter
|
||||
task.set_parameter(name='decay',value=0.001)
|
||||
```
|
||||
:::warning
|
||||
The *set_parameters* method will override any parameters already logged.
|
||||
:::
|
||||
|
||||
### Adding Parameters
|
||||
To update the parameters in an experiment, use the `set_parameters_as_dict` method . Arguments and values are inputted as a dictionary.
|
||||
Like in the `set_parameters` method, the dictionary can be nested, so the parameter's section can be specified.
|
||||
|
||||
```python
|
||||
task = Task.task_get(task_id='123456789')
|
||||
|
||||
# add parameters
|
||||
task.set_parameters_as_dict({'my_args/lr':0.3, 'epochs':10})
|
||||
```
|
||||
|
||||
|
||||
### Accessing Parameters
|
||||
|
||||
To get all Task's parameters, use the `get_parameters()` method, which will return a dictionary with the parameters, including
|
||||
their section.
|
||||
|
||||
```python
|
||||
task = Task.get_task(project_name='examples', task_name='parameters')
|
||||
|
||||
# will print a flattened dictionary of the 'section/parameter': 'value' pairs. {'Args/epochs': '7', 'General/lr': '0.5'}
|
||||
print(task.get_parameters())
|
||||
```
|
||||
See the [Configuration section](../clearml_sdk/task_sdk.md#configuration) of the Task SDK page for an overview of basic Pythonic
|
||||
methods for working with hyperparameters.
|
||||
@ -71,7 +71,7 @@ logger = task.get_logger()
|
||||
### Media Reporting
|
||||
|
||||
ClearML also supports reporting media (such as audio, video and images) for every iteration.
|
||||
This section is mostly used for debugging. It's recommended to use [artifacts](artifacts.md#artifacts) for storing script
|
||||
This section is mostly used for debugging. It's recommended to use [artifacts](task.md#artifacts) for storing script
|
||||
outputs that would be used later on.
|
||||
|
||||
Only the last X results of each title / series are saved to prevent overloading the server.
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
title: Projects
|
||||
---
|
||||
|
||||
Projects are contextual containers for [tasks](task.md) and [models](artifacts.md#models) (as well as [dataviews](../hyperdatasets/dataviews.md)
|
||||
Projects are contextual containers for [tasks](task.md) and [models](artifacts.md) (as well as [dataviews](../hyperdatasets/dataviews.md)
|
||||
when Hyper-Datasets are enabled), providing a logical structure similar to file system folders.
|
||||
An often useful method is to categorize components into projects according to models or objectives.
|
||||
Grouping into projects helps in identifying tasks, models, and dataviews when queried.
|
||||
@ -27,7 +27,7 @@ models, and dataviews, can be viewed in the project's [experiments table](../web
|
||||
|
||||
### Creating Subprojects
|
||||
|
||||
When [initializing a task](task.md#task-creation), its project needs to be specified. If the project entered does not exist, it will be created.
|
||||
When [initializing a task](../clearml_sdk/task_sdk.md#task-creation), its project needs to be specified. If the project entered does not exist, it will be created.
|
||||
Projects can contain subprojects, just like folders can contain sub-folders. Input into the `project_name`
|
||||
parameter a target project path. The project path should follow the project tree hierarchy, in which the project and
|
||||
subprojects are slash (`/`) delimited.
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
---
|
||||
title: Task / Experiment
|
||||
title: Tasks
|
||||
---
|
||||
|
||||
**ClearML Task** lies at the heart of ClearML's experiment manager.
|
||||
@ -20,7 +20,7 @@ you can work with tasks in Offline Mode, in which all information is saved in a
|
||||
In the UI and code, tasks are grouped into [projects](projects.md), which are logical entities similar to folders. Users can decide
|
||||
how to group tasks, though different models or objectives are usually grouped into different projects.
|
||||
|
||||
Tasks that are in the system can be accessed and utilized with code. To [access a task](#accessing-tasks), it can be identified either by a
|
||||
Tasks that are in the system can be accessed and utilized with code. To [access a task](../clearml_sdk/task_sdk.md#accessing-tasks), it can be identified either by a
|
||||
project name & task name combination or by a unique ID.
|
||||
|
||||
It's possible to copy ([clone](../webapp/webapp_exp_reproducing.md)) a task multiple times and to modify it for re-execution.
|
||||
@ -33,7 +33,7 @@ The sections of **ClearML Task** are made up of the information that a task capt
|
||||
execution details and execution outputs. This information is used for tracking
|
||||
and visualizing results, reproducing, tuning, and comparing experiments, and executing workflows.
|
||||
|
||||
The captured [code execution information](../webapp/webapp_exp_track_visual.md#execution-details) includes:
|
||||
The captured [code execution information](../webapp/webapp_exp_track_visual.md#execution) includes:
|
||||
* Git information
|
||||
* Uncommitted code modifications
|
||||
* Python environment
|
||||
@ -42,17 +42,53 @@ The captured [code execution information](../webapp/webapp_exp_track_visual.md#e
|
||||
The captured [execution output](../webapp/webapp_exp_track_visual.md#experiment-results) includes:
|
||||
* [Console output](../webapp/webapp_exp_track_visual.md#console)
|
||||
* [Scalars](../webapp/webapp_exp_track_visual.md#scalars)
|
||||
* [Plots](../webapp/webapp_exp_track_visual.md#other-plots)
|
||||
* [Plots](../webapp/webapp_exp_track_visual.md#plots)
|
||||
* [Debug samples](../webapp/webapp_exp_track_visual.md#debug-samples)
|
||||
* [Models](artifacts.md#models)
|
||||
* [Models](artifacts.md)
|
||||
|
||||
To view a more in depth description of each task section, see [Tracking Experiments and Visualizing Results](../webapp/webapp_exp_track_visual.md).
|
||||
|
||||
## Task Types
|
||||
### Artifacts
|
||||
|
||||
ClearML allows easy storage of experiments' output products as artifacts that can later be accessed easily and used,
|
||||
through the [web UI](../webapp/webapp_overview.md) or programmatically.
|
||||
|
||||
ClearML provides methods to easily track files generated throughout your experiments’ execution such as:
|
||||
|
||||
- Numpy objects
|
||||
- Pandas DataFrames
|
||||
- PIL
|
||||
- Files and folders
|
||||
- Python objects
|
||||
- and more!
|
||||
|
||||
Most importantly, ClearML also logs experiments’ input and output models as well as interim model snapshots (see
|
||||
[Models](artifacts.md)).
|
||||
|
||||
#### Logging Artifacts
|
||||
ClearML provides an explicit logging interface that supports manually reporting a variety of artifacts. Any type of
|
||||
artifact can be logged to a task using the [`Task.upload_artifact`](../references/sdk/task.md#upload_artifacts) method.
|
||||
See more details in the [Artifacts Reporting example](../guides/reporting/artifacts.md).
|
||||
|
||||
ClearML can be configured to upload artifacts to any of the supported types of storage, which include local and shared
|
||||
folders, AWS S3 buckets, Google Cloud Storage, and Azure Storage. For more information, see [Storage](../integrations/storage.md).
|
||||
|
||||
:::note Debug Sample Storage
|
||||
Debug samples are handled differently, see [`Logger.set_default_upload_destination`](../references/sdk/logger.md#set_default_upload_destination)
|
||||
:::
|
||||
|
||||
#### Accessing Artifacts
|
||||
Artifacts that have been logged can be accessed by other tasks [through the task](../clearml_sdk/task_sdk.md#accessing-tasks)
|
||||
they are attached to, and then retrieving the artifact with one of its following methods:
|
||||
* `get_local_copy()` - caches the files for later use and returns a path to the cached file.
|
||||
* `get()` - use for Python objects. The method that returns the Python object.
|
||||
|
||||
See more details in the [Using Artifacts example](https://github.com/allegroai/clearml/blob/master/examples/reporting/using_artifacts_example.py).
|
||||
|
||||
## Task Types
|
||||
Tasks have a *type* attribute, which denotes their purpose (Training / Testing / Data processing). This helps to further
|
||||
organize projects and ensure tasks are easy to [search and find](#querying--searching-tasks). The default task type is *training*.
|
||||
Available task types are:
|
||||
organize projects and ensure tasks are easy to [search and find](../clearml_sdk/task_sdk.md#querying--searching-tasks).
|
||||
The default task type is *training*. Available task types are:
|
||||
- Experimentation
|
||||
|
||||
- *training*, *testing*, *inference*
|
||||
@ -67,7 +103,7 @@ Available task types are:
|
||||
## Task Lifecycle
|
||||
|
||||
ClearML Tasks are created in one of the following methods:
|
||||
* Manually running code that is instrumented with the ClearML SDK and invokes `Task.init()`.
|
||||
* Manually running code that is instrumented with the ClearML SDK and invokes [`Task.init`](../references/sdk/task.md#taskinit).
|
||||
* Cloning an existing task.
|
||||
* Creating a task via CLI using [clearml-task](../apps/clearml_task.md).
|
||||
|
||||
@ -104,11 +140,11 @@ The above diagram demonstrates how a previously run task can be used as a baseli
|
||||
|
||||
## Task States
|
||||
|
||||
The state of a Task represents its stage in the Task lifecycle. It indicates whether the Task is read-write (editable) or
|
||||
The state of a task represents its stage in the task lifecycle. It indicates whether the task is read-write (editable) or
|
||||
read-only. For each state, a state transition indicates which actions can be performed on an experiment, and the new state
|
||||
after performing an action.
|
||||
|
||||
The following table describes the Task states and state transitions.
|
||||
The following table describes the task states and state transitions.
|
||||
|
||||
| State | Description / Usage | State Transition |
|
||||
|---|---|---|
|
||||
@ -120,249 +156,8 @@ The following table describes the Task states and state transitions.
|
||||
| *Aborted* | The experiment ran, and was manually or programmatically terminated. | The same as *Completed*. |
|
||||
| *Published* | The experiment is read-only. Publish an experiment to prevent changes to its inputs and outputs. | A *Published* experiment cannot be reset. If it is cloned, the state of the newly cloned experiment becomes *Draft*. |
|
||||
|
||||
## SDK Interface
|
||||
|
||||
## Usage
|
||||
|
||||
### Task Creation
|
||||
|
||||
`Task.init()` is the main method used to create Tasks in ClearML. It will create a Task, and populate it with:
|
||||
* A link to the running git repository (including commit ID and local uncommitted changes)
|
||||
* Python packages used (i.e. directly imported Python packages, and the versions available on the machine)
|
||||
* Argparse arguments (default and specific to the current execution)
|
||||
* Reports to Tensorboard & Matplotlib and model checkpoints.
|
||||
|
||||
:::note
|
||||
ClearML object (e.g. task, project) names are required to be at least 3 characters long
|
||||
:::
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
|
||||
|
||||
task = Task.init(
|
||||
project_name='example', # project name of at least 3 characters
|
||||
task_name='task template', # task name of at least 3 characters
|
||||
task_type=None,
|
||||
tags=None,
|
||||
reuse_last_task_id=True,
|
||||
continue_last_task=False,
|
||||
output_uri=None,
|
||||
auto_connect_arg_parser=True,
|
||||
auto_connect_frameworks=True,
|
||||
auto_resource_monitoring=True,
|
||||
auto_connect_streams=True,
|
||||
)
|
||||
```
|
||||
|
||||
When a Task is initialized, it automatically captures parameters and outputs from supported frameworks. To control what ClearML
|
||||
automatically logs, see this [FAQ](../faq.md#controlling_logging).
|
||||
|
||||
Once a Task is created, the Task object can be accessed from anywhere in the code by calling [`Task.current_task`](../references/sdk/task.md#taskcurrent_task).
|
||||
|
||||
If multiple Tasks need to be created in the same process (for example, for logging multiple manual runs),
|
||||
make sure to close a Task, before initializing a new one. To close a task simply call `task.close`
|
||||
(see example [here](../guides/advanced/multiple_tasks_single_process.md)).
|
||||
|
||||
When initializing a Task, its project needs to be specified. If the project entered does not exist, it will be created.
|
||||
Projects can be divided into subprojects, just like folders are broken into sub-folders.
|
||||
For example:
|
||||
```python
|
||||
Task.init(project_name='main_project/sub_project', task_name='test')
|
||||
```
|
||||
|
||||
Nesting projects works on multiple levels. For example: `project_name=main_project/sub_project/sub_sub_project`
|
||||
|
||||
|
||||
#### Task Reuse
|
||||
Every `Task.init` call will create a new Task for the current execution.
|
||||
In order to mitigate the clutter that a multitude of debugging Tasks might create, a Task will be reused if:
|
||||
* The last time it was executed (on this machine) was under 72 hours ago (configurable, see
|
||||
`sdk.development.task_reuse_time_window_in_hours` in the [`sdk.development` section](../configs/clearml_conf.md#sdkdevelopment) of
|
||||
the ClearML configuration reference)
|
||||
* The previous Task execution did not have any artifacts / models
|
||||
|
||||
It's possible to always create a new Task by passing `reuse_last_task_id=False`.
|
||||
|
||||
See full `Task.init` documentation [here](../references/sdk/task.md#taskinit).
|
||||
|
||||
### Empty Task Creation
|
||||
|
||||
A Task can also be created without the need to execute the code itself.
|
||||
Unlike the runtime detections, all the environment and configuration details needs to be provided explicitly.
|
||||
|
||||
For example:
|
||||
```python
|
||||
task = Task.create(
|
||||
project_name='example',
|
||||
task_name='task template',
|
||||
repo='https://github.com/allegroai/clearml.git',
|
||||
branch='master',
|
||||
script='examples/reporting/html_reporting.py',
|
||||
working_directory='.',
|
||||
docker=None,
|
||||
)
|
||||
```
|
||||
|
||||
See [`Task.create`](../references/sdk/task.md#taskcreate) in the Python SDK reference.
|
||||
|
||||
### Tracking Task Progress
|
||||
Track a task’s progress by setting the task progress property using the [`Task.set_progress`](../references/sdk/task.md#set_progress) method.
|
||||
Set a task’s progress to a numeric value between 0 - 100. Access the task’s current progress, using the
|
||||
[`Task.get_progress`](../references/sdk/task.md#get_progress) method.
|
||||
|
||||
```python
|
||||
task = Task.init(project_name="examples", task_name="Track experiment progress")
|
||||
task.set_progress(0)
|
||||
# task doing stuff
|
||||
task.set_progress(50)
|
||||
print(task.get_progress())
|
||||
# task doing more stuff
|
||||
task.set_progress(100)
|
||||
```
|
||||
|
||||
While the task is running, the WebApp will show the task’s progress indication in the experiment table, next to the
|
||||
task’s status. If a task failed or was aborted, you can view how much progress it had made.
|
||||
|
||||
<div class="max-w-50">
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
Additionally, you can view a task’s progress in its [INFO](../webapp/webapp_exp_track_visual.md#general-information) tab
|
||||
in the WebApp.
|
||||
|
||||
### Accessing Tasks
|
||||
A Task can be identified by its project and name, and by a unique identifier (UUID string). The name and project of
|
||||
a Task can be changed after an experiment has been executed, but its ID can't be changed.
|
||||
|
||||
:::tip Locating Task IDs
|
||||
To locate a task ID, go to the task's info panel in the [WebApp](../webapp/webapp_overview.md). In the top of the panel,
|
||||
to the right of the task name, click `ID` and the task ID appears
|
||||
:::
|
||||
|
||||
Programmatically, Task objects can be retrieved by querying the system based on either the Task ID or a project and name
|
||||
combination. If a project / name combination is used, and multiple Tasks have the exact same name, the function will return
|
||||
the *last modified Task*.
|
||||
|
||||
For example:
|
||||
* Accessing a Task object with a Task ID:
|
||||
```python
|
||||
a_task = Task.get_task(task_id='123456deadbeef')
|
||||
```
|
||||
* Accessing a Task with a project / name:
|
||||
```python
|
||||
a_task = Task.get_task(project_name='examples', task_name='artifacts')
|
||||
```
|
||||
|
||||
Once a Task object is obtained, it's possible to query the state of the Task, reported scalars, etc.
|
||||
The Task's outputs, such as artifacts and models, can also be retrieved.
|
||||
|
||||
### Querying / Searching Tasks
|
||||
|
||||
Searching and filtering Tasks can be done via the [web UI](../webapp/webapp_overview.md), but also programmatically.
|
||||
Input search parameters into the `Task.get_tasks` method, which returns a list of Task objects that match the search.
|
||||
|
||||
For example:
|
||||
```python
|
||||
task_list = Task.get_tasks(
|
||||
task_ids=None, # type Optional[Sequence[str]]
|
||||
project_name=None, # Optional[str]
|
||||
task_name=None, # Optional[str]
|
||||
task_filter=None # Optional[Dict]
|
||||
)
|
||||
```
|
||||
|
||||
It's possible to also filter Tasks by passing filtering rules to `task_filter`.
|
||||
For example:
|
||||
```python
|
||||
task_filter={
|
||||
# only Tasks with tag `included_tag` and without tag `excluded_tag`
|
||||
'tags': ['included_tag', '-excluded_tag'],
|
||||
# filter out archived Tasks
|
||||
'system_tags': ['-archived'],
|
||||
# only completed & published Tasks
|
||||
'status': ['completed', 'published'],
|
||||
# only training type Tasks
|
||||
'type': ['training'],
|
||||
# match text in Task comment or task name
|
||||
'search_text': 'reg_exp_text'
|
||||
}
|
||||
```
|
||||
|
||||
### Cloning & Executing Tasks
|
||||
|
||||
Once a Task object is created, it can be a copied (cloned). `Task.clone` returns a copy of the original Task (`source_task`).
|
||||
By default, the cloned Task is added to the same project as the original, and it's called "Clone Of ORIGINAL_NAME", but
|
||||
the name / project / comment of the cloned Task can be directly overridden.
|
||||
|
||||
```python
|
||||
cloned = Task.clone(
|
||||
source_task=task, # type: Optional[Union[Task, str]]
|
||||
# override default name
|
||||
name='newly created task', # type: Optional[str]
|
||||
comment=None, # type: Optional[str]
|
||||
# insert cloned Task into a different project
|
||||
project=None, # type: Optional[str]
|
||||
)
|
||||
```
|
||||
|
||||
A cloned Task starts in [draft](#task-states) mode, so its Task configurations can be edited (see
|
||||
[Task.set_parameters](../references/sdk/task.md#set_parameters)).
|
||||
Once a Task is modified, launch it by pushing it into an execution queue, then a [ClearML Agent](../clearml_agent.md) will pull
|
||||
it from the queue and execute the Task.
|
||||
|
||||
```python
|
||||
Task.enqueue(
|
||||
task=task, # type: Union[Task, str]
|
||||
queue_name='default', # type: Optional[str]
|
||||
queue_id=None # type: Optional[str]
|
||||
)
|
||||
```
|
||||
|
||||
See enqueue [example](https://github.com/allegroai/clearml/blob/master/examples/automation/programmatic_orchestration.py).
|
||||
|
||||
### Advanced Remote Execution
|
||||
|
||||
A compelling workflow is:
|
||||
1. Running code on the development machine for a few iterations, or just setting up the environment.
|
||||
1. Moving the execution to a beefier remote machine for the actual training.
|
||||
|
||||
For example, to stop the current manual execution, and then re-run it on a remote machine, simply add the following
|
||||
function call to the code:
|
||||
```python
|
||||
task.execute_remotely(
|
||||
queue_name='default', # type: Optional[str]
|
||||
clone=False, # type: bool
|
||||
exit_process=True # type: bool
|
||||
)
|
||||
```
|
||||
|
||||
Once the function is called on the machine, it will stop the local process and enqueue the current Task into the *default*
|
||||
queue. From there, an agent will be able to pick it up and launch it.
|
||||
|
||||
See the [Remote Execution](https://github.com/allegroai/clearml/blob/master/examples/advanced/execute_remotely_example.py) example.
|
||||
|
||||
#### Remote Function Execution
|
||||
A specific function can also be launched on a remote machine with `create_function_task`.
|
||||
|
||||
For example:
|
||||
```python
|
||||
def run_me_remotely(some_argument):
|
||||
print(some_argument)
|
||||
|
||||
a_func_task = task.create_function_task(
|
||||
func=run_me_remotely, # type: Callable
|
||||
func_name='func_id_run_me_remotely', # type:Optional[str]
|
||||
task_name='a func task', # type:Optional[str]
|
||||
# everything below will be passed directly to our function as arguments
|
||||
some_argument=123
|
||||
)
|
||||
```
|
||||
Arguments passed to the function will be automatically logged under the `Function` section in the Hyperparameters tab.
|
||||
Like any other arguments, they can be changed from the UI or programmatically.
|
||||
|
||||
:::note
|
||||
Function Tasks must be created from within a regular Task, created by calling `Task.init()`
|
||||
:::
|
||||
See [the task SDK interface](../clearml_sdk/task_sdk.md) for an overview for using the most basic Pythonic methods of the `Task`class.
|
||||
See the [Task reference page](../references/sdk/task.md) for a complete list of available list.
|
||||
|
||||
|
||||
@ -24,7 +24,7 @@ During early stages of model development, while code is still being modified hea
|
||||
The abovementioned setups might be folded into each other and that's great! If you have a GPU machine for each researcher, that's awesome!
|
||||
The goal of this phase is to get a code, dataset and environment setup, so we can start digging to find the best model!
|
||||
|
||||
- [ClearML SDK](../../clearml_sdk.md) should be integrated into your code (check out our [getting started](ds_first_steps.md)).
|
||||
- [ClearML SDK](../../clearml_sdk/clearml_sdk.md) should be integrated into your code (check out our [getting started](ds_first_steps.md)).
|
||||
This helps visualizing the results and tracking progress.
|
||||
- [ClearML Agent](../../clearml_agent.md) helps moving your work to other machines without the hassle of rebuilding the environment every time,
|
||||
while also creating an easy queue interface that easily allows you to just drop your experiments to be executed one by one
|
||||
@ -43,7 +43,7 @@ yields the best performing model for our task!
|
||||
|
||||
Visualization and comparisons dashboards keep your sanity at bay! In this stage we usually have a docker container with all the binaries
|
||||
that we need.
|
||||
- [ClearML SDK](../../clearml_sdk.md) ensures that all the metrics, parameters and Models are automatically logged and can later be
|
||||
- [ClearML SDK](../../clearml_sdk/clearml_sdk.md) ensures that all the metrics, parameters and Models are automatically logged and can later be
|
||||
accessed, [compared](../../webapp/webapp_exp_comparing.md) and [tracked](../../webapp/webapp_exp_track_visual.md).
|
||||
- [ClearML Agent](../../clearml_agent.md) does the heavy lifting. It reproduces the execution environment, clones your code,
|
||||
applies code patches, manages parameters (Including overriding them on the fly), executes the code and queues multiple tasks
|
||||
|
||||
@ -81,7 +81,7 @@ Now you can integrate ClearML into your code!
|
||||
In ClearML, experiments are organized as [Tasks](../../fundamentals/task.md).
|
||||
|
||||
ClearML will automatically log your experiment and code, including outputs and parameters from popular ML frameworks,
|
||||
once you integrate the ClearML [SDK](../../clearml_sdk.md) with your code. To control what ClearML automatically logs, see this [FAQ](../../faq.md#controlling_logging).
|
||||
once you integrate the ClearML [SDK](../../clearml_sdk/clearml_sdk.md) with your code. To control what ClearML automatically logs, see this [FAQ](../../faq.md#controlling_logging).
|
||||
|
||||
At the beginning of your code, import the `clearml` package:
|
||||
|
||||
@ -113,7 +113,7 @@ ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72a
|
||||
|
||||
**That’s it!** You are done integrating ClearML with your code :)
|
||||
|
||||
Now, [command-line arguments](../../fundamentals/hyperparameters.md#command-line-parsing), [console output](../../fundamentals/logger.md#types-of-logged-results) as well as Tensorboard and Matplotlib will automatically be logged in the UI under the created Task.
|
||||
Now, [command-line arguments](../../fundamentals/hyperparameters.md#tracking-hyperparameters), [console output](../../fundamentals/logger.md#types-of-logged-results) as well as Tensorboard and Matplotlib will automatically be logged in the UI under the created Task.
|
||||
<br/>
|
||||
|
||||
Sit back, relax, and watch your models converge :) or continue to see what else can be done with ClearML [here](ds_second_steps.md).
|
||||
|
||||
@ -68,7 +68,7 @@ If the object type is unknown ClearML pickles it and uploads the pickle file.
|
||||
task.upload_artifacts(my_numpy_matrix, name='features')
|
||||
```
|
||||
|
||||
Check out all [artifact logging](../../fundamentals/artifacts.md) options.
|
||||
Check out all [artifact logging](../../clearml_sdk/task_sdk.md#artifacts) options.
|
||||
|
||||
### Using Artifacts
|
||||
|
||||
|
||||
@ -131,7 +131,7 @@ cloned_task.set_parameter(name='internal/magic', value=42)
|
||||
```
|
||||
|
||||
#### Report Artifacts
|
||||
Artifacts are files created by your task. Users can upload [multiple types of data](../../fundamentals/artifacts.md#logging-artifacts),
|
||||
Artifacts are files created by your task. Users can upload [multiple types of data](../../clearml_sdk/task_sdk.md#logging-artifacts),
|
||||
objects and files to a task anywhere from code.
|
||||
|
||||
```python
|
||||
@ -141,7 +141,7 @@ Task.current_task().upload_artifact(name='a_file', artifact_object='local_file.b
|
||||
Task.current_task().upload_artifact(name='numpy', artifact_object=np.ones(4,4))
|
||||
```
|
||||
|
||||
Artifacts serve as a great way to pass and reuse data between tasks. Artifacts can be [retrieved](../../fundamentals/artifacts.md#using-artifacts)
|
||||
Artifacts serve as a great way to pass and reuse data between tasks. Artifacts can be [retrieved](../../clearml_sdk/task_sdk.md#using-artifacts)
|
||||
by accessing the Task that created them. These artifacts can be modified and uploaded to other tasks.
|
||||
|
||||
```python
|
||||
|
||||
@ -298,5 +298,5 @@ python pytorch_mnist_tutorial.py
|
||||
## Next Steps
|
||||
|
||||
* See the [User Interface](../../webapp/webapp_overview.md) section to learn about its features.
|
||||
* See the [ClearML Python Package Reference](../../clearml_sdk.md) to learn about
|
||||
* See the [ClearML Python Package Reference](../../clearml_sdk/clearml_sdk.md) to learn about
|
||||
all the available classes and methods.
|
||||
@ -12,7 +12,7 @@ demonstrates reporting (uploading) images in several formats, including:
|
||||
|
||||
ClearML uploads images to the bucket specified in the ClearML [configuration file](../../configs/clearml_conf.md),
|
||||
or ClearML can be configured for image storage, see [Logger.set_default_upload_destination](../../references/sdk/logger.md#set_default_upload_destination)
|
||||
(storage for [artifacts](../../fundamentals/artifacts.md#setting-upload-destination) is different). Set credentials for
|
||||
(storage for [artifacts](../../clearml_sdk/task_sdk.md#setting-upload-destination) is different). Set credentials for
|
||||
storage in the ClearML configuration file.
|
||||
|
||||
When the script runs, it creates an experiment named `image reporting`, which is associated with the `examples` project.
|
||||
|
||||
@ -10,7 +10,7 @@ method to upload from:
|
||||
* URL of media already uploaded to some storage
|
||||
|
||||
ClearML uploads media to the bucket specified in the ClearML configuration file or ClearML can be configured for image storage, see [Logger.set_default_upload_destination](../../references/sdk/logger.md#set_default_upload_destination)
|
||||
(storage for [artifacts](../../fundamentals/artifacts.md#setting-upload-destination) is different). Set credentials for storage in the ClearML
|
||||
(storage for [artifacts](../../clearml_sdk/task_sdk.md#setting-upload-destination) is different). Set credentials for storage in the ClearML
|
||||
[configuration file](../../configs/clearml_conf.md).
|
||||
|
||||
ClearML reports media in the **ClearML Web UI** **>** experiment details **>** **RESULTS** tab **>** **DEBUG SAMPLES**
|
||||
|
||||
BIN
docs/img/fundamentals_task_config_hyperparams.png
Normal file
BIN
docs/img/fundamentals_task_config_hyperparams.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
BIN
docs/img/fundamentals_task_config_object.png
Normal file
BIN
docs/img/fundamentals_task_config_object.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
BIN
docs/img/fundamentals_task_config_properties.png
Normal file
BIN
docs/img/fundamentals_task_config_properties.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
@ -28,7 +28,7 @@ The experiments table default and customizable columns are described in the foll
|
||||
| **TYPE** | Type of experiment. ClearML supports multiple [task types](../fundamentals/task.md#task-types) for experimentation, and a variety of workflows and use cases. | Default |
|
||||
| **NAME** | Experiment name. | Default |
|
||||
| **TAGS** | Descriptive, user-defined, color-coded tags assigned to experiments. Use tags to classify experiments, and filter the list. See [tagging experiments](webapp_exp_track_visual.md#tagging-experiments). | Default |
|
||||
| **STATUS** | Experiment state (status). See a list of the [task states and state transitions](../fundamentals/task.md#task-states). If you programmatically set task progress values, you will also see a progress indicator for Running, Failed, and Aborted tasks. See [here](../fundamentals/task.md#tracking-task-progress). | Default |
|
||||
| **STATUS** | Experiment state (status). See a list of the [task states and state transitions](../fundamentals/task.md#task-states). If you programmatically set task progress values, you will also see a progress indicator for Running, Failed, and Aborted tasks. See [here](../clearml_sdk/task_sdk.md#tracking-task-progress). | Default |
|
||||
| **PROJECT** | Name of experiment's project. | Default |
|
||||
| **USER** | User who created or cloned the experiment. | Default (hidden) |
|
||||
| **STARTED** | Elapsed time since the experiment started. To view the date and time of start, hover over the elapsed time. | Default |
|
||||
|
||||
@ -5,7 +5,7 @@ title: Tracking Experiments and Visualizing Results
|
||||
While an experiment is running, and any time after it finishes, track it and visualize the results in the ClearML Web UI,
|
||||
including:
|
||||
|
||||
* [Execution details](#execution-details) - Code, the base Docker image used for [ClearML Agent](../clearml_agent.md), output destination for artifacts, and the logging level.
|
||||
* [Execution details](#execution) - Code, the base Docker image used for [ClearML Agent](../clearml_agent.md), output destination for artifacts, and the logging level.
|
||||
* [Configuration](#configuration) - Hyperparameters, user properties, and configuration objects.
|
||||
* [Artifacts](#artifacts) - Input model, output model, model snapshot locations, other artifacts.
|
||||
* [General information](#general-information) - Information about the experiment, for example: the experiment start, create, and last update times and dates, user creating the experiment, and its description.
|
||||
|
||||
@ -236,7 +236,7 @@ and / or user groups have access permissions to the following workspace resource
|
||||
|
||||
* [Projects](../fundamentals/projects.md)
|
||||
* [Tasks](../fundamentals/task.md)
|
||||
* [Models](../fundamentals/artifacts.md#models)
|
||||
* [Models](../fundamentals/artifacts.md)
|
||||
* [Dataviews](../hyperdatasets/dataviews.md)
|
||||
* [Datasets](../hyperdatasets/dataset.md)
|
||||
* [Queues](../fundamentals/agents_and_queues.md#what-is-a-queue)
|
||||
|
||||
@ -15,7 +15,7 @@ module.exports = {
|
||||
}, 'getting_started/architecture']},
|
||||
{'ClearML Fundamentals': ['fundamentals/projects', 'fundamentals/task', 'fundamentals/hyperparameters', 'fundamentals/artifacts', 'fundamentals/logger', 'fundamentals/agents_and_queues',
|
||||
'fundamentals/hpo']},
|
||||
'clearml_sdk',
|
||||
{'ClearML SDK': ['clearml_sdk/clearml_sdk', 'clearml_sdk/task_sdk', 'clearml_sdk/model_sdk']},
|
||||
'clearml_agent',
|
||||
{'ClearML Pipelines':['pipelines/pipelines',
|
||||
{"Building Pipelines":
|
||||
|
||||
Loading…
Reference in New Issue
Block a user