mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-26 18:17:44 +00:00
Merge branch 'main' of https://github.com/allegroai/clearml-docs
This commit is contained in:
revital
commit
84793abc9a
@ -1,5 +1,5 @@
|
||||
---
|
||||
title: AutoKeras IMDB
|
||||
title: AutoKeras
|
||||
---
|
||||
The [autokeras_imdb_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) example
|
||||
script demonstrates the integration of ClearML into code, which uses [autokeras](https://github.com/keras-team/autokeras).
|
||||
|
||||
@ -1,42 +0,0 @@
|
||||
---
|
||||
title: AutoKeras
|
||||
displayed_sidebar: mainSidebar
|
||||
---
|
||||
Integrate ClearML into code that uses [autokeras](https://github.com/keras-team/autokeras). Initialize a ClearML
|
||||
Task in a code, and ClearML automatically logs scalars, plots, and images reported to TensorBoard, Matplotlib, Plotly,
|
||||
and Seaborn, and all other automatic logging, and explicit reporting added to the code (see [Logging](../../../fundamentals/logger.md)).
|
||||
|
||||
ClearML allows to:
|
||||
|
||||
* Visualize experiment results in the **ClearML Web UI**.
|
||||
* Track and upload models.
|
||||
* Track model performance and create tracking leaderboards.
|
||||
* Rerun experiments, reproduce experiments on any target machine, and tune experiments.
|
||||
* Compare experiments.
|
||||
|
||||
See the [AutoKeras](autokeras_imdb_example.md) example, which shows ClearML automatically logging:
|
||||
* Scalars
|
||||
* Hyperparameters
|
||||
* The console log
|
||||
* Models.
|
||||
|
||||
Once these are logged, they can be visualized in the **ClearML Web UI**.
|
||||
|
||||
:::note
|
||||
If you are not already using ClearML, see [Getting Started](/getting_started/ds/best_practices.md).
|
||||
:::
|
||||
|
||||
## Adding ClearML to Code
|
||||
|
||||
Add two lines of code:
|
||||
```python
|
||||
from clearml import Task
|
||||
task = Task.init(project_name="myProject", task_name="myExperiment")
|
||||
```
|
||||
|
||||
When the code runs, it initializes a Task in ClearML Server. A hyperlink to the experiment's log is output to the console.
|
||||
|
||||
CLEARML Task: created new task id=c1f1dc6cf2ee4ec88cd1f6184344ca4e
|
||||
CLEARML results page: https://app.clear.ml/projects/1c7a45633c554b8294fa6dcc3b1f2d4d/experiments/c1f1dc6cf2ee4ec88cd1f6184344ca4e/output/log
|
||||
|
||||
Later in the code, define callbacks using TensorBoard, and ClearML logs TensorBoard scalars, histograms, and images.
|
||||
@ -1,6 +1,5 @@
|
||||
---
|
||||
title: Keras
|
||||
displayed_sidebar: mainSidebar
|
||||
title: Keras with TensorBoard
|
||||
---
|
||||
|
||||
The example below demonstrates the integration of ClearML into code which uses Keras and TensorBoard.
|
||||
|
||||
117
docs/integrations/autokeras.md
Normal file
117
docs/integrations/autokeras.md
Normal file
@ -0,0 +1,117 @@
|
||||
---
|
||||
title: AutoKeras
|
||||
---
|
||||
|
||||
:::tip
|
||||
If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup
|
||||
instructions.
|
||||
:::
|
||||
|
||||
ClearML integrates seamlessly with [AutoKeras](https://autokeras.com/), automatically logging its models and scalars.
|
||||
|
||||
All you have to do is simply add two lines of code to your AutoKeras script:
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
task = Task.init(task_name="<task_name>", project_name="<project_name>")
|
||||
```
|
||||
|
||||
And that’s it! This creates a [ClearML Task](../fundamentals/task.md) which captures:
|
||||
* Source code and uncommitted changes
|
||||
* Installed packages
|
||||
* AutoKeras model files
|
||||
* Scalars (loss, learning rates)
|
||||
* Console output
|
||||
* General details such as machine details, runtime, creation date etc.
|
||||
* Hyperparameters created with standard python packages (e.g. argparse, click, Python Fire, etc.)
|
||||
* And more
|
||||
|
||||
You can view all the task details in the [WebApp](../webapp/webapp_exp_track_visual.md).
|
||||
|
||||
See an example of AutoKeras and ClearML in action [here](../guides/frameworks/autokeras/autokeras_imdb_example.md).
|
||||
|
||||
|
||||
|
||||
## Automatic Logging Control
|
||||
By default, when ClearML is integrated into your AutoKeras script, it captures AutoKeras models and scalars, as well as TensorFlow
|
||||
definitions and TensorBoard outputs. But, you may want to have more control over what your experiment logs.
|
||||
|
||||
To control a task's framework logging, use the `auto_connect_frameworks` parameter of [`Task.init()`](../references/sdk/task.md#taskinit).
|
||||
Completely disable all automatic logging by setting the parameter to `False`. For finer grained control of logged
|
||||
frameworks, input a dictionary, with framework-boolean pairs.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
auto_connect_frameworks={
|
||||
'tensorflow': False, 'tensorboard': False, 'pytorch': True,
|
||||
'xgboost': False, 'scikit': True, 'fastai': True, 'lightgbm': False,
|
||||
'hydra': True, 'detect_repository': True, 'tfdefines': True, 'joblib': True,
|
||||
'megengine': True, 'jsonargparse': True, 'catboost': False
|
||||
}
|
||||
```
|
||||
|
||||
To control AutoKeras logging, use the `tensorflow` and `tensorboard` keys.
|
||||
|
||||
You can also input wildcards as dictionary values, so ClearML will log a model created by a framework only if its local
|
||||
path matches at least one wildcard.
|
||||
|
||||
For example, in the code below, ClearML will log TensorFlow models only if their paths have the `.h5` extension. The
|
||||
unspecified frameworks' values default to true so all their models are automatically logged.
|
||||
|
||||
```python
|
||||
auto_connect_frameworks={'tensorflow' : '*.h5'}
|
||||
```
|
||||
|
||||
## Manual Logging
|
||||
To augment its automatic logging, ClearML also provides an explicit logging interface.
|
||||
|
||||
See more information about explicitly logging information to a ClearML Task:
|
||||
* [Models](../clearml_sdk/model_sdk.md#manually-logging-models)
|
||||
* [Configuration](../clearml_sdk/task_sdk.md#configuration) (e.g. parameters, configuration files)
|
||||
* [Artifacts](../clearml_sdk/task_sdk.md#artifacts) (e.g. output files or python objects created by a task)
|
||||
* [Scalars](../clearml_sdk/task_sdk.md#scalars)
|
||||
* [Text/Plots/Debug Samples](../fundamentals/logger.md#manual-reporting)
|
||||
|
||||
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
|
||||
|
||||
## Remote Execution
|
||||
ClearML logs all the information required to reproduce an experiment on a different machine (installed packages,
|
||||
uncommitted changes etc.). The [ClearML Agent](../clearml_agent) listens to designated queues and when a task is enqueued,
|
||||
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
|
||||
experiment manager.
|
||||
|
||||
Deploy a ClearML Agent onto any machine (e.g. a cloud VM, a local GPU machine, your own laptop) by simply running the
|
||||
following command on it:
|
||||
|
||||
```commandline
|
||||
clearml-agent daemon --queue <queues_to_listen_to> [--docker]
|
||||
```
|
||||
|
||||
Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md), to help you manage cloud workloads in the
|
||||
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
|
||||
and shuts down instances as needed, according to a resource budget that you set.
|
||||
|
||||
### Cloning, Editing, and Enqueuing
|
||||
|
||||
|
||||
|
||||
Use ClearML's web interface to edit task details, like configuration parameters or input models, then execute the task
|
||||
with the new configuration on a remote machine:
|
||||
|
||||
* Clone the experiment
|
||||
* Edit the hyperparameters and/or other details
|
||||
* Enqueue the task
|
||||
|
||||
The ClearML Agent executing the task will use the new values to [override any hard coded values](../clearml_agent).
|
||||
|
||||
### Executing a Task Remotely
|
||||
|
||||
You can set a task to be executed remotely programmatically by adding [`Task.execute_remotely()`](../references/sdk/task.md#execute_remotely)
|
||||
to your script. This method stops the current local execution of the task, and then enqueues it to a specified queue to
|
||||
re-run it on a remote machine.
|
||||
|
||||
```python
|
||||
# If executed locally, process will terminate, and a copy will be executed by an agent instead
|
||||
task.execute_remotely(queue_name='default', exit_process=True)
|
||||
```
|
||||
132
docs/integrations/keras.md
Normal file
132
docs/integrations/keras.md
Normal file
@ -0,0 +1,132 @@
|
||||
---
|
||||
title: Keras
|
||||
---
|
||||
|
||||
:::tip
|
||||
If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup
|
||||
instructions.
|
||||
:::
|
||||
|
||||
ClearML integrates with [Keras](https://keras.io/) out-of-the-box, automatically logging its models, scalars,
|
||||
TensorFlow definitions, and TensorBoard outputs.
|
||||
|
||||
All you have to do is simply add two lines of code to your Keras script:
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
task = Task.init(task_name="<task_name>", project_name="<project_name>")
|
||||
```
|
||||
|
||||
And that’s it! This creates a [ClearML Task](../fundamentals/task.md) which captures:
|
||||
* Source code and uncommitted changes
|
||||
* Installed packages
|
||||
* Keras models
|
||||
* Scalars (e.g. accuracy, loss)
|
||||
* TensorFlow definitions
|
||||
* [TensorBoard](https://www.tensorflow.org/tensorboard) outputs
|
||||
* Console output
|
||||
* General details such as machine details, runtime, creation date etc.
|
||||
* And more
|
||||
|
||||
You can view all the task details in the [WebApp](../webapp/webapp_exp_track_visual.md).
|
||||
|
||||
|
||||
|
||||
## Automatic Logging Control
|
||||
By default, when ClearML is integrated into your Keras script, it captures Keras models and scalars, as well as TensorFlow
|
||||
definitions and TensorBoard outputs. But, you may want to have more control over what your experiment logs.
|
||||
|
||||
To control a task's framework logging, use the `auto_connect_frameworks` parameter of [`Task.init()`](../references/sdk/task.md#taskinit).
|
||||
Completely disable all automatic logging by setting the parameter to `False`. For finer grained control of logged
|
||||
frameworks, input a dictionary, with framework-boolean pairs.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
auto_connect_frameworks={
|
||||
'tensorflow': False, 'tensorboard': False, 'matplotlib': True, 'pytorch': True,
|
||||
'xgboost': False, 'scikit': True, 'fastai': True, 'lightgbm': False,
|
||||
'hydra': True, 'detect_repository': True, 'tfdefines': True, 'joblib': True,
|
||||
'megengine': True, 'jsonargparse': True, 'catboost': True
|
||||
}
|
||||
```
|
||||
|
||||
To control Keras logging, use the `tensorflow` and `tensorboard` keys.
|
||||
|
||||
You can also input wildcards as dictionary values, so ClearML will log a model created by a framework only if its local
|
||||
path matches at least one wildcard.
|
||||
|
||||
For example, in the code below, ClearML will log TensorFlow (and/or keras) models only if their paths have the `.keras` extension. The
|
||||
unspecified frameworks' values default to true so all their models are automatically logged.
|
||||
|
||||
```python
|
||||
auto_connect_frameworks={'tensorflow' : '*.keras'}
|
||||
```
|
||||
|
||||
## Manual Logging
|
||||
To augment its automatic logging, ClearML also provides an explicit logging interface.
|
||||
|
||||
See more information about explicitly logging information to a ClearML Task:
|
||||
* [Models](../clearml_sdk/model_sdk.md#manually-logging-models)
|
||||
* [Configuration](../clearml_sdk/task_sdk.md#configuration) (e.g. parameters, configuration files)
|
||||
* [Artifacts](../clearml_sdk/task_sdk.md#artifacts) (e.g. output files or python objects created by a task)
|
||||
* [Scalars](../clearml_sdk/task_sdk.md#scalars)
|
||||
* [Text/Plots/Debug Samples](../fundamentals/logger.md#manual-reporting)
|
||||
|
||||
See [Explicit Reporting Tutorial](../guides/reporting/explicit_reporting.md).
|
||||
|
||||
## Examples
|
||||
|
||||
Take a look at ClearML’s Keras examples. The examples use Keras and ClearML in different configurations with
|
||||
additional tools like TensorBoard and Matplotlib:
|
||||
* [Keras with Tensorboard](../guides/frameworks/keras/keras_tensorboard.md) - Demonstrates ClearML logging a Keras model,
|
||||
and plots and scalars logged to TensorBoard
|
||||
* [Keras with Matplotlib](../guides/frameworks/keras/jupyter.md) - Demonstrates ClearML logging a Keras model, Matplotlib plots,
|
||||
and debug samples, plots, and scalars logged to TensorBoard
|
||||
|
||||
|
||||
## Remote Execution
|
||||
ClearML logs all the information required to reproduce an experiment on a different machine (installed packages,
|
||||
uncommitted changes etc.). The [ClearML Agent](../clearml_agent) listens to designated queues and when a task is enqueued,
|
||||
the agent pulls it, recreates its execution environment, and runs it, reporting its scalars, plots, etc. to the
|
||||
experiment manager.
|
||||
|
||||
Deploy a ClearML Agent onto any machine (e.g. a cloud VM, a local GPU machine, your own laptop) by simply running the
|
||||
following command on it:
|
||||
|
||||
```commandline
|
||||
clearml-agent daemon --queue <queues_to_listen_to> [--docker]
|
||||
```
|
||||
|
||||
Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md), to help you manage cloud workloads in the
|
||||
cloud of your choice (AWS, GCP, Azure) and automatically deploy ClearML agents: the autoscaler automatically spins up
|
||||
and shuts down instances as needed, according to a resource budget that you set.
|
||||
|
||||
### Cloning, Editing, and Enqueuing
|
||||
|
||||
|
||||
|
||||
Use ClearML's web interface to edit task details, like configuration parameters or input models, then execute the task
|
||||
with the new configuration on a remote machine:
|
||||
|
||||
* Clone the experiment
|
||||
* Edit the hyperparameters and/or other details
|
||||
* Enqueue the task
|
||||
|
||||
The ClearML Agent executing the task will use the new values to [override any hard coded values](../clearml_agent).
|
||||
|
||||
### Executing a Task Remotely
|
||||
|
||||
You can set a task to be executed remotely programmatically by adding [`Task.execute_remotely()`](../references/sdk/task.md#execute_remotely)
|
||||
to your script. This method stops the current local execution of the task, and then enqueues it to a specified queue to
|
||||
re-run it on a remote machine.
|
||||
|
||||
```python
|
||||
# If executed locally, process will terminate, and a copy will be executed by an agent instead
|
||||
task.execute_remotely(queue_name='default', exit_process=True)
|
||||
```
|
||||
|
||||
## Hyperparameter Optimization
|
||||
Use ClearML’s [`HyperParameterOptimizer`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md) class to find
|
||||
the hyperparameter values that yield the best performing models. See [Hyperparameter Optimization](../fundamentals/hpo.md)
|
||||
for more information.
|
||||
@ -42,6 +42,7 @@ For more information about how autoscalers work, see [Autoscalers Overview](../.
|
||||
* **Compute Resources**
|
||||
* Resource Name - Assign a name to the resource type. This name will appear in the Autoscaler dashboard
|
||||
* EC2 Instance Type - See [Instance Types](https://aws.amazon.com/ec2/instance-types) for full list of types
|
||||
* Run in CPU mode - Check box to run with CPU only
|
||||
* Use Spot Instance - Check box to use a spot instance. Else, a reserved instance is used
|
||||
* Regular Instance Rollback Timeout - Controls when the autoscaler will revert to starting a regular instance after failing to start a spot instance. It will first attempt to start a spot, and then wait and retry again and again. Once the time it waited exceeded the Regular Instance Rollback Timeout, the autoscaler will try to start a regular instance instead. This is for a specific attempt, where starting a spot fails and an alternative instance needs to be started.
|
||||
* Spot Instance Blackout Period - Specifies a blackout period after failing to start a spot instance. This is related to future attempts: after failing to start a spot instance, all requests to start additional spot instances will be converted to attempts to start regular instances, as a way of "easing" the spot requests load on the cloud provider and not creating a "DOS" situation in the cloud account which might cause the provider to refuse creating spots for a longer period.
|
||||
@ -55,13 +56,14 @@ For more information about how autoscalers work, see [Autoscalers Overview](../.
|
||||
commas
|
||||
* EBS Device (optional) - Disk mount point
|
||||
* EBS Volume Size (optional) - Disk size (GB)
|
||||
* EBS Volume Type (optional) - See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html)
|
||||
* EBS Volume Type (optional) - See [Amazon EBS volume types](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html)
|
||||
for full list of types
|
||||
* Instance Key Pair (optional) - AWS key pair that is provided to the spun EC2 instances for connecting to them via
|
||||
SSH. Provide the Key Pair's name, as was created in AWS. See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)
|
||||
SSH. Provide the Key Pair's name, as was created in AWS. See [Amazon EC2 key pairs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)
|
||||
for more details
|
||||
* Security Group ID (optional) - Comma separated list of AWS VPC Security Group IDs to attach to the launched
|
||||
instance. Read more [here](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
|
||||
* VPC Subnet ID - The subnet ID For the created instance. If more than one ID is provided, the instance will be started in the first available subnet. For more information, see [What is Amazon VPC?](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html)
|
||||
* \+ Add Item - Define another resource type
|
||||
* **IAM Instance Profile** (optional) - Set an IAM instance profile for all instances spun by the Autoscaler
|
||||
* Arn - Amazon Resource Name specifying the instance profile
|
||||
|
||||
@ -59,10 +59,10 @@ module.exports = {
|
||||
{'ClearML Serving':['clearml_serving/clearml_serving', 'clearml_serving/clearml_serving_setup', 'clearml_serving/clearml_serving_cli', 'clearml_serving/clearml_serving_tutorial']},
|
||||
{'CLI Tools': ['apps/clearml_session', 'apps/clearml_task', 'apps/clearml_param_search']},
|
||||
{'Integrations': [
|
||||
'guides/frameworks/autokeras/integration_autokeras',
|
||||
'integrations/autokeras',
|
||||
'integrations/catboost', 'integrations/click', 'guides/frameworks/fastai/fastai_with_tensorboard',
|
||||
'integrations/hydra',
|
||||
'guides/frameworks/keras/keras_tensorboard', 'guides/frameworks/tensorflow/integration_keras_tuner',
|
||||
'integrations/keras', 'guides/frameworks/tensorflow/integration_keras_tuner',
|
||||
'integrations/lightgbm', 'integrations/matplotlib',
|
||||
'integrations/megengine', 'integrations/openmmv', 'integrations/optuna',
|
||||
'integrations/python_fire', 'integrations/pytorch',
|
||||
@ -147,7 +147,7 @@ module.exports = {
|
||||
{'Distributed': ['guides/distributed/distributed_pytorch_example', 'guides/distributed/subprocess_example']},
|
||||
{'Docker': ['guides/docker/extra_docker_shell_script']},
|
||||
{'Frameworks': [
|
||||
{'Autokeras': ['guides/frameworks/autokeras/integration_autokeras', 'guides/frameworks/autokeras/autokeras_imdb_example']},
|
||||
'guides/frameworks/autokeras/autokeras_imdb_example',
|
||||
'guides/frameworks/catboost/catboost',
|
||||
'guides/frameworks/fastai/fastai_with_tensorboard',
|
||||
{'Keras': ['guides/frameworks/keras/jupyter', 'guides/frameworks/keras/keras_tensorboard']},
|
||||
|
||||
Loading…
Reference in New Issue
Block a user