2021-05-13 23:48:51 +00:00
|
|
|
---
|
2023-06-22 13:08:26 +00:00
|
|
|
title: Keras Tuner
|
2021-05-13 23:48:51 +00:00
|
|
|
---
|
|
|
|
|
2023-08-15 16:00:06 +00:00
|
|
|
:::tip
|
2023-10-02 15:26:17 +00:00
|
|
|
If you are not already using ClearML, see [Getting Started](../../../getting_started/ds/ds_first_steps.md) for setup
|
2023-08-15 16:00:06 +00:00
|
|
|
instructions.
|
|
|
|
:::
|
|
|
|
|
2022-03-13 13:07:06 +00:00
|
|
|
Integrate ClearML into code that uses [Keras Tuner](https://www.tensorflow.org/tutorials/keras/keras_tuner). By
|
2021-05-13 23:48:51 +00:00
|
|
|
specifying `ClearMLTunerLogger` (see [kerastuner.py](https://github.com/allegroai/clearml/blob/master/clearml/external/kerastuner.py))
|
2022-03-13 13:07:06 +00:00
|
|
|
as the Keras Tuner logger, ClearML automatically logs scalars and hyperparameter optimization.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
## ClearMLTunerLogger
|
|
|
|
|
|
|
|
Take a look at [keras_tuner_cifar.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/kerastuner/keras_tuner_cifar.py)
|
2022-03-13 13:07:06 +00:00
|
|
|
example script, which demonstrates the integration of ClearML in a code that uses Keras Tuner.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
The script does the following:
|
|
|
|
1. Creates a `Hyperband` object, which uses Keras Tuner's `Hyperband` tuner. It finds the best hyperparameters to train a
|
|
|
|
network on a CIFAR10 dataset.
|
|
|
|
1. When the `Hyperband` object is created, instantiates a `ClearMLTunerLogger` object and assigns it to the `Hyperband` logger.
|
2022-03-13 13:07:06 +00:00
|
|
|
The `ClearMLTunerLogger` class provides the required binding for ClearML automatic logging.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
```python
|
|
|
|
tuner = kt.Hyperband(
|
|
|
|
build_model,
|
|
|
|
project_name='kt examples',
|
|
|
|
logger=ClearMLTunerLogger(),
|
|
|
|
objective='val_accuracy',
|
|
|
|
max_epochs=10,
|
|
|
|
hyperband_iterations=6)
|
|
|
|
```
|
|
|
|
|
|
|
|
When the script runs, it logs:
|
|
|
|
* Tabular summary of hyperparameters tested and their metrics by trial ID
|
|
|
|
* Scalar plot showing metrics for all runs
|
|
|
|
* Summary plot
|
|
|
|
* Output model with configuration and snapshot location.
|
|
|
|
|
|
|
|
## Scalars
|
|
|
|
|
2023-10-02 15:26:17 +00:00
|
|
|
ClearML logs the scalars from training each network. They appear in the experiment's page in the **ClearML web UI**, under
|
2022-05-22 07:27:30 +00:00
|
|
|
**SCALARS**.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_06.png)
|
|
|
|
|
2021-09-09 10:17:46 +00:00
|
|
|
## Summary of Hyperparameter Optimization
|
2021-05-13 23:48:51 +00:00
|
|
|
|
2022-03-13 13:07:06 +00:00
|
|
|
ClearML automatically logs the parameters of each experiment run in the hyperparameter search. They appear in tabular
|
2022-05-22 07:27:30 +00:00
|
|
|
form in **PLOTS**.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_07.png)
|
|
|
|
|
|
|
|
## Artifacts
|
|
|
|
|
2022-03-13 13:07:06 +00:00
|
|
|
ClearML automatically stores the output model. It appears in **ARTIFACTS** **>** **Output Model**.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_03.png)
|
|
|
|
|
|
|
|
Model details, such as snap locations, appear in the **MODELS** tab.
|
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_04.png)
|
|
|
|
|
|
|
|
The model configuration is stored with the model.
|
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_05.png)
|
|
|
|
|
2021-09-09 10:17:46 +00:00
|
|
|
## Configuration Objects
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
### Hyperparameters
|
|
|
|
|
2023-01-12 10:49:55 +00:00
|
|
|
ClearML automatically logs the TensorFlow Definitions, which appear in **CONFIGURATION** **>** **HYPERPARAMETERS**.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_01.png)
|
|
|
|
|
|
|
|
### Configuration
|
|
|
|
|
2022-05-22 07:27:30 +00:00
|
|
|
The Task configuration appears in **CONFIGURATION** **>** **General**.
|
2021-05-13 23:48:51 +00:00
|
|
|
|
|
|
|
![image](../../../img/integration_keras_tuner_02.png)
|
|
|
|
|
|
|
|
|