mirror of
https://github.com/clearml/clearml-docs
synced 2025-01-31 22:48:40 +00:00
57 lines
2.4 KiB
Markdown
57 lines
2.4 KiB
Markdown
---
|
|
title: Transformers
|
|
---
|
|
|
|
The [HuggingFace Transformers example](https://github.com/allegroai/clearml/blob/master/examples/frameworks/huggingface/transformers.ipynb)
|
|
demonstrates how to integrate ClearML into your Transformer's [Trainer](https://huggingface.co/docs/transformers/v4.34.1/en/main_classes/trainer)
|
|
code. The HuggingFace Trainer automatically uses the built-in [`ClearMLCallback`](https://huggingface.co/docs/transformers/v4.34.1/en/main_classes/callback#transformers.integrations.ClearMLCallback)
|
|
if the `clearl` package is already installed, to log Transformers models, parameters, scalars, and more.
|
|
|
|
In the example, ClearML is installed and set up in the training environment. This way ClearML can log models, parameters,
|
|
scalars, and more.
|
|
|
|
When the example runs, it creates a ClearML task called `Trainer` in the `HuggingFace Transformers` project. To change
|
|
the task's name or project, use the `CLEARML_PROJECT` and `CLEARML_TASK` environment variables respectively.
|
|
|
|
For more information about integrating ClearML into your Transformers code, see [HuggingFace Transformers](../../../integrations/transformers.md).
|
|
|
|
## WebApp
|
|
|
|
### Hyperparameters
|
|
|
|
ClearML automatically captures all the Trainer [parameters](https://huggingface.co/docs/transformers/v4.34.1/en/main_classes/trainer#transformers.TrainingArguments).
|
|
Notice in the code example that only a few of the `TrainingArguments` are explicitly set:
|
|
|
|
```python
|
|
training_args = TrainingArguments(
|
|
output_dir="path/to/save/folder/",
|
|
learning_rate=2e-5,
|
|
per_device_train_batch_size=8,
|
|
per_device_eval_batch_size=8,
|
|
num_train_epochs=2,
|
|
)
|
|
```
|
|
|
|
ClearML captures all of the `TrainingArguments` passed to the Trainer.
|
|
|
|
View these parameters in the experiment's **CONFIGURATION** tab **> Hyperparameters** section.
|
|
|
|
|
|
|
|
|
|
### Models
|
|
|
|
In order for ClearML to log the models created during training in this example, the `CLEARML_LOG_MODEL` environment
|
|
variable is set to `True`.
|
|
|
|
ClearML automatically captures the model snapshots created by the Trainer, and saves them as artifacts. View the snapshots in the
|
|
experiment's **ARTIFACTS** tab.
|
|
|
|
|
|
|
|
### Scalars
|
|
|
|
ClearML automatically captures the Trainer's scalars, which can be viewed in the experiment's **Scalars** tab.
|
|
|
|
|