diff --git a/docs/guides/reporting/model_config.md b/docs/guides/reporting/model_config.md index 30f36936..6c9f4808 100644 --- a/docs/guides/reporting/model_config.md +++ b/docs/guides/reporting/model_config.md @@ -1,76 +1,56 @@ --- -title: Configuring Models +title: Model Reporting --- -The [model_config.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/model_config.py) example demonstrates -configuring a model and defining label enumeration. Connect the configuration and label enumeration to a Task and, once -connected, **ClearML** tracks any changes to them. When **ClearML** stores a model in any framework, **ClearML** stores -the configuration and label enumeration with it. +The [model_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/model_reporting.py) example +demonstrates logging a model using the [OutputModel](../../references/sdk/model_outputmodel.md) +class. -When the script runs, it creates an experiment named `Model configuration example`, which is associated with the `examples` project. +The example does the following: +* Creates a task named `Model reporting example` in the `examples` project. +* Uses an OutputModel object to register a previously trained model and log its label enumeration. -## Configuring Models - -### Using a Configuration File - -Connect a configuration file to a Task by calling the [Task.connect_configuration](../../references/sdk/task.md#connect_configuration) -method with the file location and the configuration object's name as arguments. In this example, we connect a JSON file and a YAML file -to a Task. +## Initialization +An OutputModel object is instantiated for the task. ```python -config_file_json = 'data_samples/sample.json' -task.connect_configuration(name="json file", configuration=config_file_json) -config_file_yaml = 'data_samples/config_yaml.yaml' -task.connect_configuration(configuration=config_file_yaml, name="yaml file") +from clearml import Task, OutputModel + +# Connecting ClearML with the current process, +task = Task.init(project_name="examples", task_name="Model logging example") + +# Create output model and connect it to the task +output_model = OutputModel(task=task) ``` -The configuration is logged to the ClearML Task and can be viewed in the **ClearML Web UI** experiment details **>** **CONFIGURATION** tab **>** **CONFIGURATION OBJECTS** -section. The contents of the JSON file will appear in the **json file** object, and the contents of the YAML file will appear -in the **yaml file** object, as specified in the `name` parameter of the `connect_configuration` method. - -![image](../../img/examples_reporting_config.png) - -### Configuration Dictionary - -Connect a configuration dictionary to a Task by creating a dictionary, and then calling the [Task.connect_configuration](../../references/sdk/task.md#connect_configuration) -method with the dictionary and the object name as arguments. After the configuration is connected, **ClearML** tracks changes to it. - -```python -model_config_dict = { - 'CHANGE ME': 13.37, - 'dict': {'sub_value': 'string', 'sub_integer': 11}, - 'list_of_ints': [1, 2, 3, 4], -} -model_config_dict = task.connect_configuration( - name='dictionary', - configuration=model_config_dict -) - -# Update the dictionary after connecting it, and the changes will be tracked as well. -model_config_dict['new value'] = 10 -model_config_dict['CHANGE ME'] *= model_config_dict['new value'] -``` -The configurations are connected to the ClearML Task and can be viewed in the **ClearML Web UI** **>** experiment details **>** **CONFIGURATION** tab **>** -**CONFIGURATION OBJECTS** area **>** **dictionary** object. - -![image](../../img/examples_reporting_config_3.png) - ## Label Enumeration -Connect a label enumeration dictionary by creating the dictionary, and then calling the [Task.connect_label_enumeration](../../references/sdk/task.md#connect_label_enumeration) -method with the dictionary as an argument. +Set the model’s label enumeration using the [`OutputModel.update_labels`](../../references/sdk/model_outputmodel.md#update_labels) +method. ```python -# store the label enumeration of the training model -labels = {'background': 0, 'cat': 1, 'dog': 2} -task.connect_label_enumeration(labels) +labels = {"background": 0, "cat": 1, "dog": 2} +output_model.update_labels(labels) ``` -Log a local model file. +## Registering Models +Register a previously trained model using the [`OutputModel.update_weights`](../../references/sdk/model_outputmodel.md#update_weights) +method. The example code uses a model stored in S3. + ```python -OutputModel().update_weights('my_best_model.bin') -``` +# Manually log a model file, which will have the labels connected above +output_model.update_weights(register_uri=model_url) +``` -The model which is stored contains the model configuration and the label enumeration. +## WebApp +The model appears in the task’s **ARTIFACTS** tab. + +![Task artifacts](../../img/examples_model_logging_artifacts.png) + +Clicking on the model name takes you to the [model’s page](../../webapp/webapp_model_viewing.md), where you can view the +model’s details and access the model. + +The model’s **LABELS** tab displays its label enumeration. + +![Model Labels tab](../../img/examples_model_logging_labels.png) -![image](../../img/examples_reporting_config_2.png) diff --git a/docs/guides/reporting/model_updating.md b/docs/guides/reporting/model_updating.md new file mode 100644 index 00000000..724a1869 --- /dev/null +++ b/docs/guides/reporting/model_updating.md @@ -0,0 +1,99 @@ +--- +title: Model Updating +--- + +The [model_update_pytorch.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/model_update_pytorch.py) +example demonstrates training a model and logging it using the [OutputModel](../../references/sdk/model_outputmodel.md) +class. + +The example does the following: +* Creates a task named `Model update pytorch` in the `examples` project. +* Trains a neural network on the CIFAR10 dataset for image classification. +* Uses an OutputModel object to log the model, its label enumeration and configuration dictionary. + +:::note Disabling automatic framework logging +This example disables the default automatic capturing of PyTorch outputs, to demonstrate how to manually control what is +logged from PyTorch. See [this FAQ](../../faq.md#controlling_logging) for more information. +::: + +## Initialization +An OutputModel object is instantiated for the task. + +```python +from clearml import Task, OutputModel + +task = Task.init( + project_name="examples", + task_name="Model update pytorch", + auto_connect_frameworks={"pytorch": False} +) + +output_model = OutputModel(task=task) +``` + +## Label Enumeration +The label enumeration dictionary is logged using the [`Task.connect_label_enumeration`](../../references/sdk/task.md#connect_label_enumeration) +method which will update the task’s resulting model information. The current running task is accessed using the +[`Task.current_task`](../../references/sdk/task.md#taskcurrent_task) class method. + +```python +# store the label enumeration of the training model +classes = ("plane", "car", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck",) +enumeration = {k: v for v, k in enumerate(classes, 1)} +Task.current_task().connect_label_enumeration(enumeration) +``` + +:::note Directly Setting Model Enumeration +You can set a model’s label enumeration directly using the [`OutputModel.update_labels`](../../references/sdk/model_outputmodel.md#update_labels) +method +::: + +## Model Configuration + +Add a configuration dictionary to the model using the [`OutputModel.update_design`](../../references/sdk/model_outputmodel.md#update_design) +method. + +```python +model_config_dict = { + "list_of_ints": [1, 2, 3, 4], + "dict": { + "sub_value": "string", + "sub_integer": 11 + }, + "value": 13.37 +} + +model.update_design(config_dict=model_config_dict) +``` + +## Updating Models +To update a model, use the [OutputModel.update_weights](../../references/sdk/model_outputmodel.md#update_weights) method. +This uploads the model to the set storage destination (see [Setting Upload Destination](../../fundamentals/artifacts.md#setting-upload-destination)), +and registers that location to the task as the output model. + +```python +# CONDITION depicts a custom condition for when to save the model. The model is saved and then updated in ClearML +CONDITION = True + +if CONDITION: + torch.save(net.state_dict(), PATH) + model.update_weights(weights_filename=PATH) +``` + +## WebApp +The model appears in the task’s **ARTIFACTS** tab. + +![Task artifacts](../../img/examples_model_update_artifacts.png) + +Clicking on the model name takes you to the [model’s page](../../webapp/webapp_model_viewing.md), where you can view the +model’s details and access the model. + +![Model page](../../img/examples_model_update_model.png) + +The model’s **NETWORK** tab displays its configuration. + +![Model network tab](../../img/examples_model_update_network.png) + +The model’s **LABELS** tab displays its label enumeration. + +![Model labels](../../img/examples_model_update_labels.png) diff --git a/docs/img/examples_model_logging_artifacts.png b/docs/img/examples_model_logging_artifacts.png new file mode 100644 index 00000000..b15e111f Binary files /dev/null and b/docs/img/examples_model_logging_artifacts.png differ diff --git a/docs/img/examples_model_logging_labels.png b/docs/img/examples_model_logging_labels.png new file mode 100644 index 00000000..aa6bb18c Binary files /dev/null and b/docs/img/examples_model_logging_labels.png differ diff --git a/docs/img/examples_model_update_artifacts.png b/docs/img/examples_model_update_artifacts.png new file mode 100644 index 00000000..a63e30d6 Binary files /dev/null and b/docs/img/examples_model_update_artifacts.png differ diff --git a/docs/img/examples_model_update_labels.png b/docs/img/examples_model_update_labels.png new file mode 100644 index 00000000..cdd5eaaa Binary files /dev/null and b/docs/img/examples_model_update_labels.png differ diff --git a/docs/img/examples_model_update_model.png b/docs/img/examples_model_update_model.png new file mode 100644 index 00000000..761969d0 Binary files /dev/null and b/docs/img/examples_model_update_model.png differ diff --git a/docs/img/examples_model_update_network.png b/docs/img/examples_model_update_network.png new file mode 100644 index 00000000..55e8c43a Binary files /dev/null and b/docs/img/examples_model_update_network.png differ diff --git a/sidebars.js b/sidebars.js index f3265642..3bdd2161 100644 --- a/sidebars.js +++ b/sidebars.js @@ -106,7 +106,7 @@ module.exports = { {'Reporting': ['guides/reporting/explicit_reporting','guides/reporting/3d_plots_reporting', 'guides/reporting/artifacts', 'guides/reporting/using_artifacts', 'guides/reporting/clearml_logging_example', 'guides/reporting/html_reporting', 'guides/reporting/hyper_parameters', 'guides/reporting/image_reporting', 'guides/reporting/manual_matplotlib_reporting', 'guides/reporting/media_reporting', - 'guides/reporting/model_config', 'guides/reporting/pandas_reporting', 'guides/reporting/plotly_reporting', + 'guides/reporting/model_config', 'guides/reporting/model_updating', 'guides/reporting/pandas_reporting', 'guides/reporting/plotly_reporting', 'guides/reporting/scalar_reporting', 'guides/reporting/scatter_hist_confusion_mat_reporting', 'guides/reporting/text_reporting']}, {'Services': ['guides/services/aws_autoscaler', 'guides/services/cleanup_service', 'guides/services/slack_alerts']}, {'Storage': ['guides/storage/examples_storagehelper']},