mirror of
https://github.com/clearml/clearml-docs
synced 2025-02-07 13:21:46 +00:00
Add metric encoding information (#578)
This commit is contained in:
parent
8d46338267
commit
7cd3a46930
@ -261,10 +261,33 @@ task_filter={
|
|||||||
# only training type tasks
|
# only training type tasks
|
||||||
'type': ['training'],
|
'type': ['training'],
|
||||||
# match text in task comment or task name
|
# match text in task comment or task name
|
||||||
'search_text': 'reg_exp_text'
|
'search_text': 'reg_exp_text',
|
||||||
|
# order return task lists by their update time in ascending order
|
||||||
|
'order_by': ['last_update']
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
:::tip Order tasks by metrics
|
||||||
|
You can order the returned tasks by performance in a specific metric with `'order_by': [last_metrics.<md5-encoded-metric-title>.<md5-encoded-metric-variant>.<value_type>]`.
|
||||||
|
* `<md5-encoded-metric-title>` and `<md5-encoded-metric-variant>` - MD5 encoded metric and variant names. In Python, you
|
||||||
|
can encode the strings with `hashlib.md5(str("<metric_name_string>").encode("utf-8")).hexdigest()`
|
||||||
|
* `<value_type>` - Specify which metric values to use. The options are: `value` (last value), `min_value`, or `max_value`
|
||||||
|
|
||||||
|
Use the `-` prefix to order the results in descending order.
|
||||||
|
|
||||||
|
```python
|
||||||
|
title = hashlib.md5(str("testing").encode("utf-8")).hexdigest()
|
||||||
|
series = hashlib.md5(str("epoch_accuracy").encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
tasks = Task.get_tasks(
|
||||||
|
project_name='Example Project',
|
||||||
|
# order tasks by metric performance in descending order
|
||||||
|
task_filter={'order_by': [f'-last_metrics.{title}.{series}.max_value']}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
:::
|
||||||
|
|
||||||
|
|
||||||
See [`Task.get_tasks`](../references/sdk/task.md#taskget_tasks) for all `task_filter` options.
|
See [`Task.get_tasks`](../references/sdk/task.md#taskget_tasks) for all `task_filter` options.
|
||||||
|
|
||||||
### Tag Filters
|
### Tag Filters
|
||||||
|
@ -94,7 +94,7 @@ resources will be displayed. See [Dynamic Queries](#dynamic-queries) below.
|
|||||||
* `single` (single-scalar values table)
|
* `single` (single-scalar values table)
|
||||||
* `sample` (debug sample)
|
* `sample` (debug sample)
|
||||||
* `parcoords` (hyperparameter comparison plots) - for this option, you need to also specify the following parameters:
|
* `parcoords` (hyperparameter comparison plots) - for this option, you need to also specify the following parameters:
|
||||||
* `metrics` - Unique metric/variant ID formatted like `metric_id.variant_id` (find with your browser's inspect. See note [below](#event_id))
|
* `metrics` - Unique metric/variant ID formatted like `metric_id.variant_id` (see note [below](#event_id))
|
||||||
* `variants` - Parameters to include in the plot (write in following format `<section_name>.<parameter_1>&<section_name>.<parameter_2>`)
|
* `variants` - Parameters to include in the plot (write in following format `<section_name>.<parameter_1>&<section_name>.<parameter_2>`)
|
||||||
* `value_type` - Specify which metric values to use. The options are:
|
* `value_type` - Specify which metric values to use. The options are:
|
||||||
* `min_value`
|
* `min_value`
|
||||||
@ -144,22 +144,18 @@ used.
|
|||||||
```
|
```
|
||||||
A list of tags that the experiment should contain is specified in the `tags` argument. You can also specify tags that
|
A list of tags that the experiment should contain is specified in the `tags` argument. You can also specify tags that
|
||||||
exclude experiments. See tag filter syntax examples [here](../clearml_sdk/task_sdk.md#tag-filters).
|
exclude experiments. See tag filter syntax examples [here](../clearml_sdk/task_sdk.md#tag-filters).
|
||||||
* Request the `training/accuracy` scalar plot of the 5 experiments with the best accuracy scores
|
* Request the `training/accuracy` scalar plot of the 5 experiments with the best accuracy scores (see Metric/Variant IDs note [below](#event_id)):
|
||||||
|
|
||||||
```
|
```
|
||||||
src="<web_server>?objectType=task&type=scalar&metrics=training&variants=accuracy&project=4043a1657f374e9298649c6ba72ad233&page_size=5&page=0&order_by[]=-last_metrics.<metric_event_id>.<variant_event_id>.value"
|
src="<web_server>/widgets/?objectType=task&type=scalar&metrics=training&variants=accuracy&project=4043a1657f374e9298649c6ba72ad233&page_size=5&page=0&order_by[]=-last_metrics.<metric_id>.<variant_id>.value"
|
||||||
```
|
```
|
||||||
|
|
||||||
<a id="event_id"></a>
|
<a id="event_id"></a>
|
||||||
|
|
||||||
:::tip Event IDs
|
:::tip Metric/Variant IDs
|
||||||
The `tasks.get_all` and `models.get_all` API calls' parameters sometimes need event IDs, instead of names. To find event IDs:
|
Metric names need to be MD5 encoded for parallel coordinate plots and for ordering query results by metric
|
||||||
1. Go to the relevant Experiments/Model table > Open the **Developer Tools** window (inspect) > click **Network**.
|
performance. You can encode the strings in Python with `hashlib.md5(str("<metric_string>").encode("utf-8")).hexdigest()`,
|
||||||
1. Execute the action you want the embed code to do (e.g. sort by update time, sort by accuracy).
|
and use the returned MD5 hash in your query.
|
||||||
1. Click on the API call `tasks.get_all_ex`/`models.get_all_ex` that appears in the **Network** tab.
|
|
||||||
1. Click on the **Payload** panel.
|
|
||||||
1. Click on the relevant parameter to see the relevant event's ID. For example, if you sorted by experiment accuracy,
|
|
||||||
you will see the metric's event ID under the `order_by` parameter.
|
|
||||||
:::
|
:::
|
||||||
|
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user