diff --git a/docs/clearml_sdk/task_sdk.md b/docs/clearml_sdk/task_sdk.md index ad088eda..a92ab46e 100644 --- a/docs/clearml_sdk/task_sdk.md +++ b/docs/clearml_sdk/task_sdk.md @@ -261,10 +261,33 @@ task_filter={ # only training type tasks 'type': ['training'], # match text in task comment or task name - 'search_text': 'reg_exp_text' + 'search_text': 'reg_exp_text', + # order return task lists by their update time in ascending order + 'order_by': ['last_update'] } ``` +:::tip Order tasks by metrics +You can order the returned tasks by performance in a specific metric with `'order_by': [last_metrics...]`. +* `` and `` - MD5 encoded metric and variant names. In Python, you +can encode the strings with `hashlib.md5(str("").encode("utf-8")).hexdigest()` +* `` - Specify which metric values to use. The options are: `value` (last value), `min_value`, or `max_value` + +Use the `-` prefix to order the results in descending order. + +```python +title = hashlib.md5(str("testing").encode("utf-8")).hexdigest() +series = hashlib.md5(str("epoch_accuracy").encode("utf-8")).hexdigest() + +tasks = Task.get_tasks( + project_name='Example Project', + # order tasks by metric performance in descending order + task_filter={'order_by': [f'-last_metrics.{title}.{series}.max_value']} +) +``` +::: + + See [`Task.get_tasks`](../references/sdk/task.md#taskget_tasks) for all `task_filter` options. ### Tag Filters diff --git a/docs/webapp/webapp_reports.md b/docs/webapp/webapp_reports.md index 1d66b49e..42a392b2 100644 --- a/docs/webapp/webapp_reports.md +++ b/docs/webapp/webapp_reports.md @@ -94,7 +94,7 @@ resources will be displayed. See [Dynamic Queries](#dynamic-queries) below. * `single` (single-scalar values table) * `sample` (debug sample) * `parcoords` (hyperparameter comparison plots) - for this option, you need to also specify the following parameters: - * `metrics` - Unique metric/variant ID formatted like `metric_id.variant_id` (find with your browser's inspect. See note [below](#event_id)) + * `metrics` - Unique metric/variant ID formatted like `metric_id.variant_id` (see note [below](#event_id)) * `variants` - Parameters to include in the plot (write in following format `.&.`) * `value_type` - Specify which metric values to use. The options are: * `min_value` @@ -144,22 +144,18 @@ used. ``` A list of tags that the experiment should contain is specified in the `tags` argument. You can also specify tags that exclude experiments. See tag filter syntax examples [here](../clearml_sdk/task_sdk.md#tag-filters). -* Request the `training/accuracy` scalar plot of the 5 experiments with the best accuracy scores +* Request the `training/accuracy` scalar plot of the 5 experiments with the best accuracy scores (see Metric/Variant IDs note [below](#event_id)): ``` - src="?objectType=task&type=scalar&metrics=training&variants=accuracy&project=4043a1657f374e9298649c6ba72ad233&page_size=5&page=0&order_by[]=-last_metrics...value" + src="/widgets/?objectType=task&type=scalar&metrics=training&variants=accuracy&project=4043a1657f374e9298649c6ba72ad233&page_size=5&page=0&order_by[]=-last_metrics...value" ``` -:::tip Event IDs -The `tasks.get_all` and `models.get_all` API calls' parameters sometimes need event IDs, instead of names. To find event IDs: -1. Go to the relevant Experiments/Model table > Open the **Developer Tools** window (inspect) > click **Network**. -1. Execute the action you want the embed code to do (e.g. sort by update time, sort by accuracy). -1. Click on the API call `tasks.get_all_ex`/`models.get_all_ex` that appears in the **Network** tab. -1. Click on the **Payload** panel. -1. Click on the relevant parameter to see the relevant event's ID. For example, if you sorted by experiment accuracy, -you will see the metric's event ID under the `order_by` parameter. +:::tip Metric/Variant IDs +Metric names need to be MD5 encoded for parallel coordinate plots and for ordering query results by metric +performance. You can encode the strings in Python with `hashlib.md5(str("").encode("utf-8")).hexdigest()`, +and use the returned MD5 hash in your query. :::