mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-26 18:17:44 +00:00
Edit hyperparameter-metric comparison FAQ (#344)
This commit is contained in:
parent
845c73eae5
commit
dece7a9358
34
docs/faq.md
34
docs/faq.md
@ -37,7 +37,7 @@ title: FAQ
|
|||||||
**Graphs and Logs**
|
**Graphs and Logs**
|
||||||
|
|
||||||
* [The first log lines are missing from the experiment console tab. Where did they go?](#first-log-lines-missing)
|
* [The first log lines are missing from the experiment console tab. Where did they go?](#first-log-lines-missing)
|
||||||
* [Can I create a graph comparing hyperparameters vs model accuracy?](#compare-graph-parameters)
|
* [How do I create a graph comparing hyperparameters vs model accuracy?](#compare-graph-parameters)
|
||||||
* [I want to add more graphs, not just with TensorBoard. Is this supported?](#more-graph-types)
|
* [I want to add more graphs, not just with TensorBoard. Is this supported?](#more-graph-types)
|
||||||
* [How can I report more than one scatter 2D series on the same plot?](#multiple-scatter2D)
|
* [How can I report more than one scatter 2D series on the same plot?](#multiple-scatter2D)
|
||||||
|
|
||||||
@ -525,24 +525,39 @@ info panel > CONSOLE tab, use the *Download full log* feature.
|
|||||||
|
|
||||||
<br/>
|
<br/>
|
||||||
|
|
||||||
**Can I create a graph comparing hyperparameters vs. model accuracy?** <a id="compare-graph-parameters"></a>
|
**How do I create a graph comparing hyperparameters vs. model accuracy?** <a id="compare-graph-parameters"></a>
|
||||||
|
|
||||||
Yes! You can manually create a plot with a single point X-axis for the hyperparameter value, and Y-axis for the accuracy.
|
You can use the UI's [experiment comparison features](webapp/webapp_exp_comparing.md) to compare the logged hyperparameter
|
||||||
For example:
|
and accuracy values of several experiments.
|
||||||
|
|
||||||
|
In an experiment comparison page, under the **HYPER PARAMETERS** tab, you can view the experiments' hyperparameter values
|
||||||
|
in relation to a specific metric (e.g. accuracy) in a parallel coordinates plot.
|
||||||
|
|
||||||
|
The image below show a parallel coordinates plot which displays the values of selected hyperparameters (`base_lr`,
|
||||||
|
`batch_size`, and `number_of_epochs`) and a performance metric (`accuracy`) of three experiments.
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
You can also visualize the differences in a scatter plot. In each experiment whose values wil compared, report a plot
|
||||||
|
with a single point, x-axis for the hyperparameter value, and Y-axis for the accuracy.
|
||||||
|
|
||||||
|
In the code below, the task reports a single-point scatter plot with `number_layers` as the x-axis and
|
||||||
|
`accuracy` as the Y-axis :
|
||||||
|
|
||||||
```python
|
```python
|
||||||
number_layers = 10
|
number_layers = 10
|
||||||
accuracy = 0.95
|
accuracy = 0.95
|
||||||
Task.current_task().get_logger().report_scatter2d(
|
Task.current_task().get_logger().report_scatter2d(
|
||||||
"performance",
|
title="performance",
|
||||||
"accuracy",
|
series="accuracy",
|
||||||
iteration=0,
|
iteration=0,
|
||||||
mode='markers',
|
mode='markers',
|
||||||
scatter=[(number_layers, accuracy)]
|
scatter=[(number_layers, accuracy)]
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
Assuming the hyperparameter is `number_layers` with current value `10`, and the `accuracy` for the trained model is `0.95`. Then, the experiment comparison graph shows:
|
When these experiments are compared in the UI's experiment comparison, all the reported `performance/accuracy` values
|
||||||
|
are displayed in a single plot.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
@ -552,14 +567,13 @@ Another option is a histogram chart:
|
|||||||
number_layers = 10
|
number_layers = 10
|
||||||
accuracy = 0.95
|
accuracy = 0.95
|
||||||
Task.current_task().get_logger().report_vector(
|
Task.current_task().get_logger().report_vector(
|
||||||
"performance",
|
title="performance",
|
||||||
"accuracy",
|
series="accuracy",
|
||||||
iteration=0,
|
iteration=0,
|
||||||
labels=['accuracy'],
|
labels=['accuracy'],
|
||||||
values=[accuracy],
|
values=[accuracy],
|
||||||
xlabels=['number_layers %d' % number_layers]
|
xlabels=['number_layers %d' % number_layers]
|
||||||
)
|
)
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||

|

|
||||||
|
Binary file not shown.
After Width: | Height: | Size: 184 KiB |
Binary file not shown.
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 59 KiB |
Binary file not shown.
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 59 KiB |
Loading…
Reference in New Issue
Block a user