mirror of
				https://github.com/clearml/clearml-docs
				synced 2025-06-26 18:17:44 +00:00 
			
		
		
		
	Edit hyperparameter-metric comparison FAQ (#344)
This commit is contained in:
		
							parent
							
								
									845c73eae5
								
							
						
					
					
						commit
						dece7a9358
					
				
							
								
								
									
										34
									
								
								docs/faq.md
									
									
									
									
									
								
							
							
						
						
									
										34
									
								
								docs/faq.md
									
									
									
									
									
								
							| @ -37,7 +37,7 @@ title: FAQ | ||||
| **Graphs and Logs** | ||||
| 
 | ||||
| * [The first log lines are missing from the experiment console tab. Where did they go?](#first-log-lines-missing) | ||||
| * [Can I create a graph comparing hyperparameters vs model accuracy?](#compare-graph-parameters) | ||||
| * [How do I create a graph comparing hyperparameters vs model accuracy?](#compare-graph-parameters) | ||||
| * [I want to add more graphs, not just with TensorBoard. Is this supported?](#more-graph-types) | ||||
| * [How can I report more than one scatter 2D series on the same plot?](#multiple-scatter2D) | ||||
| 
 | ||||
| @ -525,24 +525,39 @@ info panel > CONSOLE tab, use the *Download full log* feature. | ||||
| 
 | ||||
| <br/> | ||||
| 
 | ||||
| **Can I create a graph comparing hyperparameters vs. model accuracy?** <a id="compare-graph-parameters"></a> | ||||
| **How do I create a graph comparing hyperparameters vs. model accuracy?** <a id="compare-graph-parameters"></a> | ||||
| 
 | ||||
| Yes! You can manually create a plot with a single point X-axis for the hyperparameter value, and Y-axis for the accuracy.  | ||||
| For example: | ||||
| You can use the UI's [experiment comparison features](webapp/webapp_exp_comparing.md) to compare the logged hyperparameter  | ||||
| and accuracy values of several experiments.  | ||||
| 
 | ||||
| In an experiment comparison page, under the **HYPER PARAMETERS** tab, you can view the experiments' hyperparameter values  | ||||
| in relation to a specific metric (e.g. accuracy) in a parallel coordinates plot.  | ||||
| 
 | ||||
| The image below show a parallel coordinates plot which displays the values of selected hyperparameters (`base_lr`,  | ||||
| `batch_size`, and `number_of_epochs`) and a performance metric (`accuracy`) of three experiments.  | ||||
| 
 | ||||
|  | ||||
| 
 | ||||
| You can also visualize the differences in a scatter plot. In each experiment whose values wil compared, report a plot  | ||||
| with a single point, x-axis for the hyperparameter value, and Y-axis for the accuracy.  | ||||
| 
 | ||||
| In the code below, the task reports a single-point scatter plot with `number_layers` as the x-axis and | ||||
| `accuracy` as the Y-axis : | ||||
| 
 | ||||
| ```python | ||||
| number_layers = 10 | ||||
| accuracy = 0.95 | ||||
| Task.current_task().get_logger().report_scatter2d( | ||||
|     "performance",  | ||||
|     "accuracy",  | ||||
|     title="performance",  | ||||
|     series="accuracy",  | ||||
|     iteration=0,  | ||||
|     mode='markers',  | ||||
|     scatter=[(number_layers, accuracy)] | ||||
| ) | ||||
| ``` | ||||
| 
 | ||||
| Assuming the hyperparameter is `number_layers` with current value `10`, and the `accuracy` for the trained model is `0.95`. Then, the experiment comparison graph shows: | ||||
| When these experiments are compared in the UI's experiment comparison, all the reported `performance/accuracy` values  | ||||
| are displayed in a single plot. | ||||
| 
 | ||||
|  | ||||
| 
 | ||||
| @ -552,14 +567,13 @@ Another option is a histogram chart: | ||||
| number_layers = 10 | ||||
| accuracy = 0.95 | ||||
| Task.current_task().get_logger().report_vector( | ||||
|     "performance",  | ||||
|     "accuracy",  | ||||
|     title="performance",  | ||||
|     series="accuracy",  | ||||
|     iteration=0,  | ||||
|     labels=['accuracy'], | ||||
|     values=[accuracy],  | ||||
|     xlabels=['number_layers %d' % number_layers] | ||||
| ) | ||||
| 
 | ||||
| ``` | ||||
| 
 | ||||
|  | ||||
|  | ||||
										
											Binary file not shown.
										
									
								
							| After Width: | Height: | Size: 184 KiB | 
										
											Binary file not shown.
										
									
								
							| Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 59 KiB | 
										
											Binary file not shown.
										
									
								
							| Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 59 KiB | 
		Loading…
	
		Reference in New Issue
	
	Block a user