mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-03 02:32:49 +00:00
Small edits (#534)
This commit is contained in:
pollfly
GitHub
parent
48e0a1d453
commit
1a098e9b09
@ -151,7 +151,7 @@ Pass one of the following in the `continue_last_task` parameter:
|
||||
[Task Reuse](#task-reuse)).
|
||||
* `True` - Continue the previously run Task.
|
||||
* Task ID (string) - The ID of the task to be continued.
|
||||
* Initial iteration offset (Integer) - Specify the initial iteration offset. By default, the task will continue one
|
||||
* Initial iteration offset (integer) - Specify the initial iteration offset. By default, the task will continue one
|
||||
iteration after the last reported one. Pass `0`, to disable the automatic last iteration offset. To also specify a
|
||||
task ID, use the `reuse_last_task_id` parameter .
|
||||
|
||||
|
||||
@ -434,7 +434,7 @@ match_rules: [
|
||||
|
||||
**`agent.package_manager.conda_env_as_base_docker`** (*bool*)
|
||||
|
||||
* Uses conda environment for execution (Like a docker)
|
||||
* Uses conda environment for execution (like a docker)
|
||||
|
||||
___
|
||||
|
||||
|
||||
@ -46,7 +46,7 @@ that we need.
|
||||
- [ClearML SDK](../../clearml_sdk/clearml_sdk.md) ensures that all the metrics, parameters and Models are automatically logged and can later be
|
||||
accessed, [compared](../../webapp/webapp_exp_comparing.md) and [tracked](../../webapp/webapp_exp_track_visual.md).
|
||||
- [ClearML Agent](../../clearml_agent.md) does the heavy lifting. It reproduces the execution environment, clones your code,
|
||||
applies code patches, manages parameters (Including overriding them on the fly), executes the code and queues multiple tasks
|
||||
applies code patches, manages parameters (including overriding them on the fly), executes the code and queues multiple tasks
|
||||
It can even [build](../../clearml_agent.md#exporting-a-task-into-a-standalone-docker-container) the docker container for you!
|
||||
- [ClearML Pipelines](../../pipelines/pipelines.md) ensure that steps run in the same order,
|
||||
programmatically chaining tasks together, while giving an overview of the execution pipeline's status.
|
||||
|
||||
@ -11,7 +11,7 @@ This tutorial goes over how to create a ClearML worker node in a Google Colab no
|
||||
and running, users can send Tasks to be executed on Google Colab's HW.
|
||||
|
||||
## Prerequisites
|
||||
* Be signed up for ClearML (Or have a server deployed).
|
||||
* Be signed up for ClearML (or have a server deployed).
|
||||
* Have a Google account to access Google Colab
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 64 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 50 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 51 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 49 KiB |
@ -1 +0,0 @@
|
||||
<?xml version="1.0" encoding="utf-8"?> <!-- Generator: IcoMoon.io --> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <svg version="1.1" id="Layer_2_copy" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" width="16" height="16" viewBox="0 0 16 16" enable-background="new 0 0.028 24 16" xml:space="preserve" data-license="&nbsp;" data-author="&nbsp;" fill="#666666"> <g><path d="M 16,14l-4-4.017c 0.641-1.008, 1-2.201, 1-3.483c0-3.589-2.912-6.5-6.5-6.5c-3.589,0-6.5,2.911-6.5,6.5 s 2.923,6.5, 6.512,6.5c 1.287,0, 2.487-0.355, 3.495-1L 14,15.993L 16,14z M 3,10l0-7 l 7,0 l0,7 L 3,10 z M 4,4l 5,0 l0,5 L 4,9 L 4,4 z"></path></g></svg>
|
||||
|
Before Width: | Height: | Size: 735 B |
Binary file not shown.
|
Before Width: | Height: | Size: 446 KiB After Width: | Height: | Size: 276 KiB |
@ -1,14 +0,0 @@
|
||||
<svg width="24" height="24" xmlns="http://www.w3.org/2000/svg">
|
||||
|
||||
<g>
|
||||
<title>background</title>
|
||||
<rect fill="none" id="canvas_background" height="26" width="26" y="-1" x="-1"/>
|
||||
</g>
|
||||
<g>
|
||||
<title>Layer 1</title>
|
||||
<g id="svg_1" fill-rule="evenodd" fill="none">
|
||||
<path id="svg_2" d="m5.874997,0.125l18,0l0,18l-18,0l0,-18z"/>
|
||||
<path id="svg_3" opacity="0.3" d="m17.874997,6.125c-0.148,0.11 -0.315,0 -0.75,0l-10.5,0c-0.435,0 -0.602,0.11 -0.75,0c-0.045,0.565 -0.006,0.764 0,0.75l4.5,4.5l0,3.75c-0.134,0.42 -0.08,0.548 0,0.75l2.25,2.25c0.058,-0.054 0.186,0 0,0c0.407,0 0.478,-0.014 0.75,0c0.023,-0.14 0.134,-0.307 0,-0.75l0,-6l4.5,-4.5c0.006,0.014 0.045,-0.185 0,-0.75z" fill-rule="nonzero" fill="#8492C2"/>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 730 B |
@ -1,3 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="40" height="40" viewBox="-5 -5 50 50">
|
||||
<path fill="#5A658E" fill-rule="evenodd" d="M23 17h17v6H23v17h-6V23H0v-6h17V0h6v17z"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 184 B |
@ -1,10 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="28" height="25" viewBox="0 0 28 25">
|
||||
<g fill="none" fill-rule="evenodd" opacity=".8">
|
||||
<path d="M-18-17h64v64h-64z"/>
|
||||
<g fill="#8492C2" opacity=".8">
|
||||
<rect width="14" height="11.375" x="7" opacity=".4" rx="2"/>
|
||||
<rect width="21" height="14.625" x="3.5" y="3.25" opacity=".5" rx="2"/>
|
||||
<rect width="28" height="16.25" y="8.125" rx="2"/>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 465 B |
@ -1,11 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 26 26.004">
|
||||
<defs>
|
||||
<style>
|
||||
.cls-1 {
|
||||
fill: #636e97;
|
||||
fill-rule: evenodd;
|
||||
}
|
||||
</style>
|
||||
</defs>
|
||||
<path id="np_settings_1215011_000000" class="cls-1" d="M17.509,5.992a.614.614,0,0,0-.157.028l-3.416.914a.561.561,0,0,0-.2.1.6.6,0,0,0-.157.166.606.606,0,0,0-.074.222.559.559,0,0,0,.018.231l.776,2.9a9.626,9.626,0,0,0-2.234,1.717l-2.6-1.5a.583.583,0,0,0-.212-.074.6.6,0,0,0-.434.12.616.616,0,0,0-.157.175L6.891,14.043a.606.606,0,0,0-.074.222.6.6,0,0,0,.009.231.561.561,0,0,0,.1.2.6.6,0,0,0,.175.148l2.6,1.5a9.642,9.642,0,0,0-.369,2.788l-2.9.785a.6.6,0,0,0-.212.1.55.55,0,0,0-.148.166A.628.628,0,0,0,6,20.414a.559.559,0,0,0,.018.231l.914,3.416a.561.561,0,0,0,.1.2.616.616,0,0,0,.175.157.562.562,0,0,0,.212.074.625.625,0,0,0,.231-.018l2.9-.776a9.553,9.553,0,0,0,1.717,2.225l-1.5,2.6a.606.606,0,0,0-.074.222.6.6,0,0,0,.12.434.566.566,0,0,0,.166.148l3.065,1.773a.583.583,0,0,0,.452.055.561.561,0,0,0,.2-.1.616.616,0,0,0,.157-.175l1.5-2.594a9.592,9.592,0,0,0,2.788.369l.776,2.9a.561.561,0,0,0,.1.2.616.616,0,0,0,.175.157.583.583,0,0,0,.452.055l3.416-.914a.561.561,0,0,0,.2-.1.584.584,0,0,0,.148-.175.555.555,0,0,0,.074-.212.6.6,0,0,0-.009-.231l-.785-2.9a9.57,9.57,0,0,0,2.234-1.717l2.6,1.5a.606.606,0,0,0,.222.074.625.625,0,0,0,.231-.018.561.561,0,0,0,.2-.1.633.633,0,0,0,.157-.175l1.764-3.056a.628.628,0,0,0,.074-.222.6.6,0,0,0-.12-.434.6.6,0,0,0-.166-.157l-2.6-1.5a9.5,9.5,0,0,0,.36-2.77l2.908-.794a.59.59,0,0,0,.2-.1.616.616,0,0,0,.157-.175A.583.583,0,0,0,32,17.579a.559.559,0,0,0-.018-.231l-.914-3.416a.561.561,0,0,0-.1-.2.616.616,0,0,0-.175-.157.628.628,0,0,0-.222-.074.559.559,0,0,0-.231.018l-2.9.776a9.609,9.609,0,0,0-1.708-2.234l1.5-2.6a.583.583,0,0,0,.074-.212.56.56,0,0,0-.286-.591L23.953,6.888a.606.606,0,0,0-.222-.074.6.6,0,0,0-.231.009.561.561,0,0,0-.2.1.616.616,0,0,0-.157.175l-1.5,2.6a9.659,9.659,0,0,0-2.8-.369l-.776-2.9a.586.586,0,0,0-.2-.314.612.612,0,0,0-.36-.129Zm1.5,7.035a5.966,5.966,0,1,1-5.189,2.982,5.943,5.943,0,0,1,5.189-2.982Z" transform="translate(-6 -5.992)"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 2.0 KiB |
@ -1,36 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 26.93 26.93">
|
||||
<defs>
|
||||
<style>
|
||||
.a{fill:none;}.b{clip-path:url(#a);}.c{fill:#8f9dc9;}.d{clip-path:url(#b);}.e{clip-path:url(#c);}.f{clip-path:url(#d);}
|
||||
</style>
|
||||
<clipPath id="a">
|
||||
<rect class="a" x="1.46" y="1.46" width="2.18" height="2.18"/>
|
||||
</clipPath>
|
||||
<clipPath id="b">
|
||||
<rect class="a" x="23.28" y="1.46" width="2.18" height="2.18"/>
|
||||
</clipPath>
|
||||
<clipPath id="c">
|
||||
<rect class="a" x="1.46" y="23.28" width="2.18" height="2.18"/>
|
||||
</clipPath>
|
||||
<clipPath id="d">
|
||||
<rect class="a" x="23.28" y="23.28" width="2.18" height="2.18"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
<title>sidebar-annotator-icon</title>
|
||||
<g class="b">
|
||||
<path class="c" d="M0,5.11H5.11V0H0Z"/>
|
||||
</g>
|
||||
<g class="d">
|
||||
<path class="c" d="M21.82,5.11h5.11V0H21.82Z"/>
|
||||
</g>
|
||||
<path class="a" d="M2.55,2.55H24.37V24.37H2.55Z"/>
|
||||
<path class="c" d="M3.55,3.55V23.37H23.37V3.55H3.55m-1-1H24.37V24.37H2.55Z"/>
|
||||
<g class="e">
|
||||
<path class="c" d="M0,26.93H5.11V21.82H0Z"/>
|
||||
</g>
|
||||
<g class="f">
|
||||
<path class="c" d="M21.82,26.93h5.11V21.82H21.82Z"/>
|
||||
</g>
|
||||
<path class="c"
|
||||
d="M21.39,8.38l-1.9-.77a1.61,1.61,0,0,0-1.18-1.25l-.1-1.19a.42.42,0,0,0-.29-.36.41.41,0,0,0-.45.12l-.5.54-.3-1.06a.41.41,0,0,0-.39-.31.42.42,0,0,0-.42.28l-2.62,7.25A7.65,7.65,0,0,0,8.68,15a6.37,6.37,0,0,0-.81,4.87,3.83,3.83,0,0,0,1.15,2H8.65a4.36,4.36,0,0,1-4-2.48.42.42,0,0,0-.56-.2h0a.42.42,0,0,0-.2.57,5.2,5.2,0,0,0,4.7,3h8.12a.42.42,0,0,0,.43-.42,2,2,0,0,0-1.72-1.86,3.26,3.26,0,0,0,.36-1.81l.12,0L18.4,22.5a.44.44,0,0,0,.36.19H20.3a.43.43,0,0,0,.38-.22.42.42,0,0,0,0-.43s-1.61-2.57-2.47-4V10.88l.39,0A3.39,3.39,0,0,0,21.6,9l0-.05A.42.42,0,0,0,21.39,8.38Z"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 1.7 KiB |
@ -1,10 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24">
|
||||
<defs>
|
||||
<style>.a{fill:#8f9dc9;}</style>
|
||||
</defs>
|
||||
<title>sidebar-data-icon</title>
|
||||
<path class="a"
|
||||
d="M17.6,4.9H1.3A1.24,1.24,0,0,0,0,6.09V22.3A1.65,1.65,0,0,0,1.5,24H17.4a1.74,1.74,0,0,0,1.7-1.7V6.3A1.5,1.5,0,0,0,17.6,4.9ZM4.9,8.5a1.5,1.5,0,0,1,0,3A1.54,1.54,0,0,1,3.4,10,1.48,1.48,0,0,1,4.84,8.5ZM1.7,20.3l4.6-6,2.2,2.8,3.4-4.8,5.6,8Z"/>
|
||||
<path class="a"
|
||||
d="M21.8,0H6.9C6.4,0,6,.6,6,1.4s.4,1.4.9,1.4H19A2.22,2.22,0,0,1,21.2,5V17c0,.5.6,1,1.4,1s1.4-.4,1.4-1V2.2A2.22,2.22,0,0,0,21.8,0Z"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 585 B |
@ -1,7 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 20">
|
||||
<defs>
|
||||
<style>.a{fill:#8f9dc9;}</style>
|
||||
</defs>
|
||||
<title>sidebar-home-icon</title>
|
||||
<path class="a" d="M19.68,7.77V2.14H16.32V4.37L12,0,0,12.14H2.88V20h7.44V13.57h3.36V20h7.44V12.14H24Z"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 266 B |
@ -1,10 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 26 24.46">
|
||||
<defs>
|
||||
<style>.a{fill:#8f9dc9;}</style>
|
||||
</defs>
|
||||
<title>sidebar-projects-icon</title>
|
||||
<path class="a"
|
||||
d="M5.08,13.61a1.42,1.42,0,0,1-1.26-.81H0a3.82,3.82,0,0,0,2.88,3.25A4,4,0,0,0,5,20.6a3.93,3.93,0,0,0,7.39,1.79V15.93H9.59v1.66a1.36,1.36,0,0,1,.81,1.26,1.33,1.33,0,0,1-1.31,1.34h0a1.35,1.35,0,0,1-1.34-1.34,1.45,1.45,0,0,1,.81-1.26V14.83h3.94V10H9.34V7.8H8a1.34,1.34,0,0,1-1.26.81,1.34,1.34,0,0,1-.07-2.68h.07A1.43,1.43,0,0,1,8,6.74h2.4v2.2h2.08V2.07A4.07,4.07,0,0,0,9,0,3.92,3.92,0,0,0,5,3.86,3.93,3.93,0,0,0,2.92,8.41,4,4,0,0,0,0,11.66H3.9a1.35,1.35,0,0,1,1.26-.81A1.33,1.33,0,0,1,6.5,12.16v0a1.44,1.44,0,0,1-1.42,1.42Z"/>
|
||||
<path class="a"
|
||||
d="M23.08,8.41A4,4,0,0,0,21,3.86a3.93,3.93,0,0,0-7.39-1.79V5h2.68a1.35,1.35,0,0,1,1.26-.81,1.32,1.32,0,0,1,1.34,1.31v0a1.36,1.36,0,0,1-1.34,1.34,1.43,1.43,0,0,1-1.26-.81H13.53v13H15.6V16.53h2.48a1.36,1.36,0,1,1,1.33,1.87h-.07a1.42,1.42,0,0,1-1.26-.81H16.66v2.56H13.53V22.3A3.86,3.86,0,0,0,17,24.37a3.94,3.94,0,0,0,3.94-3.86A3.91,3.91,0,0,0,23,16c.17,0,.29-.12.45-.16V13.65H19V10.93A1.37,1.37,0,1,1,20.84,9.6v.07A1.43,1.43,0,0,1,20,10.93v1.62h4.51v2.6a3.58,3.58,0,0,0,1.46-3A4,4,0,0,0,23.08,8.41Z"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 1.2 KiB |
@ -1,8 +0,0 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 23.22 24">
|
||||
<defs>
|
||||
<style>.a{fill:#8f9dc9;}</style>
|
||||
</defs>
|
||||
<title>sidebar-workers-icon</title>
|
||||
<path class="a"
|
||||
d="M22.72,8H.5a.5.5,0,0,0-.5.51H0v6.19a.5.5,0,0,0,.5.51H22.72a.5.5,0,0,0,.5-.51h0V8.5a.51.51,0,0,0-.5-.52ZM4,12.71A1.12,1.12,0,1,1,5.1,11.59h0A1.12,1.12,0,0,1,4,12.71Zm4.22,0a1.12,1.12,0,1,1,1.12-1.12A1.12,1.12,0,0,1,8.2,12.71ZM21,12.43a.26.26,0,0,1-.26.26h-5a.26.26,0,0,1-.26-.26V10.75a.26.26,0,0,1,.26-.25h5a.26.26,0,0,1,.26.25ZM22.72,0H.5A.5.5,0,0,0,0,.51H0V6.71a.5.5,0,0,0,.5.51H22.72a.5.5,0,0,0,.5-.51h0V.52a.51.51,0,0,0-.5-.52ZM4,4.73A1.12,1.12,0,1,1,5.1,3.61h0A1.12,1.12,0,0,1,4,4.73Zm4.22,0A1.12,1.12,0,1,1,9.32,3.61h0A1.13,1.13,0,0,1,8.2,4.73ZM21,4.45a.26.26,0,0,1-.26.26h-5a.26.26,0,0,1-.26-.26V2.77a.26.26,0,0,1,.26-.25h5a.26.26,0,0,1,.26.25ZM22.72,16H.5a.5.5,0,0,0-.5.51H0v6.19a.5.5,0,0,0,.5.51H3.37v.56a.26.26,0,0,0,.26.26H7.5a.26.26,0,0,0,.26-.26h0v-.56h7.7v.56a.26.26,0,0,0,.25.26h3.88a.26.26,0,0,0,.25-.26h0v-.56h2.88a.5.5,0,0,0,.5-.51h0V16.48a.51.51,0,0,0-.5-.52ZM4,20.69A1.12,1.12,0,1,1,5.1,19.57h0A1.12,1.12,0,0,1,4,20.69Zm4.22,0a1.12,1.12,0,1,1,1.12-1.12A1.12,1.12,0,0,1,8.2,20.69ZM21,20.41a.26.26,0,0,1-.26.26h-5a.26.26,0,0,1-.26-.26V18.73a.26.26,0,0,1,.26-.25h5a.26.26,0,0,1,.26.25Z"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 1.3 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 58 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 9.9 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 28 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 68 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 58 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 78 KiB |
@ -87,7 +87,7 @@ def step_one(pickle_data_url: str, extra: int = 43):
|
||||
### Arguments
|
||||
* `return_values` - The artifact names for the step’s corresponding ClearML task to store the step’s returned objects.
|
||||
In the example above, a single object is returned and stored as an artifact named `data_frame`
|
||||
* `name` (Optional) - The name for the pipeline step. If not provided, the function name is used
|
||||
* `name` (optional) - The name for the pipeline step. If not provided, the function name is used
|
||||
* `cache` - If `True`, the pipeline controller checks if a step with the same code (including setup, see task [Execution](../webapp/webapp_exp_track_visual.md#execution)
|
||||
section) and input arguments was already executed. If found, the cached step's outputs are used
|
||||
instead of rerunning the step.
|
||||
@ -101,8 +101,8 @@ def step_one(pickle_data_url: str, extra: int = 43):
|
||||
* `docker` - Specify the Docker image to be used when executing the pipeline step remotely
|
||||
* `docker_args` - Add Docker execution arguments for the remote execution (use single string for all docker arguments).
|
||||
* `docker_bash_setup_script` - Add a bash script to be executed inside the docker before setting up the Task's environment
|
||||
* `task_type` (Optional) - The [task type](../fundamentals/task.md#task-types) to be created
|
||||
* `repo` (Optional) - Specify a repository to attach to the function when remotely executing. Allow users to execute the function inside the specified repository, enabling them to load modules / script from the repository. Notice the execution work directory will be the repository root folder. Supports both git repo url link, and local repository path (automatically converted into the remote git / commit as is currently checkout).
|
||||
* `task_type` (optional) - The [task type](../fundamentals/task.md#task-types) to be created
|
||||
* `repo` (optional) - Specify a repository to attach to the function when remotely executing. Allow users to execute the function inside the specified repository, enabling them to load modules / script from the repository. Notice the execution work directory will be the repository root folder. Supports both git repo url link, and local repository path (automatically converted into the remote git / commit as is currently checkout).
|
||||
* Examples:
|
||||
* remote url: `"https://github.com/user/repo.git"`
|
||||
* local repo copy: `"./repo"` -> will automatically store the remote repo url and commit ID based on the locally cloned copy
|
||||
|
||||
@ -94,7 +94,7 @@ pipe.add_step(
|
||||
* `cache_executed_step` – If `True`, the controller will check if an identical task with the same code (including setup,
|
||||
e.g. required packages, docker image, etc.) and input arguments was already executed. If found, the cached step's
|
||||
outputs are used instead of launching a new task.
|
||||
* `execution_queue` (Optional) - the queue to use for executing this specific step. If not provided, the task will be sent to the default execution queue, as defined on the class
|
||||
* `execution_queue` (optional) - the queue to use for executing this specific step. If not provided, the task will be sent to the default execution queue, as defined on the class
|
||||
* `parents` – Optional list of parent steps in the pipeline. The current step in the pipeline will be sent for execution only after all the parent steps have been executed successfully.
|
||||
* `parameter_override` - Dictionary of parameters and values to override in the current step. See [parameter_override](#parameter_override).
|
||||
* `configuration_overrides` - Dictionary of configuration objects and values to override in the current step. See [configuration_overrides](#configuration_overrides)
|
||||
@ -209,19 +209,19 @@ def step_completed_callback(
|
||||
|
||||
You can enable automatic logging of a step’s metrics /artifacts / models to the pipeline task using the following arguments:
|
||||
|
||||
* `monitor_metrics` (Optional) - Automatically log the step's reported metrics also on the pipeline Task. The expected
|
||||
* `monitor_metrics` (optional) - Automatically log the step's reported metrics also on the pipeline Task. The expected
|
||||
format is one of the following:
|
||||
* List of pairs metric (title, series) to log: [(step_metric_title, step_metric_series), ]. Example: `[('test', 'accuracy'), ]`
|
||||
* List of tuple pairs, to specify a different target metric to use on the pipeline Task: [((step_metric_title, step_metric_series), (target_metric_title, target_metric_series)), ].
|
||||
Example: `[[('test', 'accuracy'), ('model', 'accuracy')], ]`
|
||||
* `monitor_artifacts` (Optional) - Automatically log the step's artifacts on the pipeline Task.
|
||||
* `monitor_artifacts` (optional) - Automatically log the step's artifacts on the pipeline Task.
|
||||
* Provided a list of artifact names created by the step function, these artifacts will be logged automatically also
|
||||
on the Pipeline Task itself. Example: `['processed_data', ]` (target artifact name on the Pipeline Task will have
|
||||
the same name as the original artifact).
|
||||
* Alternatively, provide a list of pairs (source_artifact_name, target_artifact_name), where the first string is the
|
||||
artifact name as it appears on the step Task, and the second is the target artifact name to put on the Pipeline
|
||||
Task. Example: `[('processed_data', 'final_processed_data'), ]`
|
||||
* `monitor_models` (Optional) - Automatically log the step's output models on the pipeline Task.
|
||||
* `monitor_models` (optional) - Automatically log the step's output models on the pipeline Task.
|
||||
* Provided a list of model names created by the step's Task, they will also appear on the Pipeline itself. Example: `['model_weights', ]`
|
||||
* To select the latest (lexicographic) model, use `model_*`, or the last created model with just `*`. Example: `['model_weights_*', ]`
|
||||
* Alternatively, provide a list of pairs (source_model_name, target_model_name), where the first string is the model
|
||||
|
||||
@ -31,11 +31,11 @@ each instance is spun up.
|
||||
repositories to retrieve the code for their jobs
|
||||
* Git User
|
||||
* Git Password / Personal Access Token
|
||||
* **Max Idle Time** (Optional) - Maximum time in minutes that an EC2 instance can be idle before the autoscaler spins it
|
||||
* **Max Idle Time** (optional) - Maximum time in minutes that an EC2 instance can be idle before the autoscaler spins it
|
||||
down
|
||||
* **Workers Prefix** (Optional) - A Prefix added to workers’ names, associating them with this autoscaler
|
||||
* **Polling Interval** (Optional) - Time period in minutes at which the designated queue is polled for new tasks
|
||||
* **Base Docker Image** (Optional) - Default Docker image in which the ClearML Agent will run. Provide a Docker stored
|
||||
* **Workers Prefix** (optional) - A Prefix added to workers’ names, associating them with this autoscaler
|
||||
* **Polling Interval** (optional) - Time period in minutes at which the designated queue is polled for new tasks
|
||||
* **Base Docker Image** (optional) - Default Docker image in which the ClearML Agent will run. Provide a Docker stored
|
||||
in a Docker artifactory so instances can automatically fetch it
|
||||
* **Compute Resources**
|
||||
* Resource Name - Assign a name to the resource type. This name will appear in the Autoscaler dashboard
|
||||
@ -49,24 +49,24 @@ each instance is spun up.
|
||||
* Max Number of Instances - Maximum number of concurrent running instances of this type allowed
|
||||
* Monitored Queue - Queue associated with this instance type. The tasks enqueued to this queue will be executed on
|
||||
instances of this type
|
||||
* EC2 Tags (Optional) - AWS instance tags to attach to launched EC2 instances. Insert key=value pairs, separated by
|
||||
* EC2 Tags (optional) - AWS instance tags to attach to launched EC2 instances. Insert key=value pairs, separated by
|
||||
commas
|
||||
* EBS Device (Optional) - Disk mount point
|
||||
* EBS Volume Size (Optional) - Disk size (GB)
|
||||
* EBS Volume Type (Optional) - See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html)
|
||||
* EBS Device (optional) - Disk mount point
|
||||
* EBS Volume Size (optional) - Disk size (GB)
|
||||
* EBS Volume Type (optional) - See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html)
|
||||
for full list of types
|
||||
* Instance Key Pair (Optional) - AWS key pair that is provided to the spun EC2 instances for connecting to them via
|
||||
* Instance Key Pair (optional) - AWS key pair that is provided to the spun EC2 instances for connecting to them via
|
||||
SSH. Provide the Key Pair's name, as was created in AWS. See [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)
|
||||
for more details
|
||||
* Security Group ID (Optional) - Comma separated list of AWS VPC Security Group IDs to attach to the launched
|
||||
* Security Group ID (optional) - Comma separated list of AWS VPC Security Group IDs to attach to the launched
|
||||
instance. Read more [here](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html)
|
||||
* \+ Add Item - Define another resource type
|
||||
* **IAM Instance Profile** (Optional) - Set an IAM instance profile for all instances spun by the Autoscaler
|
||||
* **IAM Instance Profile** (optional) - Set an IAM instance profile for all instances spun by the Autoscaler
|
||||
* Arn - Amazon Resource Name specifying the instance profile
|
||||
* Name - Name identifying the instance profile
|
||||
* **Autoscaler Instance Name** (Optional) - Name for the Autoscaler instance. This will appear in the instance list
|
||||
* **Init script** (Optional) - A bash script to execute after launching the EC2 instance
|
||||
* **Additional ClearML Configuration** (Optional) - A ClearML configuration file to use by the ClearML Agent when
|
||||
* **Autoscaler Instance Name** (optional) - Name for the Autoscaler instance. This will appear in the instance list
|
||||
* **Init script** (optional) - A bash script to execute after launching the EC2 instance
|
||||
* **Additional ClearML Configuration** (optional) - A ClearML configuration file to use by the ClearML Agent when
|
||||
executing your experiments
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create
|
||||
a new instance with the same configuration
|
||||
|
||||
@ -19,7 +19,7 @@ on failed Tasks via Slack integration.
|
||||
* Monitored Metric - Series - Metric series (variant) to track
|
||||
* Monitored Metric - Trend - Choose whether to track the monitored metric's highest or lowest values
|
||||
* **Dashboard Title** - Name of the project dashboard instance, which will appear in the instance list
|
||||
* **Failed Task Slack Monitor** (Optional)
|
||||
* **Failed Task Slack Monitor** (optional)
|
||||
* API Token - Slack workspace access token
|
||||
* Channel Name - Slack channel to which task failure alerts will be posted
|
||||
* Fail Iteration Threshold - Minimum number of iterations to trigger Slack alerts about task failure (failed tasks that do not meet the threshold will be ignored)
|
||||
|
||||
@ -30,7 +30,7 @@ when each VM instance is spun up.
|
||||
repositories to retrieve the code for their jobs
|
||||
* Git User
|
||||
* Git Password / Personal Access Token
|
||||
* **Base Docker Image** (Optional) - Default Docker image in which the ClearML Agent will run. Provide a Docker stored in a
|
||||
* **Base Docker Image** (optional) - Default Docker image in which the ClearML Agent will run. Provide a Docker stored in a
|
||||
Docker artifactory so VM instances can automatically fetch it
|
||||
* **Compute Resources**
|
||||
* Resource Name - Assign a name to the resource type. This name will appear in the Autoscaler dashboard
|
||||
@ -40,15 +40,15 @@ when each VM instance is spun up.
|
||||
* Use Preemptible Instance - Choose whether VM instances of this type will be [preemptible](https://cloud.google.com/compute/docs/instances/preemptible)
|
||||
* Max Number of Instances - Maximum number of concurrent running VM instances of this type allowed
|
||||
* Monitored Queue - Queue associated with this VM instance type. The tasks enqueued to this queue will be executed on VM instances of this type
|
||||
* Machine Image (Optional) - The GCP machine image to launch
|
||||
* Disc Size (in GB) (Optional)
|
||||
* Machine Image (optional) - The GCP machine image to launch
|
||||
* Disc Size (in GB) (optional)
|
||||
* \+ Add Item - Define another resource type
|
||||
* **Autoscaler Instance Name** (Optional) - Name for the Autoscaler instance. This will appear in the instance list
|
||||
* **Max Idle Time** (Optional) - Maximum time in minutes that a VM instance can be idle before the autoscaler spins it down
|
||||
* **Workers Prefix** (Optional) - A Prefix added to workers’ names, associating them with this autoscaler
|
||||
* **Polling Interval** (Optional) - Time period in minutes at which the designated queue is polled for new tasks
|
||||
* **Init Script** (Optional) - A bash script to execute after launching the VM instance
|
||||
* **Additional ClearML Configuration** (Optional) - A ClearML configuration file to use by the ClearML Agent when executing your experiments
|
||||
* **Autoscaler Instance Name** (optional) - Name for the Autoscaler instance. This will appear in the instance list
|
||||
* **Max Idle Time** (optional) - Maximum time in minutes that a VM instance can be idle before the autoscaler spins it down
|
||||
* **Workers Prefix** (optional) - A Prefix added to workers’ names, associating them with this autoscaler
|
||||
* **Polling Interval** (optional) - Time period in minutes at which the designated queue is polled for new tasks
|
||||
* **Init Script** (optional) - A bash script to execute after launching the VM instance
|
||||
* **Additional ClearML Configuration** (optional) - A ClearML configuration file to use by the ClearML Agent when executing your experiments
|
||||
* **Export Configuration** - Export the app instance configuration as a JSON file, which you can later import to create
|
||||
a new instance with the same configuration.
|
||||
|
||||
|
||||
@ -27,13 +27,13 @@ queue. The app instance automatically terminates idle machines based on a specif
|
||||
* Monitored Queue - Queue associated with application instance. The tasks enqueued to this queue will be executed on
|
||||
machines of this specification
|
||||
* Cloud Machine Limit - Maximum number of concurrent machines to launch
|
||||
* **Idle Time Limit** (Optional) - Maximum time in minutes that a cloud machine can be idle before it is spun down
|
||||
* **Default Docker Image** (Optional) - Default Docker image in which the ClearML Agent will run. Provide a Docker stored
|
||||
* **Idle Time Limit** (optional) - Maximum time in minutes that a cloud machine can be idle before it is spun down
|
||||
* **Default Docker Image** (optional) - Default Docker image in which the ClearML Agent will run. Provide a Docker stored
|
||||
in a Docker artifactory so instances can automatically fetch it
|
||||
* **Git Configuration** - Git credentials with which the ClearML Agents running on your cloud instances will access your repositories to retrieve the code for their jobs
|
||||
* Git User
|
||||
* Git Password / Personal Access Token
|
||||
* **Cloud Storage Access** (Optional) - Access credentials to cloud storage service. Provides ClearML Tasks running on cloud
|
||||
* **Cloud Storage Access** (optional) - Access credentials to cloud storage service. Provides ClearML Tasks running on cloud
|
||||
machines access to your storage
|
||||
|
||||
|
||||
|
||||
@ -54,11 +54,11 @@ limits.
|
||||
Additionally, make sure that the initial experiment's `_allow_omegaconf_edit_` parameter is set to `False` (in experiment's
|
||||
**CONFIGURATION > HYPERPARAMETERS > Hydra**).
|
||||
:::
|
||||
* **Optimization Job Title** (Optional) - Name for the HPO instance. This will appear in the instance list
|
||||
* **Optimization Experiments Destination Project** (Optional) - The project where optimization tasks will be saved.
|
||||
* **Optimization Job Title** (optional) - Name for the HPO instance. This will appear in the instance list
|
||||
* **Optimization Experiments Destination Project** (optional) - The project where optimization tasks will be saved.
|
||||
Leave empty to use the same project as the Initial task.
|
||||
* **Maximum Concurrent Tasks** - The maximum number of simultaneously running optimization experiments
|
||||
* **Advanced Configuration** (Optional)
|
||||
* **Advanced Configuration** (optional)
|
||||
* Limit Total HPO Experiments - Maximum total number of optimization experiments
|
||||
* Number of Top Experiments to Save - Number of best performing experiments to save (the rest are archived)
|
||||
* Limit Single Experiment Running Time (Minutes) - Time limit per optimization experiment. Experiments will be
|
||||
|
||||
@ -45,7 +45,7 @@ The **DETAILS** tab includes deep comparisons of the following:
|
||||
**To Locate the Source Differences:**
|
||||
|
||||
Click the **DETAILS** tab **>** Expand highlighted sections, or, in the header, click <img src="/docs/latest/icons/ico-previous-diff.svg" alt="Up arrow" className="icon size-md" />
|
||||
(Previous diff) or <img src="/docs/latest/icons/ico-next-diff.svg" alt="Down arrow" className="icon size-md space-sm" /> (Next diff).
|
||||
(previous diff) or <img src="/docs/latest/icons/ico-next-diff.svg" alt="Down arrow" className="icon size-md space-sm" /> (next diff).
|
||||
|
||||
For example, in the image below, expanding **ARTIFACTS** **>** **Output Model** **>** **Model** shows that the model names
|
||||
are different.
|
||||
@ -68,8 +68,8 @@ The Values mode is a side-by-side comparison that shows hyperparameter value dif
|
||||
1. In the dropdown menu (on the upper left, next to **+ Add Experiments**), choose **Values**.
|
||||
1. To show only differences, move the **Hide Identical Fields** slider to on.
|
||||
1. Locate differences by either:
|
||||
* Clicking <img src="/docs/latest/icons/ico-previous-diff.svg" alt="Up arrow" className="icon size-md space-sm" /> (Previous diff) or
|
||||
<img src="/docs/latest/icons/ico-next-diff.svg" alt="Down arrow" className="icon size-md space-sm" /> (Next diff).
|
||||
* Clicking <img src="/docs/latest/icons/ico-previous-diff.svg" alt="Up arrow" className="icon size-md space-sm" /> (previous diff) or
|
||||
<img src="/docs/latest/icons/ico-next-diff.svg" alt="Down arrow" className="icon size-md space-sm" /> (next diff).
|
||||
* Scrolling to see highlighted hyperparameters.
|
||||
|
||||
For example, expanding **General** shows that the `batch_size` and `epochs` differ between the experiments.
|
||||
@ -190,8 +190,8 @@ first. Use the viewer / player to inspect images, audio, video samples and do an
|
||||
1. Locate debug samples by doing the following:
|
||||
|
||||
* Filter by metric. In the **Metric** list, choose a metric.
|
||||
* Show other iterations. Click <img src="/docs/latest/icons/ico-circle-older.svg" alt="Left arrow" className="icon size-md space-sm" /> (Older images),
|
||||
<img src="/docs/latest/icons/ico-circle-newer.svg" alt="Right arrow" className="icon size-md space-sm" /> (New images), or <img src="/docs/latest/icons/ico-circle-newest.svg" alt="right arrow, newest image" className="icon size-md space-sm" /> (Newest images).
|
||||
* Show other iterations. Click <img src="/docs/latest/icons/ico-circle-older.svg" alt="Left arrow" className="icon size-md space-sm" /> (older images),
|
||||
<img src="/docs/latest/icons/ico-circle-newer.svg" alt="Right arrow" className="icon size-md space-sm" /> (new images), or <img src="/docs/latest/icons/ico-circle-newest.svg" alt="right arrow, newest image" className="icon size-md space-sm" /> (newest images).
|
||||
* Click <img src="/docs/latest/icons/ico-disconnect.svg" alt="Sync selection" className="icon size-md space-sm" /> in
|
||||
order to synchronize iteration and metric selection across experiments. For example, if you select a metric for
|
||||
one experiment’s debug samples, the same metric will be automatically selected for the rest of the experiments in the comparison.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user