mirror of
https://github.com/clearml/clearml-docs
synced 2025-04-02 12:21:08 +00:00
parent
d339f81ccb
commit
77ddd5b382
@ -37,12 +37,10 @@ VS Code remote sessions use ports 8878 and 8898 respectively.
|
||||
</div>
|
||||
</details>
|
||||
|
||||
<!---->
|
||||
|
||||
## How it Works
|
||||
|
||||
ClearML allows to leverage a resource (e.g. GPU or CPU machine) by utilizing the [ClearML Agent](../clearml_agent).
|
||||
A ClearML Agent will run on a target machine, and ClearML Session will instruct it to execute the Jupyter \ VS Code
|
||||
A ClearML Agent will run on a target machine, and ClearML Session will instruct it to execute the Jupyter / VS Code
|
||||
server to develop remotely.
|
||||
After entering a `clearml-session` command with all specifications:
|
||||
|
||||
|
@ -16,7 +16,7 @@ but can be overridden by command-line arguments.
|
||||
|**CLEARML_TASK_NO_REUSE** | Control Task reuse|
|
||||
|**CLEARML_CACHE_DIR** | Sets the location of the cache directory|
|
||||
|**CLEARML_DOCKER_IMAGE** | Sets the default docker image to run from|
|
||||
|**CLEARML_LOG_LEVEL** | debug \ warning \ error \ info | Sets the ClearML package's log verbosity|
|
||||
|**CLEARML_LOG_LEVEL** | debug / warning / error / info - Sets the ClearML package's log verbosity|
|
||||
|**CLEARML_SUPPRESS_UPDATE_MESSAGE** | Suppresses the message that notifies users of new ClearML package version|
|
||||
|
||||
### VCS
|
||||
@ -38,7 +38,7 @@ Overrides Repository Auto-logging
|
||||
|**CLEARML_FILES_HOST** | Sets the File Server URL
|
||||
|**CLEARML_API_ACCESS_KEY** | Sets the Server's Public Access Key|
|
||||
|**CLEARML_API_SECRET_KEY** | Sets the Server's Private Access Key|
|
||||
|**CLEARML_API_HOST_VERIFY_CERT**| Enables / Disables server certificate verification (If behind a firewall)|
|
||||
|**CLEARML_API_HOST_VERIFY_CERT**| Enables / Disables server certificate verification (if behind a firewall)|
|
||||
|**CLEARML_OFFLINE_MODE** | Sets Offline mode|
|
||||
|**CLEARML_NO_DEFAULT_SERVER** | Disables sending information to demo server when no HOST server is set|
|
||||
|
||||
@ -60,6 +60,6 @@ Overrides Repository Auto-logging
|
||||
|**CLEARML_AGENT_EXTRA_DOCKER_ARGS**| Overrides extra docker args configuration |
|
||||
|**CLEARML_AGENT_EXTRA_PYTHON_PATH**| Sets extra python path|
|
||||
|**CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE**| Overrides initial server connection behavior (true by default), allows explicit number to specify number of connect retries) |
|
||||
|**CLEARML_AGENT_K8S_HOST_MOUNT / CLEARML_AGENT_DOCKER_HOST_MOUNT**| Specifies Agent's mount point for Docker \ K8s|
|
||||
|**CLEARML_AGENT_K8S_HOST_MOUNT / CLEARML_AGENT_DOCKER_HOST_MOUNT**| Specifies Agent's mount point for Docker / K8s|
|
||||
|**CLEARML_AGENT_SKIP_PIP_VENV_INSTALL**| Skips Python virtual env installation on execute and provides a custom venv binary |
|
||||
|**CLEARML_AGENT_VENV_CACHE_PATH**|Overrides venv cache folder configuration|
|
@ -8,9 +8,9 @@ ClearML logs hyperparameters used in experiments from multiple different sources
|
||||
In ClearML, parameters are split into 3 sections:
|
||||
- User Properties - Modifiable section that can be edited post execution.
|
||||
- Hyperparameters - Individual parameters for configuration.
|
||||
- Configuration Objects - Usually configuration files (Json \ YAML) or python objects.
|
||||
- Configuration Objects - Usually configuration files (Json / YAML) or python objects.
|
||||
|
||||
These sections are further broken down into sub-sections (General \ Args \ TF_Define) for convenience.
|
||||
These sections are further broken down into sub-sections (General / Args / TF_Define) for convenience.
|
||||
|
||||

|
||||
|
||||
|
@ -71,7 +71,7 @@ ClearML also supports reporting media (such as audio, video and images) for ever
|
||||
This section is mostly used for debugging. It's recommended to use [artifacts](artifacts.md#artifacts) for storing script
|
||||
outputs that would be used later on.
|
||||
|
||||
Only the last X results of each title \ series are saved to prevent overloading the server.
|
||||
Only the last X results of each title / series are saved to prevent overloading the server.
|
||||
See details in [Logger.report_media](../references/sdk/logger.md#report_media).
|
||||
|
||||

|
||||
|
@ -226,7 +226,7 @@ a_task = Task.get_task(project_name='examples', task_name='artifacts')
|
||||
Once a Task object is obtained, it's possible to query the state of the Task, reported scalars, etc.
|
||||
The Task's outputs, such as artifacts and models, can also be retrieved.
|
||||
|
||||
### Querying \ Searching Tasks
|
||||
### Querying / Searching Tasks
|
||||
|
||||
Searching and filtering Tasks can be done via the [web UI](../webapp/webapp_overview.md), but also programmatically.
|
||||
Input search parameters into the `Task.get_tasks` method, which returns a list of Task objects that match the search.
|
||||
|
@ -40,7 +40,7 @@ Check [this](../../fundamentals/hyperparameters.md) out for all Hyperparameter l
|
||||
|
||||
## Log Artifacts
|
||||
|
||||
ClearML allows you to easily store the output products of an experiment - Model snapshot \ weights file, a preprocessing of your data, feature representation of data and more!
|
||||
ClearML allows you to easily store the output products of an experiment - Model snapshot / weights file, a preprocessing of your data, feature representation of data and more!
|
||||
|
||||
Essentially, artifacts are files (or python objects) uploaded from a script and are stored alongside the Task.
|
||||
These Artifacts can be easily accessed by the web UI or programmatically.
|
||||
|
@ -58,7 +58,7 @@ new_dataset.tags = ['latest']
|
||||
|
||||
We passed the `parents` argument when we created v2 of the Dataset, this inherits all the parent's version content.
|
||||
This will not only help us in tracing back dataset changes with full genealogy, but will also make our storage more efficient,
|
||||
as it will only store the files that were changed \ added from the parent versions.
|
||||
as it will only store the files that were changed / added from the parent versions.
|
||||
When we will later need access to the Dataset it will automatically merge the files from all parent versions
|
||||
in a fully automatic and transparent process, as if they were always part of the requested Dataset.
|
||||
|
||||
|
@ -218,7 +218,7 @@ In the Step 3 Task ([step3_train_model.py](https://github.com/allegroai/clearml/
|
||||
|
||||
* Run the script.
|
||||
|
||||
python pipeline_controller.py
|
||||
python pipeline_from_tasks.py
|
||||
|
||||
* Remotely execute the Task - If the Task `pipeline demo` in the project `examples` already exists in ClearML Server, clone it and enqueue it to execute.
|
||||
|
||||
|
@ -57,7 +57,7 @@ executing_pipeline(
|
||||
|
||||
By default, the pipeline controller and the pipeline steps are launched through ClearML [queues](../../fundamentals/agents_and_queues.md#what-is-a-queue).
|
||||
Use the [`PipelineDecorator.set_default_execution_queue`](../../references/sdk/automation_controller_pipelinecontroller.md#pipelinedecoratorset_default_execution_queue)
|
||||
method to specify the execution queue of all pipeline steps. The` execution_queue` parameter of the `PipelineDecorator.component`
|
||||
method to specify the execution queue of all pipeline steps. The `execution_queue` parameter of the `@PipelineDecorator.component`
|
||||
decorator overrides the default queue value for the specific step for which it was specified.
|
||||
|
||||
:::note Execution Modes
|
||||
@ -95,7 +95,7 @@ Each function step’s arguments are stored in their respective task’s **CONFI
|
||||

|
||||
|
||||
|
||||
Values that were listed in the `return_values`parameter of the `PipelineDecorator.component` decorator are stored as
|
||||
Values that were listed in the `return_values`parameter of the `@PipelineDecorator.component` decorator are stored as
|
||||
artifacts in the relevant step's task. These artifacts can be viewed in the step task’s **ARTIFACTS** tab.
|
||||
|
||||

|
||||
|
@ -19,7 +19,7 @@ example script from ClearML's GitHub repo:
|
||||
|
||||
## Before Starting
|
||||
|
||||
Make a copy of [`pytorch_mnist.py`](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
|
||||
Make a copy of [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py)
|
||||
in order to add explicit reporting to it.
|
||||
|
||||
```bash
|
||||
|
@ -134,7 +134,7 @@ Add, change, or delete hyperparameters, which are organized in the **ClearML Web
|
||||
|
||||
* Environment variables - Tracked if the `CLEARML_LOG_ENVIRONMENT` environment variable was set (see this [FAQ](../faq#track-env-vars)).
|
||||
|
||||
* Custom named parameter groups (see the `name` parameter in [Task.connect](../references/sdk/task.md#connectmutable-namenone)).
|
||||
* Custom named parameter groups (see the `name` parameter in [Task.connect](../references/sdk/task.md#connect)).
|
||||
|
||||
**To add, change, or delete hyperparameters:**
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user