mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-03 10:42:51 +00:00
Small edits (#888)
This commit is contained in:
parent
b377e940aa
commit
9cf5d480eb
@ -35,7 +35,7 @@ Single IP (domain) with the following open ports:
|
|||||||
|
|
||||||
### Subdomain Configuration
|
### Subdomain Configuration
|
||||||
|
|
||||||
Subdomain configuration with default http/s ports (`80` or `443`):
|
Subdomain configuration with default HTTP/S ports (`80` or `443`):
|
||||||
|
|
||||||
* Web application on subdomain: `app.*.*`
|
* Web application on subdomain: `app.*.*`
|
||||||
* API service on subdomain: `api.*.*`
|
* API service on subdomain: `api.*.*`
|
||||||
|
@ -107,7 +107,7 @@ in parallel (each task will register itself as a sub-agent, visible in the [Orch
|
|||||||
This mode is intended for running maintenance tasks. Some suitable tasks include:
|
This mode is intended for running maintenance tasks. Some suitable tasks include:
|
||||||
|
|
||||||
- [Pipeline controller](../guides/pipeline/pipeline_controller.md) - Implementing the pipeline scheduling and logic
|
- [Pipeline controller](../guides/pipeline/pipeline_controller.md) - Implementing the pipeline scheduling and logic
|
||||||
- [Hyper-Parameter Optimization](../guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md) - Implementing an active selection of experiments
|
- [Hyperparameter Optimization](../guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md) - Implementing an active selection of experiments
|
||||||
- [Control Service](../guides/services/aws_autoscaler.md) - AWS Autoscaler for example
|
- [Control Service](../guides/services/aws_autoscaler.md) - AWS Autoscaler for example
|
||||||
- [External services](../guides/services/slack_alerts.md) - Such as Slack integration alert service
|
- [External services](../guides/services/slack_alerts.md) - Such as Slack integration alert service
|
||||||
|
|
||||||
|
@ -45,8 +45,8 @@ through a `Task` or `OutputModel` object.
|
|||||||
|
|
||||||
ClearML provides flexibility for explicitly connecting input models and experimentation, including:
|
ClearML provides flexibility for explicitly connecting input models and experimentation, including:
|
||||||
|
|
||||||
* Importing pre-trained models from external sources such as Amazon AWS, GIT repositories, PyTorch, and TensorFlow.
|
* Importing pre-trained models from external sources such as Amazon AWS, GIT repositories, PyTorch, and TensorFlow
|
||||||
* Using standalone models already registered in ClearML by previously run experiments.
|
* Using standalone models already registered in ClearML by previously run experiments
|
||||||
* Defining your own input models in scripts
|
* Defining your own input models in scripts
|
||||||
|
|
||||||
## Setting Upload Destination
|
## Setting Upload Destination
|
||||||
|
@ -57,7 +57,7 @@ optimization.
|
|||||||
documentation and a [code example](../guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md).
|
documentation and a [code example](../guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md).
|
||||||
* **Random** uniform sampling of hyperparameters - [`automation.RandomSearch`](../references/sdk/hpo_optimization_randomsearch.md).
|
* **Random** uniform sampling of hyperparameters - [`automation.RandomSearch`](../references/sdk/hpo_optimization_randomsearch.md).
|
||||||
* **Full grid** sampling strategy of every hyperparameter combination - [`automation.GridSearch`](../references/sdk/hpo_optimization_gridsearch.md).
|
* **Full grid** sampling strategy of every hyperparameter combination - [`automation.GridSearch`](../references/sdk/hpo_optimization_gridsearch.md).
|
||||||
* **Custom** - [`automation.optimization.SearchStrategy`](https://github.com/allegroai/clearml/blob/master/clearml/automation/optimization.py#L268) - Use a custom class and inherit from the ClearML automation base strategy class
|
* **Custom** - [`automation.optimization.SearchStrategy`](https://github.com/allegroai/clearml/blob/master/clearml/automation/optimization.py#L268) - Use a custom class and inherit from the ClearML automation base strategy class.
|
||||||
|
|
||||||
|
|
||||||
## Defining a Hyperparameter Optimization Search Example
|
## Defining a Hyperparameter Optimization Search Example
|
||||||
|
@ -22,10 +22,10 @@ and tracks hyperparameters of various types, supporting automatic logging and ex
|
|||||||
Once a ClearML Task has been [initialized](../references/sdk/task.md#taskinit) in a script, ClearML automatically captures and tracks
|
Once a ClearML Task has been [initialized](../references/sdk/task.md#taskinit) in a script, ClearML automatically captures and tracks
|
||||||
the following types of parameters:
|
the following types of parameters:
|
||||||
* Command line parsing - command line parameters passed when invoking code that uses standard python packages, including:
|
* Command line parsing - command line parameters passed when invoking code that uses standard python packages, including:
|
||||||
* [click](../integrations/click.md).
|
* [click](../integrations/click.md)
|
||||||
* [argparse](../guides/reporting/hyper_parameters.md#argparse-command-line-options).
|
* [argparse](../guides/reporting/hyper_parameters.md#argparse-command-line-options)
|
||||||
* [Python Fire](../integrations/python_fire.md).
|
* [Python Fire](../integrations/python_fire.md)
|
||||||
* [LightningCLI](../integrations/pytorch_lightning.md).
|
* [LightningCLI](../integrations/pytorch_lightning.md)
|
||||||
* TensorFlow Definitions (`absl-py`). See examples of ClearML's automatic logging of TF Defines:
|
* TensorFlow Definitions (`absl-py`). See examples of ClearML's automatic logging of TF Defines:
|
||||||
* [TensorFlow MNIST](../guides/frameworks/tensorflow/tensorflow_mnist.md)
|
* [TensorFlow MNIST](../guides/frameworks/tensorflow/tensorflow_mnist.md)
|
||||||
* [TensorBoard PR Curve](../guides/frameworks/tensorflow/tensorboard_pr_curve.md)
|
* [TensorBoard PR Curve](../guides/frameworks/tensorflow/tensorboard_pr_curve.md)
|
||||||
@ -118,9 +118,9 @@ including their subsections (see [WebApp sections](#webapp-interface) below).
|
|||||||
Configurations can be viewed in web UI experiment pages, in the **CONFIGURATION** tab.
|
Configurations can be viewed in web UI experiment pages, in the **CONFIGURATION** tab.
|
||||||
|
|
||||||
The configuration panel is split into three sections according to type:
|
The configuration panel is split into three sections according to type:
|
||||||
- **User Properties** - Modifiable section that can be edited post-execution.
|
- **User Properties** - Modifiable section that can be edited post-execution
|
||||||
- **Hyperparameters** - Individual parameters for configuration
|
- **Hyperparameters** - Individual parameters for configuration
|
||||||
- **Configuration Objects** - Usually configuration files (JSON / YAML) or Python objects.
|
- **Configuration Objects** - Usually configuration files (JSON / YAML) or Python objects
|
||||||
|
|
||||||
These sections are further broken down into subsections based on how the parameters were logged (General / Args / TF_Define / Environment).
|
These sections are further broken down into subsections based on how the parameters were logged (General / Args / TF_Define / Environment).
|
||||||
|
|
||||||
|
@ -72,7 +72,7 @@ This section is mostly used for debugging. It's recommended to use [artifacts](t
|
|||||||
outputs that would be used later on.
|
outputs that would be used later on.
|
||||||
|
|
||||||
Only the last X results of each title / series are saved to prevent overloading the server.
|
Only the last X results of each title / series are saved to prevent overloading the server.
|
||||||
See details in [Logger.report_media](../references/sdk/logger.md#report_media).
|
See details in [`Logger.report_media`](../references/sdk/logger.md#report_media).
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
|
@ -18,14 +18,14 @@ the project are executed, the model checkpoints (snapshots) and artifacts are st
|
|||||||
|
|
||||||
## WebApp
|
## WebApp
|
||||||
|
|
||||||
Users can create and modify projects, and see project details in the WebApp (see [WebApp Home](../webapp/webapp_home.md)).
|
Users can create and modify projects, and see project details in the [WebApp](../webapp/webapp_home.md).
|
||||||
The project's description can be edited in the [overview](../webapp/webapp_overview.md) page. Each project's experiments,
|
A project's description can be edited in its [overview](../webapp/webapp_project_overview.md) page. Each project's experiments,
|
||||||
models, and dataviews, can be viewed in the project's [experiments table](../webapp/webapp_exp_table.md),
|
models, and dataviews, can be viewed in the project's [experiments table](../webapp/webapp_exp_table.md),
|
||||||
[models table](../webapp/webapp_model_table.md), and [dataviews table](../hyperdatasets/webapp/webapp_dataviews.md).
|
[models table](../webapp/webapp_model_table.md), and [dataviews table](../hyperdatasets/webapp/webapp_dataviews.md).
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
||||||
### Creating Subprojects
|
### Creating Projects and Subprojects
|
||||||
|
|
||||||
When [initializing a task](../clearml_sdk/task_sdk.md#task-creation), its project needs to be specified. If the project entered does not exist, it will be created.
|
When [initializing a task](../clearml_sdk/task_sdk.md#task-creation), its project needs to be specified. If the project entered does not exist, it will be created.
|
||||||
Projects can contain subprojects, just like folders can contain subfolders. Input into the `project_name`
|
Projects can contain subprojects, just like folders can contain subfolders. Input into the `project_name`
|
||||||
@ -40,7 +40,7 @@ from clearml import Task
|
|||||||
Task.init(project_name='main_project/sub_project', task_name='test')
|
Task.init(project_name='main_project/sub_project', task_name='test')
|
||||||
```
|
```
|
||||||
|
|
||||||
Nesting projects works on multiple levels. For example: `project_name=main_project/sub_project/sub_sub_project`
|
Nesting projects works on multiple levels. For example: `project_name=main_project/sub_project/sub_sub_project`.
|
||||||
|
|
||||||
Projects can also be created using the [`projects.create`](../references/api/projects.md#post-projectscreate) REST API call.
|
Projects can also be created using the [`projects.create`](../references/api/projects.md#post-projectscreate) REST API call.
|
||||||
|
|
||||||
@ -56,7 +56,8 @@ This returns a list of projects sorted by last update time.
|
|||||||
|
|
||||||
### More Actions
|
### More Actions
|
||||||
|
|
||||||
For additional ways to work with projects, use the REST API `projects` resource. Some of the available actions include:
|
For additional ways to work with projects, use the REST API [`projects`](../references/api/projects.md) resource. Some
|
||||||
|
of the available actions include:
|
||||||
* [`projects.create`](../references/api/projects.md#post-projectscreate) and [`projects.delete`](../references/api/projects.md#post-projectsdelete) - create and delete projects
|
* [`projects.create`](../references/api/projects.md#post-projectscreate) and [`projects.delete`](../references/api/projects.md#post-projectsdelete) - create and delete projects
|
||||||
* [`projects.get_hyper_parameters`](../references/api/projects.md#post-projectsget_hyper_parameters) - get a list of all hyperparameter sections and names used in a project
|
* [`projects.get_hyper_parameters`](../references/api/projects.md#post-projectsget_hyper_parameters) - get a list of all hyperparameter sections and names used in a project
|
||||||
* [`projects.merge_projects`](../references/api/projects.md#post-projectsmerge) - merge projects into a single project
|
* [`projects.merge_projects`](../references/api/projects.md#post-projectsmerge) - merge projects into a single project
|
||||||
|
@ -31,10 +31,10 @@ You can create copies of a task ([clone](../webapp/webapp_exp_reproducing.md)) t
|
|||||||
* Set hyperparameter and run-time configuration values
|
* Set hyperparameter and run-time configuration values
|
||||||
|
|
||||||
Modifying a task clone's configuration will have the executing ClearML agent override the original values:
|
Modifying a task clone's configuration will have the executing ClearML agent override the original values:
|
||||||
* Modified package requirements will have the experiment script run with updated packages.
|
* Modified package requirements will have the experiment script run with updated packages
|
||||||
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
|
* Modified recorded command line arguments will have the ClearML agent inject the new values in their stead
|
||||||
* Code-level configuration instrumented with [`Task.connect`](../references/sdk/task.md#connect) will be overridden by
|
* Code-level configuration instrumented with [`Task.connect`](../references/sdk/task.md#connect) will be overridden by
|
||||||
modified hyperparameters.
|
modified hyperparameters
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@ -149,7 +149,7 @@ The above diagram describes how execution information is recorded when running c
|
|||||||
including:
|
including:
|
||||||
* Source code
|
* Source code
|
||||||
* Python environment
|
* Python environment
|
||||||
* Configuration parameters.
|
* Configuration parameters
|
||||||
1. As the execution progresses, any outputs produced are recorded including:
|
1. As the execution progresses, any outputs produced are recorded including:
|
||||||
* Console logs
|
* Console logs
|
||||||
* Metrics and graphs
|
* Metrics and graphs
|
||||||
|
@ -10,7 +10,7 @@ class. The storage examples include:
|
|||||||
* [Setting cache limits](#setting-cache-limits) - Set the maximum number of objects.
|
* [Setting cache limits](#setting-cache-limits) - Set the maximum number of objects.
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
`StorageManager` supports http(s), S3, Google Cloud Storage, Azure, and file system folders.
|
`StorageManager` supports HTTP(S), S3, Google Cloud Storage, Azure, and file system folders.
|
||||||
:::
|
:::
|
||||||
|
|
||||||
## StorageManager
|
## StorageManager
|
||||||
|
Loading…
Reference in New Issue
Block a user