diff --git a/docs/clearml_agent.md b/docs/clearml_agent.md index 9e33ad6a..6b36894e 100644 --- a/docs/clearml_agent.md +++ b/docs/clearml_agent.md @@ -48,7 +48,7 @@ While the agent is running, it continuously reports system metrics to the ClearM Continue using ClearML Agent once it is running on a target machine. Reproduce experiments and execute automated workflows in one (or both) of the following ways: -* Programmatically (using [`Task.enqueue`](references/sdk/task.md#taskenqueue) or [`Task.execute_remotely`](references/sdk/task.md#execute_remotely)) +* Programmatically (using [`Task.enqueue()`](references/sdk/task.md#taskenqueue) or [`Task.execute_remotely()`](references/sdk/task.md#execute_remotely)) * Through the ClearML Web UI (without working directly with code), by cloning experiments and enqueuing them to the queue that a ClearML Agent is servicing. @@ -57,7 +57,7 @@ code modification. Modifying a task clone’s configuration will have the ClearM original values: * Modified package requirements will have the experiment script run with updated packages * Modified recorded command line arguments will have the ClearML agent inject the new values in their stead -* Code-level configuration instrumented with [`Task.connect`](references/sdk/task.md#connect) will be overridden by modified hyperparameters +* Code-level configuration instrumented with [`Task.connect()`](references/sdk/task.md#connect) will be overridden by modified hyperparameters For more information, see [ClearML Agent Reference](clearml_agent/clearml_agent_ref.md), and [configuration options](configs/clearml_conf.md#agent-section). @@ -109,7 +109,7 @@ it can't do that when running from a virtual environment. Detected credentials key="********************" secret="*******" ``` -1. **Enter** to accept default server URL, which is detected from the credentials or enter a ClearML web server URL. +1. **Enter** to accept the default server URL, which is detected from the credentials or enter a ClearML web server URL. A secure protocol, https, must be used. **Do not use http.** diff --git a/docs/clearml_agent/clearml_agent_env_var.md b/docs/clearml_agent/clearml_agent_env_var.md index b557ccf1..ad1f2b4f 100644 --- a/docs/clearml_agent/clearml_agent_env_var.md +++ b/docs/clearml_agent/clearml_agent_env_var.md @@ -28,7 +28,7 @@ but can be overridden by command-line arguments. |**CLEARML_AGENT_EXEC_USER** | User for Agent executing tasks (root by default) | |**CLEARML_AGENT_EXTRA_DOCKER_ARGS** | Overrides extra docker args configuration | |**CLEARML_AGENT_EXTRA_DOCKER_LABELS** | List of labels to add to docker container. See [Docker documentation](https://docs.docker.com/config/labels-custom-metadata/). | -|**CLEARML_EXTRA_PIP_INSTALL_FLAGS**| List of additional flags to use when the agent install packages. For example: `["--use-deprecated=legacy-resolver", ]`| +|**CLEARML_EXTRA_PIP_INSTALL_FLAGS**| List of additional flags to use when the agent installs packages. For example: `["--use-deprecated=legacy-resolver", ]`| |**CLEARML_AGENT_EXTRA_PYTHON_PATH** | Sets extra python path | |**CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE** | Overrides initial server connection behavior (true by default), allows explicit number to specify number of connect retries) | |**CLEARML_AGENT_NO_UPDATE** | Boolean. Set to `true` to skip agent update in the k8s pod container before the agent executes the task | diff --git a/docs/clearml_data/best_practices.md b/docs/clearml_data/best_practices.md index 966e396c..472d14da 100644 --- a/docs/clearml_data/best_practices.md +++ b/docs/clearml_data/best_practices.md @@ -29,19 +29,19 @@ Like any ClearML tasks, datasets can be organized into [projects (and subproject Additionally, when creating a dataset, tags can be applied to the dataset, which will make searching for the dataset easier. Organizing your datasets into projects by use-case makes it easier to access the most recent dataset version for that use-case. -If only a project is specified when using [`Dataset.get`](../references/sdk/dataset.md#datasetget), the method returns the +If only a project is specified when using [`Dataset.get()`](../references/sdk/dataset.md#datasetget), the method returns the most recent dataset in a project. The same is true with tags; if a tag is specified, the method will return the most recent dataset that is labeled with that tag. In cases where you use a dataset in a task (e.g. consuming a dataset), you can easily track which dataset the task is -using by using `Dataset.get`'s `alias` parameter. Pass `alias=`, and the task using the dataset +using by using `Dataset.get()`'s `alias` parameter. Pass `alias=`, and the task using the dataset will store the dataset's ID in the `dataset_alias_string` parameter under the task's **CONFIGURATION > HYPERPARAMETERS > Datasets** section. ## Document your Datasets -Attach informative metrics or debug samples to the Dataset itself. Use the [`get_logger`](../references/sdk/dataset.md#get_logger) -method to access the dataset's logger object, then add any additional information to the dataset, using the methods +Attach informative metrics or debug samples to the Dataset itself. Use [`Dataset.get_logger()`](../references/sdk/dataset.md#get_logger) +to access the dataset's logger object, then add any additional information to the dataset, using the methods available with a [logger](../references/sdk/logger.md) object. You can add some dataset summaries (like [table reporting](../references/sdk/logger.md#report_table)) to create a preview diff --git a/docs/fundamentals/artifacts.md b/docs/fundamentals/artifacts.md index da0e7e49..40e9e983 100644 --- a/docs/fundamentals/artifacts.md +++ b/docs/fundamentals/artifacts.md @@ -13,14 +13,15 @@ interface. Once integrated into code, ClearML automatically logs and tracks models and any snapshots created by the following frameworks: -- TensorFlow (see [code example](../guides/frameworks/tensorflow/tensorflow_mnist.md)) -- Keras (see [code example](../guides/frameworks/keras/keras_tensorboard.md)) -- PyTorch (see [code example](../guides/frameworks/pytorch/pytorch_mnist.md)) -- scikit-learn (only using joblib) (see [code example](../guides/frameworks/scikit-learn/sklearn_joblib_example.md)) -- XGBoost (only using joblib) (see [code example](../guides/frameworks/xgboost/xgboost_sample.md)) -- FastAI (see [code example](../guides/frameworks/fastai/fastai_with_tensorboard.md)) -- MegEngine (see [code example](../guides/frameworks/megengine/megengine_mnist.md)) -- CatBoost (see [code example](../guides/frameworks/catboost/catboost.md)) +* [TensorFlow](../integrations/tensorflow.md) +* [Keras](../integrations/keras.md) +* [PyTorch](../integrations/pytorch.md) +* [scikit-learn](../integrations/scikit_learn.md) (only using joblib) +* [XGBoost](../integrations/xgboost.md) (only using joblib) +* [Fast.ai](../integrations/fastai.md) +* [MegEngine](../integrations/megengine.md) +* [CatBoost](../integrations/catboost.md) +* [MONAI](../integrations/monai.md)) When a supported framework loads a weights file, the running task will be automatically updated, with its input model pointing directly to the original training task's model. diff --git a/docs/integrations/optuna.md b/docs/integrations/optuna.md index 1666b21c..59ec7b39 100644 --- a/docs/integrations/optuna.md +++ b/docs/integrations/optuna.md @@ -9,11 +9,11 @@ Optuna into ClearML's automated hyperparameter optimization. The [HyperParameterOptimizer](../references/sdk/hpo_optimization_hyperparameteroptimizer.md) class contains ClearML's hyperparameter optimization modules. Its modular design enables using different optimizers, including existing software frameworks, like Optuna, enabling simple, -accurate, and fast hyperparameter optimization. The Optuna ([`automation.optuna.OptimizerOptuna`](../references/sdk/hpo_optuna_optuna_optimizeroptuna.md)), +accurate, and fast hyperparameter optimization. The Optuna ([`automation.optuna.OptimizerOptuna`](../references/sdk/hpo_optuna_optuna_optimizeroptuna.md)) optimizer lets you simultaneously optimize many hyperparameters efficiently by relying on early stopping (pruning) and smart resource allocation. -To use optuna in ClearML's hyperparameter optimization, you must first install it. When you instantiate `HyperParameterOptimizer`, +To use Optuna in ClearML's hyperparameter optimization, you must first install it. When you instantiate `HyperParameterOptimizer`, pass `OptimizerOptuna` as the `optimizer_class` argument: ```python diff --git a/docs/integrations/transformers.md b/docs/integrations/transformers.md index 08574b86..a4dcfc90 100644 --- a/docs/integrations/transformers.md +++ b/docs/integrations/transformers.md @@ -56,6 +56,7 @@ Additionally, you can view all of your Transformers runs tracked by ClearML in t Add custom columns to the table, such as mAP values, so you can easily sort and see what is the best performing model. You can also select multiple experiments and directly [compare](../webapp/webapp_exp_comparing.md) them. +See an example of Transformers and ClearML in action [here](../guides/frameworks/huggingface/transformers.md). ## Remote Execution ClearML logs all the information required to reproduce an experiment on a different machine (installed packages,