From 1adfc186962175c9f394ba043c7dd19708806723 Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Sun, 23 Jul 2023 11:52:14 +0300 Subject: [PATCH 1/2] Update clearml-session command line options (#620) --- docs/apps/clearml_session.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/apps/clearml_session.md b/docs/apps/clearml_session.md index f53c3823..dede291a 100644 --- a/docs/apps/clearml_session.md +++ b/docs/apps/clearml_session.md @@ -165,14 +165,14 @@ The Task must be connected to a git repository, since currently single script de | `--requirements`| Specify requirements.txt file to install when setting the interactive session. | `none` or previously used requirements (can be overridden by calling `--packages`)| | `--packages`| Additional packages to add. Supports version numbers. Example: `--packages torch==1.7 tqdm` | Previously added packages.| | `--git-credentials` | If `True`, local `.git-credentials` file is sent to the interactive session.| `false`| -| `--verbose ` | Increase verbosity of logging | `none` | | `--docker`| Select the docker image to use in the interactive session on |`nvidia/cuda:10.1-runtime-ubuntu18.04` or previously used docker image| | `--docker-args ` | Add additional arguments for the docker image to use in the interactive session | `none` or the previously used docker-args | | `--debugging-session` | Pass existing Task ID, create a copy of the experiment on a remote machine, and launch Jupyter/SSH for interactive access. Example `--debugging-session `| `none`| | `--queue`| Select the queue to launch the interactive session on | Previously used queue| | `--jupyter-lab` | Install a JupyterLab on interactive session | `true` | | `--vscode-server` | Install VSCode on interactive session | `true` | -| `--vscode-version` | Set VSCode server (code-server) version, as well as VSCode python extension version (example: "3.7.4:2020.10.332292344")| `3.12.0:2021.10.1365161279`| +| `--vscode-version` | Set VSCode server (code-server) version, as well as VSCode python extension version (example: "3.7.4:2020.10.332292344")| `4.14.1:2023.12.0`| +| `--vscode-extensions` |Install additional VSCode extensions and VSCode python extensions (example: `ms-python.python,ms-python.black-formatter,ms-python.pylint,ms-python.flake8`)|`None`| | `--public-ip` | If `true`, register the public IP of the remote machine (if you are running the session on a public cloud) | `false` - Session runs on the machine whose agent is executing the session| | `--remote-ssh-port`| Set the remote SSH server port, running on the agent's machine | 10022| | `--init-script` | Specify a BASH init script file to be executed when the interactive session is being set up | `none` or previously entered BASH script | @@ -186,7 +186,9 @@ The Task must be connected to a git repository, since currently single script de | `--skip-docker-network` | Don't pass the `--network host` flag to the Docker that is launching the remote session. See [Networking using the host network](https://docs.docker.com/network/network-tutorial-host/) | `false`| | `--username`| Set your own SSH username for the interactive session | `root` or a previously used username | | `--password`| Set your own SSH password for the interactive session | A randomly generated password or a previously used one | +| `--force_dropbear`| Force using `dropbear` instead of SSHd |`None`| | `--version`| Display the clearml-session utility version| N/A| +| `--verbose ` | Increase verbosity of logging | `none` | | `--yes`, `-y`| Automatic yes to prompts; assume "yes" as answer to all prompts and run non-interactively |N/A| \ No newline at end of file From f2491cf9f0eb243dc3731cbdb964bab6c677118d Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Sun, 23 Jul 2023 12:11:32 +0300 Subject: [PATCH 2/2] Small edits (#621) --- docs/fundamentals/hpo.md | 16 ++++++++++------ docs/integrations/hydra.md | 6 +++--- docs/integrations/yolov5.md | 2 +- docs/release_notes/ver_1_12.md | 9 ++++----- 4 files changed, 18 insertions(+), 15 deletions(-) diff --git a/docs/fundamentals/hpo.md b/docs/fundamentals/hpo.md index c6dadf70..1fcff191 100644 --- a/docs/fundamentals/hpo.md +++ b/docs/fundamentals/hpo.md @@ -121,14 +121,14 @@ optimization. ## Optimizer Execution Options The `HyperParameterOptimizer` provides options to launch the optimization tasks locally or through a ClearML [queue](agents_and_queues.md#what-is-a-queue). -Start a `HyperParameterOptimizer` instance using either [`HyperParameterOptimizer.start`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md#start) -or [`HyperParameterOptimizer.start_locally`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md#start_locally). -Both methods run the optimizer controller locally. The `start` method launches the base task clones through a queue -specified when instantiating the controller, while `start_locally` runs the tasks locally. +Start a `HyperParameterOptimizer` instance using either [`HyperParameterOptimizer.start()`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md#start) +or [`HyperParameterOptimizer.start_locally()`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md#start_locally). +Both methods run the optimizer controller locally. `start()` launches the base task clones through a queue +specified when instantiating the controller, while `start_locally()` runs the tasks locally. :::tip Remote Execution -You can also launch the optimizer controller through a queue by using the [`Task.execute_remotely`](../references/sdk/task.md#execute_remotely) -method before starting the optimizer. +You can also launch the optimizer controller through a queue by using [`Task.execute_remotely()`](../references/sdk/task.md#execute_remotely) +before starting the optimizer. ::: @@ -147,5 +147,9 @@ ClearML also provides `clearml-param-search`, a CLI utility for managing the hyp ## UI Application +:::info Pro Plan Offering +The ClearML HPO App is available under the ClearML Pro plan +::: + ClearML provides the [Hyperparameter Optimization GUI application](../webapp/applications/apps_hpo.md) for launching and managing the hyperparameter optimization process. diff --git a/docs/integrations/hydra.md b/docs/integrations/hydra.md index 5e7ff7fa..654ac831 100644 --- a/docs/integrations/hydra.md +++ b/docs/integrations/hydra.md @@ -26,10 +26,10 @@ The agent executes the code with the modifications you made in the UI, even over Clone your experiment, then modify your Hydra parameters via the UI in one of the following ways: * Modify the OmegaConf directly: - 1. In the experiment’s **CONFIGURATION > HYPERPARAMETERS > HYDRA** section, set `_allow_omegaconf_edit_` to `True` - 1. In the experiment’s **CONFIGURATION > CONFIGURATION OBJECTS > OmegaConf** section, modify the OmegaConf values + 1. In the experiment’s **CONFIGURATION > HYPERPARAMETERS > HYDRA** section, set `_allow_omegaconf_edit_` to `True` + 1. In the experiment’s **CONFIGURATION > CONFIGURATION OBJECTS > OmegaConf** section, modify the OmegaConf values * Add an experiment hyperparameter: - 1. In the experiment’s **CONFIGURATION > HYPERPARAMETERS > HYDRA** section, make sure `_allow_omegaconf_edit_` is set + 1. In the experiment’s **CONFIGURATION > HYPERPARAMETERS > HYDRA** section, make sure `_allow_omegaconf_edit_` is set to `False` 1. In the same section, click `Edit`, which gives you the option to add parameters. Input parameters from the OmegaConf that you want to modify using dot notation. For example, if your OmegaConf looks like this: diff --git a/docs/integrations/yolov5.md b/docs/integrations/yolov5.md index 0c198487..ca039725 100644 --- a/docs/integrations/yolov5.md +++ b/docs/integrations/yolov5.md @@ -142,7 +142,7 @@ New dataset created id= ``` ### Run Training Using a ClearML Dataset -Now that you have a ClearML dataset, you can very simply use it to train custom YOLOv5 models: +Now that you have a ClearML dataset, you can very simply use it to train custom YOLOv5 models: ```commandline python train.py --img 640 --batch 16 --epochs 3 --data clearml:// --weights yolov5s.pt --cache diff --git a/docs/release_notes/ver_1_12.md b/docs/release_notes/ver_1_12.md index 63193278..cec43b87 100644 --- a/docs/release_notes/ver_1_12.md +++ b/docs/release_notes/ver_1_12.md @@ -12,17 +12,16 @@ the instructions [here](https://github.com/allegroai/clearml/tree/master/docs/er ::: **New Features** -* Add `include_archive` parameter to `Dataset.list_datasets()`: include archived datasets in list [ClearML GitHub issue #1069](https://github.com/allegroai/clearml/issues/1069) +* Add `include_archive` parameter to `Dataset.list_datasets()`: include archived datasets in list [ClearML GitHub issue #1067](https://github.com/allegroai/clearml/issues/1067) * Add support to specify the multipart chunk size and threshold using the `aws.boto3.multipart_chunksize` and -`aws.boto3.multipart_threshold` configuration options in the clearml.conf [ClearML GitHub issue #1059](https://github.com/allegroai/clearml/issues/1059) +`aws.boto3.multipart_threshold` configuration options in the clearml.conf [ClearML GitHub issue #1058](https://github.com/allegroai/clearml/issues/1058) * Add `PipelineController.get_pipeline()` for retrieving previously run pipelines. **Bug Fixes** -* Fix `continue_last_task=0` is ignored in pipelines run with `retry_on_failure` [ClearML GitHub issue #1054](https://github.com/allegroai/clearml/issues/1054) -* Fix AWS driver issues: [ClearML GitHub issue #1000](https://github.com/allegroai/clearml/issues/1000) +* Fix AWS driver issues: [ClearML GitHub PR #1000](https://github.com/allegroai/clearml/pull/1000) * Fix credential authentication failure when attempting to use token * Fix instantiation within VPC without AvailabilityZones -* Fix Error accessing GCP artifacts when using special characters in task name [ClearML GitHub issue #1051](https://github.com/allegroai/clearml/issues/1051) +* Fix `continue_last_task=0` is ignored in pipelines run with `retry_on_failure` [ClearML GitHub issue #1054](https://github.com/allegroai/clearml/issues/1054) * Fix `Task.connect_configuration()` doesn't handle dictionaries with special characters * Fix pipeline steps created with `PipelineDecorator` aren't cached * Fix `Task.get_by_name()` doesn't return the most recent task when multiple tasks have same name