ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-06-26 18:17:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Refactor Release Notes (#938)

Browse Source
This commit is contained in:
pollfly
2024-09-26 16:09:25 +03:00
committed by GitHub
parent eb36f48413
commit 8249768619
89 changed files with 3127 additions and 2660 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
docs/release_notes/clearml_agent/ver_0_12.md Normal file
View File

@@ -0,0 +1,39 @@
---
title: Version 0.12
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Agent 0.12.2
**Features and Bug Fixes**
* Improve configuration wizard.
* Improve Docker and Kubernetes support.
* Add `docker build` command.
* Add initial Poetry support.
* Fix Docker CUDA support.
### Trains Agent 0.12.1
**Features and Bug Fixes**
* Windows support! YES, you can now have Windows machines as part of your cluster (notice --docker is not supported on Windows).
* Add initial Conda package manager support (still in beta).
* Add --gpus and --cpu-only for easier GPU control when running multiple `trains-agent` instances on the same machine.
* [python_binary](https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf#L35) can now be specified (the default is the same Python binary executing the `trains-agent`).
* Fix Issue ([GitHub Issue #2](https://github.com/allegroai/trains-agent/issues/2)).
### Trains Agent 0.12.0
TRAINS Agent - Initial Release
* TRAINS Agent currently supports Linux and Mac.
* Python packaging system supported: pip.
* Containers supported: Docker v19.03 and above.
* Version Control support: git
* Support for standalone scripts and Jupyter Notebooks.
* TRAINS-server v0.12 and above.
* TRAINS v0.11.3 and above.
* Sample configuration file available [here](https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf).

34
docs/release_notes/clearml_agent/ver_0_13.md Normal file
View File

@@ -0,0 +1,34 @@
---
title: Version 0.13
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Agent 0.13.3
**Features and Bug Fixes**
* Allow providing queue names instead of queue IDs in daemon mode.
* Improve Docker mode:
* Support running as a specific user inside a docker using the `TRAINS_AGENT_EXEC_USER` environment flag.
* Pass the correct GPU limit when skipping gpus flag.
* Add the `--force-current-version` daemon command-line flag.
* Add K8s/trains glue service example.
* Add K8s support in daemon mode.
* Running inside a K8s pod.
* Mounting dockerized experiment folders to host.
* Allow a specific network for the docker.
* Add default storage environment vars (for AWS, GS and Azure) to generated agent configuration.
* Improve Unicode/UTF stdout handling.
### Trains Agent 0.13.0
**Features**
* Add support for Docker pre-installed pytorch versions that do not exist on PyPI/PyTorch.org.
* Add AWS dynamic cluster management service.
* Add support for various event query endpoints in APIClient.
* Improve the configuration wizard.

32
docs/release_notes/clearml_agent/ver_0_14.md Normal file
View File

@@ -0,0 +1,32 @@
---
title: Version 0.14
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Agent 0.14.1
**Features and Bug Fixes**
* Add `daemon` detached mode (`--detached`, `-d`) that runs the agent as daemon in the background and returns immediately.
* Auto mount `~/.git-credentials` into a Docker container, if the file exists.
* Add `TRAINS_AGENT_EXTRA_PYTHON_PATH` environment variable to allow the adding of an additional Python path during experiment execution (helpful when using extra un-tracked modules).
* Fix "run as user" feature (using `TRAINS_AGENT_EXEC_USER` environment variable).
* Fix PyTorch support to ignore minor versions when looking for package to install / download.
* Fix experiment execution output handling.
### Trains Agent 0.14.0
**Features and Bug Fixes**
* Add support for `trains-agent execute --id <experiment-id> --docker` that allows executing a specific experiment inside a docker container.
* Add support for `trains-agent execute --id <template-experiment-id> --clone` that clones the provided experiment and executes the cloned experiment.
* Add support for `APIClient.models.delete` to allow programmatically deleting a model ([GitHub trains-server Issue #32](https://github.com/allegroai/trains-server/issues/32)).
* Add daemon support for passing storage-related OS environment variables to experiments executed inside a Docker container (supported by trains>=0.13.3):
* AWS: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_DEFAULT_REGION`
* Azure: `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_KEY`
* Google: `GOOGLE_APPLICATION_CREDENTIALS`
* Fix Git checkout with submodules ([GitHub Issue #112](https://github.com/allegroai/trains/issues/112)).
* Docker image in command line takes precedence over the Docker image specified in an experiment.

45
docs/release_notes/clearml_agent/ver_0_15.md Normal file
View File

@@ -0,0 +1,45 @@
---
title: Version 0.15
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Agent 0.15.1
**Features**
* Add Trains Agent Daemon and Services docker files.
**Bug Fixes**
* Fix initialization wizard (allow at most two verification retries, then print error).
* Add warning on `--gpus` with no detected CUDA version ([GitHub trains-agent Issue 24](https://github.com/allegroai/trains-agent/issues/24)).
* Add `agent.force_git_ssh_protocol` configuration option to force all Git links to `ssh://` ([GitHub trains-agent Issue 16](https://github.com/allegroai/trains-agent/issues/16)).
* Add Git user / pass permission into pip package installation from Git repository ([GitHub trains-agent Issue 22](https://github.com/allegroai/trains-agent/issues/22)).
### Trains Agent 0.15.0
**Features**
* Add daemon Services Mode (`daemon --services-mode`), where the daemon spins a task in its own Docker container, and verifies start-up and shut-down. This allows multiple tasks to be launched simultaneously on the same machine (currently in CPU mode only), where each task service will register itself as a worker for the lifetime of the task.
* Enhance `build --docker` mode:
* Add `--install-globally` option to install required packages in the Docker's system Python.
* Add `--entry-point option` to allow automatic task cloning when running the Docker container.
* Support PyTorch Nightly builds using the `agent.torch_nightly` configuration flag. If true, the agent looks for a nightly build when a stable torch wheel is not found.
* Add environment variables support for git user / password:
* Using `TRAINS_AGENT_GIT_USER` / `TRAINS_AGENT_GIT_PASS`.
* Pass git credentials to dockerized experiment execution.
* Support running code from module (i.e. -m in execution entry point).
* Add `daemon --create-queue` to automatically create a queue and use it, if the queue name doesn't exist in the server.
* Move `--gpus` and `--cpu-only` to worker args (used by `daemon`, `execute` and `build`).
**Bug Fixes**
* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/allegroai/trains-agent/issues/19)).
* Improve docker host-mount support, use `TRAINS_AGENT_DOCKER_HOST_MOUNT` environment variable.
* Support pip v20.1 local/http package reference in pip freeze.
* Fix detached mode to correctly use cache folder slots.
* Fix `CUDA_VISIBLE_DEVICES` should never be set to "all".
* Do not monitor GPU when running with `--cpu-only`

95
docs/release_notes/clearml_agent/ver_0_16.md Normal file
View File

@@ -0,0 +1,95 @@
---
title: Version 0.16
---
:::important
**Trains** is now **ClearML**.
:::
### Trains-Agent 0.16.2
**Features**
- conda:
- Add `agent.package_manager.conda_env_as_base_docker` allowing "docker_cmd" to contain link to a full pre-packaged conda environment (`tar.gz` created by `conda-pack`). Use `TRAINS_CONDA_ENV_PACKAGE` environment variable to specify `conda tar.gz` file.
- Add conda support for read-only pre-built environment (pass conda folder as `docker_cmd` on Task).
- Improve trying to find conda executable.
- k8s glue:
- Add support for limited number of services exposing ports.
- Add support for k8s pod custom user properties.
- Allow selecting external `trains.conf` file for the pod itself.
- Allow providing pod template, extra bash init script, alternate SSH server port, gateway address (k8s ingress / ELB).
- Allow specifying `cudatoolkit` version in the "installed packages" section when using conda as package manager (<a href="https://github.com/allegroai/trains/issues/229" target="_blank">GitHub trains Issue 229</a>).
- Add `agent.package_manager.force_repo_requirements_txt`. If True, "Installed Packages" on Task are ignored, and only repository `requirements.txt` is used.
- Pass `TRAINS_DOCKER_IMAGE` into docker for interactive sessions.
- Add `torchcsprng` and `torchtext` to PyTorch resolving.
**Bug Fixes**
- When logging suppress "\r" when reading a current chunk of a file / stream. Add `agent.suppress_carriage_return` (default True) to support previous behavior.
- Make sure `TRAINS_AGENT_K8S_HOST_MOUNT` is used only once per mount.
- Fix k8s glue script to trains-agent default docker script.
- Fix apply git diff from submodule only.
- conda:
- Fix conda pip freeze to be consistent with trains 0.16.3.
- Fix conda environment support for trains 0.16.3 full env. Add `agent.package_manager.conda_full_env_update` to allow conda to update back the requirements (default False, to preserve previous behavior).
- Fix running from conda environment - `conda.sh` not found in first conda PATH match.
- Fix docker mode ubuntu / debian support by making sure not to ask for input (fix `tzdata` install).
- Fix repository detection - ignore environment `SSH_AUTH_SOCK`, only check if git user/pass are configured.
- git diff:
- Fix support for non-ascii diff.
- Fix diff with empty line at the end will cause corrupt diff apply message.
- Allow zero context diffs (useful when blind patching repository).
- Fix `daemon --stop` when agent UID cannot be located.
- Fix nvidia docker support on some linux distros (SUSE).
- Fix nvidia pytorch dockers support.
- Fix torch CUDA 11.1 support.
- Fix requirements dict with null entry in `pip` should be considered None install from repository's `requirements.txt`.
### Trains Agent 0.16.1
**Features**
* Add `sdk.metrics.plot_max_num_digits` configuration option to reduce plot storage size.
* Add `agent.package_manager.post_packages` and `agent.package_manager.post_optional_packages` configuration options to control packages install order (e.g. horovod).
* Add `agent.git_host` configuration option for limiting git credential usage for a specific host (overridable using `TRAINS_AGENT_GIT_HOST` environment variable).
* Add `agent.force_git_ssh_port` configuration option to control HTTPS to SSH link conversion for non-standard SSH ports.
* Add requirements detection features. Improve support for detecting new pip version (20+) supporting `package @ scheme://link`.
**Bug Fixes**
* Fix pre-installed packages are ignored when installing a git package wheel. Reinstalling a `git+http` link is enough
to make sure all requirements are met / installed ([GitHub Issue #196](https://github.com/allegroai/trains/issues/196)).
* Fix incorrect check for spaces in current execution folder.
* Fix requirements detection:
* Update torch version after using downloaded / system pre-installed version.
* Do not install git packages twice when a new pip version is used (pip freeze will detect the correct git link version).
### Trains Agent 0.16.0
**Features**
* Add `agent.docker_init_bash_script` configuration section to allow finer control over Docker startup script.
* Changed default Docker image from `nvidia/cuda` to `nvidia/cuda:10.1-runtime-ubuntu18.04` to support `cudnn` frameworks (e.g. TF).
* Improve support for Dockers with preinstalled `conda` environment.
* Improve trains-agent-docker spinning.
* Add `daemon --order-fairness` for round-robin queue pulling.
* Add `daemon --stop` to terminate a running agent (assuming other arguments are the same). If no additional arguments, Agents are terminated in lexicographical order.
* Support cleanup of all log files on termination unless executed with `--debug`.
* Add error message when Trains API Server is not accessible on startup.
**Bug Fixes**
* Fix GPU Windows monitoring support ([GitHub Issue #177](https://github.com/allegroai/trains/issues/177)).
* Fix `.git-credentials` and `.gitconfig` mapping into docker.
* Fix non-root docker image usage.
* Fix docker to use `UTF-8` encoding, so prints won't break it.
* Fix `--debug` to set all loggers to `DEBUG`.
* Fix task status change to `queued` should never happen during Task runtime.
* Fix `requirement_parser` to support `package @ git+http` lines.
* Fix GIT user/password in requirements and support for `-e git+http` lines.
* Fix configuration wizard to generate `trains.conf` matching latest Trains definitions.

46
docs/release_notes/clearml_agent/ver_0_17.md Normal file
View File

@@ -0,0 +1,46 @@
---
title: Version 0.17
---
## ClearML Agent 0.17.2
**Features**
- Add virtual environment caching:
* Supports venv caching both in standard and docker mode.
* Configurable using the `agent.venvs_cache` configuration section.
* Disabled by default, enable [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L101).
- Add support for `--services-mode` with venvs.
- Add `agent.force_git_ssh_user` configuration value (default `git`, see [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L29))
([clearml-agent GitHub Issue 42](https://github.com/allegroai/clearml-agent/issues/42)).
- Add `agent.ignore_requested_python_version` configuration option for multi Python environments (default `false`).
- Add `agent.enable_task_env` configuration option to set the OS environment based on the Environment section of the Task (default `false`, see
[here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L151)).
- K8s glue:
* Add support for detecting and deleting k8s pods that fail to start.
* Allow providing namespace in k8s glue and k8s glue example.
* Add base-pod-number parameter to k8s glue and example.
- Change `agent.default_docker.image` to `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` (see [here](https://github.com/allegroai/clearml-agent/blob/8a46dc6b03860bc1cc022fd19e86bf4524986a0e/docs/clearml.conf#L144)).
- Use shared git cache for multiple agents on the same machine.
- Upgrade pynvml add detect CUDA version from driver level.
- Update agent and services docker files.
- Update documentation.
**Bug Fixes**
- Fix `docker --network` returns `None`.
- Fix docker mode without venvs cache dir.
- Fix applying git diff on a newly added file.
- Fix environment variables `CLEARML_WEB_HOST`/`CLEARML_FILES_HOST` not passed to running tasks (or updated on the config object).
- Fix `--detached` command line option not supported on Windows (ignore and issue warning).
- Fix file not found error (errno 2) interpreted as aborted (i.e. `Ctrl-C`).
- Fix `from clearml` runtime diff patching.
- Fix cache to take cuda version into account.
- Fix CPU mode.
- Fix multi instances on Windows.
- Fix conda support for `git+http` links.
- Fix k8s glue does not pass docker environment variables, remove deprecated flags.
### ClearML Agent 0.17.0
* Renamed `trains-agent` as `clearml-agent` (see [clearml-agent](https://github.com/allegroai/clearml-agent) in GitHub).

37
docs/release_notes/clearml_agent/ver_1_0.md Normal file
View File

@@ -0,0 +1,37 @@
---
title: Version 1.0
---
### ClearML Agent 1.0.0
**Features**
* Add conda and pip environment debug prints (using `--debug`)
* Add support for PyJWT v2
* Change the default conda channel order, so it pulls the correct `pytorch` package
* Improve k8s glue support
* Support k8s glue container env vars merging
* Add number of pods limit to k8s glue using the `max_pods_limit` argument (use `--max-pods` switch in the k8s glue example)
* Add k8s glue default `restartPolicy=Never` to template to prevent pods from restarting
* Add `--stop` switch support for dynamic gpus
* Verify `docker` command exists when running in docker mode
* Add support for terminating dockers on `sig_term` in dynamic mode
* Add stopping message on Task process termination
* Add `agent.docker_install_opencv_libs` configuration option to enable automatic opencv libs install for faster docker spin-up (default: `true`, see [here](https://github.com/allegroai/clearml-agent/blob/4f18bb7ea0600db6ee63ecb5ad0ea8d048a272ca/docs/clearml.conf#L139))
* Add support for new container base setup script feature
* Bump virtualenv dependency version (support `v>=16,<21`)
* Add support for dynamic gpus opportunistic scheduling (with min/max gpus per queue)
* Deprecate `venv_update` in configuration (replaced by the more robust `venvs_cache`)
* Add Python 3.9 to the support table
**Bug Fixes**
* Fix agent can return non-zero error code and pods will end up restarting forever ([clearml-agent GitHub Issue 56](https://github.com/allegroai/clearml-agent/issues/56))
* Fix poetry support ([clearml-agent GitHub Issue 57](https://github.com/allegroai/clearml-agent/issues/57))
* Fix cuda version from driver does not return minor version
* Fix requirements local path replace back when using cache
* Fix k8s glue
* Fix broken k8s glue docker args parsing
* Fix empty env prevents override when merging template
* Fix venv cache crash on bad symbolic links
* Fix no docker arguments provided

77
docs/release_notes/clearml_agent/ver_1_1.md Normal file
View File

@@ -0,0 +1,77 @@
---
title: Version 1.1
---
### ClearML Agent 1.1.2
**Bug Fixes**
- Fix the `six` conflict with the new `pathlib2` version 2.3.7 and up.
### ClearML Agent 1.1.1
**Features and Bug Fixes**
- Add support for truncating task log file after reporting to server using `agent.truncate_task_output_files` configuration setting
- Fix PyJWT resiliency support
- Fix `--stop` checking default queue tag ([ClearML Agent GitHub issue #80](https://github.com/allegroai/clearml-agent/issues/80))
- Fix queue tag `default` does not exist and `--queue` not specified (try queue named `"default"`)
- Fix Python 3.5 compatibility
- Fix PY2.7 support for PyTorch
### ClearML Agent 1.1.0
:::info Breaking Changes
ClearML Agent will no longer try to use the demo server by default (change this by setting the `CLEARML_NO_DEFAULT_SERVER=0` environment variable)
ClearML k8s glue default pod label was changed to `CLEARML=agent` (instead of `TRAINS=agent`)
:::
**Features**
- Add poetry cache into docker mapping ([ClearML Agent GitHub issue #74](https://github.com/allegroai/clearml-agent/issues/74))
- Allow rewriting SSH URLs (see [here](https://github.com/allegroai/clearml-agent/commit/9456e493ac6d6495310ee084db906f9cdca8218c)), refers to [ClearML Agent GitHub PR #72](https://github.com/allegroai/clearml-agent/pull/72) ([ClearML Agent GitHub issue #42](https://github.com/allegroai/clearml-agent/issues/42))
- Add docker environment arguments log masking support, customizable using the `agent.hide_docker_command_env_vars` configuration value (see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L172)) ([ClearML Agent GitHub issue #67](https://github.com/allegroai/clearml-agent/issues/67))
- Add support for naming docker containers using the `agent.docker_container_name_format` configuration option to set a name format (disabled by default) ([ClearML issue #412](https://github.com/allegroai/clearml/issues/412))
- k8s glue
- Remove queue name from pod name, add queue name and ID to pod labels ([ClearML Agent GitHub issue #64](https://github.com/allegroai/clearml-agent/issues/64))
- Update task `status_message` for non-responsive or hanging pods
- Support the `agent.docker_force_pull` configuration option for scheduled pods
- Add docker example for running the k8s glue as a pod in a k8s cluster
- Add `agent.ignore_requested_python_version` configuration option to ignore any requested python version (default false, see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L45))
- Add `agent.docker_internal_mounts` configuration option to control containers internal mounts (non-root containers, see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L184))
- Add support for `-r requirements.txt` in the Installed Packages section
- Add support for `CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE` environment variable to override initial server connection behavior (defaults to true, allows boolean value or an explicit number specifying the number of connect retries)
- Add support for `CLEARML_AGENT_DISABLE_SSH_MOUNT` environment variable allowing to disable the auto `.ssh` mount into the docker
- Add support for `CLEARML_AGENT_SKIP_PIP_VENV_INSTALL` environment variable to skip Python virtual env installation on execute and allow providing a custom venv binary
- Add support for `CLEARML_AGENT_VENV_CACHE_PATH` environment variable to allow overriding venv cache folder configuration
- Add support for `CLEARML_AGENT_EXTRA_DOCKER_ARGS` environment variable to allow overriding extra docker args configuration
- Add support for environment variables containing bash-style string lists using shlex
- Add printout when using ClearML key/secret from environment variables
- Increase worker keep-alive timeout to 10 minutes instead of 1 minute
- Update documentation
**Bug Fixes**
- Fix auto mount `SSH_AUTH_SOCK` into docker ([ClearML Agent GitHub issue #45](https://github.com/allegroai/clearml-agent/issues/45))
- Fix package manager configuration documentation ([ClearML Agent GitHub issue #78](https://github.com/allegroai/clearml-agent/issues/78))
- Fix support for spaces in docker arguments ([ClearML GitHub issue #358](https://github.com/allegroai/clearml/issues/358))
- Fix standalone script with pre-exiting conda venv
- Fix PyYAML v5.4, v5.4.1 versions not supported
- Fix parsing VCS links starting with `git+git@` (notice `git+git://` was already supported)
- Fix Python package with `git+git://` links or `git+ssh://` conversion
- Fix `--services-mode` if the execute agent fails when starting to run with error code 0
- Fix `--stop` with dynamic gpus
- Fix support for unicode standalone scripts, changing default `ascii` encoding to `UTF-8`
- Fix venv cache cannot reinstall package from git with http credentials
- Fix `PYTHONIOENCODING` environment variable is overwritten when already defined
- k8s glue
- Fix support for multiple k8s glue instances with pod limits
- Fix task container handling fails parsing docker image
- Fix task container is not set when using default image/arguments
- Fix task container image arguments are used when no image is specified
- Fix task container arguments not supported in when template is not provided
- Fix `agent.extra_docker_bash_script` not applied correctly
- Fix task runtime properties are removed when re-enqueuing task
- Fix error is not thrown when failing to push task to queue

59
docs/release_notes/clearml_agent/ver_1_2.md Normal file
View File

@@ -0,0 +1,59 @@
---
title: Version 1.2
---
### ClearML Agent 1.2.3
**Bug Fixes**
- Fix PYTHONPATH is overwritten when executing a task (append to it instead)
- Fix pytorch package is reinstalled when the same version is already installed
- Fix copying configuration sets an empty worker name
- Protect dynamic GPUs from failing to parse worker GPU index
### ClearML Agent 1.2.2
**Bug Fixes**
- Fix `CLEARML_AGENT_SKIP_PIP_VENV_INSTALL` fails to find python executable
- Fix `apt-get update` failure causes `apt-get install` not to be executed
### ClearML Agent 1.2.1
**New Features and Improvements**
- Update S3 bucket verify option for minio ([ClearML Agent GitHub PR #83](https://github.com/allegroai/clearml-agent/pull/83))
- Add environment variable for request method ([ClearML Agent GitHub PR #91](https://github.com/allegroai/clearml-agent/pull/91))
- Add additional k8s-glue dockerfiles ([ClearML Agent GitHub PR #94](https://github.com/allegroai/clearml-agent/pull/94) )
- Update default docker image to `nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04`
- Add support for custom docker image resolving using the `agent.default_docker.match_rules` configuration setting (see [here](https://github.com/allegroai/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L169))
- Add `agent.force_git_root_python_path` configuration setting to force adding the git repository root folder to the `PYTHONPATH` (if set working directory is not added to the `PYHTONPATH`)
- Add `build --force-docker` command line argument to the to allow ignoring task container data
- Add `agent.poetry_version` configuration setting to specify poetry version (and force installation of poetry if missing, see [here](https://github.com/allegroai/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L65))
- Add custom build script support
- Add extra configurations when starting daemon
- Add `agent.package_manager.force_original_requirements` configuration option, allowing to only use original requirements produced by local execution (note that using this configuration option prevents editing installed packages using the UI)
- Add support for the `CLEARML_AGENT_PROPAGATE_EXITCODE` environment variable. Set this variable to `1` to allow ClearML Agent to return a nonzero exit code on failure
- Update `clearml-agent init` (use `app.clear.ml` as default server, add git token references)
**Bug Fixes**
- Fix virtualenv python interpreter used ([ClearML Agent GitHub PR #98](https://github.com/allegroai/clearml-agent/pull/98))
- Fix typing package incorrectly required for Python>3.5 ([ClearML Agent GitHub PR #103](https://github.com/allegroai/clearml-agent/pull/103))
- Fix symbolic links not copied from cached VCS into working copy (windows platform will result with default copy content instead of original symbolic link) ([ClearML Agent GitHub PR #89](https://github.com/allegroai/clearml-agent/pull/89))
- Fix agent fails to check out code from main branch when branch/commit is not explicitly specified ([ClearML GitHub issue #551](https://github.com/allegroai/clearml/issues/551))
- Fix `git+git://` requirements
- Fix `default_python` calculation (and verbosity)
- Fix using deprecated `abc` support (Python 3.10 compatibility)
- Fix no default value for `CLEARML_API_DEFAULT_REQ_METHOD` causes `ValueError` if not specified
- Fix `agent.hide_docker_command_env_vars` mode to include URL passwords and handle environment vars containing docker commands
- Fix conda package manager listed packages with local links (`@ file://`) should ignore the local package if it does not exist
- Fix cuda patch version support in conda
- Fix agent attempts to check out code when in standalone mode
- Fix `FORCE_LOCAL_CLEARML_AGENT_WHEEL` environment variable handling when running from a Windows host
- Fix user-provided `"` is unnecessarily replaced to `\\"`
- Fix token is not propagated to docker in case credentials are not available
- Fix PyTorch aarch64 and windows support
- Fix VCS packages are reinstalled when the same commit version is already installed
- Fix git packages are installed even if commit is given and is preinstalled when using cached virtual environment

20
docs/release_notes/clearml_agent/ver_1_3.md Normal file
View File

@@ -0,0 +1,20 @@
---
title: Version 1.3
---
### ClearML Agent 1.3.0
**New Features and Improvements**
* Support private repos from `requirements.txt` file ([ClearML Agent GitHub PR #107](https://github.com/allegroai/clearml-agent/pull/107))
* Bump PyJWT version due to "*Key confusion through non-blocklisted public key formats*" vulnerability
* Add support for additional command line arguments in k8s glue example
* Add Python 3.10 support
**Bug Fixes**
* Fix git unsafe directory issue (disable check on cached vcs folder)
* Fix dynamic GPUs with "all" GPUs on the same worker
* Fix broken pytorch setuptools incompatibility (force setuptools < 59 if torch is below 1.11)
* Fix setuptools requirement issue by making sure that if we have "setuptools" in the original required packages, we preserve the line in the pip freeze list
* Fix optional priority packaged always compare lower case package name
* Fix potential requirement installation failure by making `pygobject` an optional package (i.e. if installation fails
continue the Task package environment setup)
* Fix repository URL contains credentials even when `agent.force_git_ssh_protocol: true`

45
docs/release_notes/clearml_agent/ver_1_4.md Normal file
View File

@@ -0,0 +1,45 @@
---
title: Version 1.4
---
### ClearML Agent 1.4.1
**New Features and Improvements**
* Add warning if venv cache is disabled
* Add `agent.disable_ssh_mount` configuration option (same as the `CLEARML_AGENT_DISABLE_SSH_MOUNT` environment variable)
**Bug Fixes**
* Fix docker command for monitoring child agents
* Fix `--gpus all` not reporting GPU stats on worker machine
### ClearML Agent 1.4.0
**New Features and Improvements**
* Add support for MIG devices (use `0:1` for GPU 0 slice 1, or use 0.1)
* Add `agent.enable_git_ask_pass` to improve passing user/pass to git commands
* Add docker `ssh_ro_folder` (default `/.ssh`) and changed docker `ssh_folder `(default: `~/.ssh`)
* Allow overriding pytorch lookup page (See `torch_page`, `torch_nightly_page` and `torch_url_template_prefix` under the
`agent.package_manager` configuration settings)
* Add support for abort callback registration
* K8s glue
* Add `CLEARML_K8S_GLUE_START_AGENT_SCRIPT_PATH` environment variable to allow customizing the agent startup script location
* Add debug environment variable `CLEARML_AGENT_DEBUG_INFO`
* Add `CLEARML_AGENT_CHILD_AGENTS_COUNT_CMD` environment variable to allow overriding child agent count command in k8s
* Refactor template handling
**Bug Fixes**
* Fix Python 3.10+ support
* Fix `use_credentials_chain` is missing in config file example
* Fix Git PAT messages
* Fix home folder in `clearml.conf` to ~ (instead of `/root`)
* Fix docker mode uses `~/.clearml/venvs-builds` as default for easier user-mode containers
* Fix package `@ file://` with quoted (URL style) links should not be ignored
* Fix name not escaped as regex (all services "get_all" use regex for name)
* Fix second .ssh temp mount fails if container changes the files inside
* Fix GCP load balancer does not forward GET request body (allow changing default request action to `PUT/POST/GET`. See
`api.http.default_method` or `CLEARML_API_DEFAULT_REQ_METHOD`)
* K8s glue
* Fix resolving k8s pending queue may cause a queue with a UUID name to be created
* Fix template namespace should override default namespace
* Fix `extra_bash_init_cmd` location in initial bash script
* Fix debug mode
* Fixed documentation ([ClearML Agent PR #117](https://github.com/allegroai/clearml-agent/pull/117))

67
docs/release_notes/clearml_agent/ver_1_5.md Normal file
View File

@@ -0,0 +1,67 @@
---
title: Version 1.5
---
### ClearML Agent 1.5.2
**New Features and Improvements**
* Switch services agent entrypoint shell from sh to bash ([ClearML Agent GitHub issue #140](https://github.com/allegroai/clearml-agent/issues/140))
* Improve poetry support
* Add poetry cwd support ([ClearML Agent GitHub issue #138](https://github.com/allegroai/clearml-agent/issues/138))
* Add `agent.package_manager.poetry_install_extra_args` configuration option
* Do not allow request exceptions (keep retrying, throw error only on the initial login call)
**Bug Fixes**
* Fix agent update version ([ClearML Agent GitHub PR #132](https://github.com/allegroai/clearml-agent/pull/132))
* Fix login uses `GET` with payload, which breaks when trying to connect a server running in GCP
* Fix `clearml-agent build --docker` stuck on certain containers
* Fix build fails when target is relative path
* Fix pinging running task (change default to once a minute)
* Fix `_` is allowed in k8s label names
* Fix k8s glue does not delete pending pods if the tasks they represent were aborted
* Reintroduce `CLEARML_AGENT_SERVICES_DOCKER_RESTART` accidentally reverted by a previous merge
* Fix `git+ssh://` links inside installed packages not being converted properly to HTTPS-authenticated links
### ClearML Agent 1.5.1
**New Features and Improvements**
* Upgrade requirements for `attrs`, `jsonschema`, `pyparsing`, `six`, and `pyjwt` ([ClearML Agent GitHub issue #129](https://github.com/allegroai/clearml-agent/issues/129))
* Add default output URI selection to `clearml-agent init`
* Add `agent.disable_task_docker_override` configuration option to disable docker override specified in executing tasks
* Add `CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGES` env var (default `true`) to allow overriding default `system_site_packages: true`
behavior when running tasks in containers (docker mode and k8s-glue)
**Bug Fixes**
* Fix using deprecated types validator argument raises an error (deprecated even before `jsonschema` 3.0.0 and unsupported
since 4.0.0)
* Fix pip support allowing multiple pip version constraints (by default, one for < Python 3.10 and one for >= Python 3.10)
### ClearML Agent 1.5.0
**New Features and Improvements**
* Add option to crash agent on exception using `agent.crash_on_exception` configuration setting ([ClearML Agent GitHub issue #122](https://github.com/allegroai/clearml-agent/issues/122))
* Improve venv cache disabled message
* Upgrade packages for better Python 3.10 support
* Remove future package dependency (Python 2 is not supported for `clearml-agent`)
* Change default pip version used to `pip<21` for better Python 3.10 support
* Add support for operator `!=` in package version (mostly for better PyTorch resolving)
* Add support for PyTorch new `extra_index_url` repo (find the correct index url based on the cuda version, and let pip
do the rest)
* Make venv caching the default behavior
* Add support for `CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV` environment variable (see `agent.hide_docker_command_env_vars`
config option)
* Ping executing tasks to make sure the server does not consider them stale (set using the `agent.task_ping_interval_sec`
configuration option, defaults to every 120 seconds)
**Bug Fixes**
* Fix docker extra arguments showing up in configuration printout
* Fix an issue with running on Python 3.10 / 3.11
* Fix cached git token prevents cloning repository (using `agent.enable_git_ask_pass` forcing the agent to use `GIT_ASKPASS`
for user/password when cloning/fetching repositories)
* Fix setting `CLEARML_API_DEFAULT_REQ_METHOD` raises an error
* Fix `get_task_session()` may cause an old copy of the `APIClient` to be used containing a reference to the previous session
* K8s Glue
* Fix `agent.system_site_packages` is not turned on by default in k8s glue
* Make sure git_user/pass is passed to the task pod
* Remove support for `kubectl run`

40
docs/release_notes/clearml_agent/ver_1_6.md Normal file
View File

@@ -0,0 +1,40 @@
---
title: Version 1.6
---
### ClearML Agent 1.6.1
**Bug Fix**
* Fix `requests` requirement lower constraint breaks backwards compatibility for Python 3.6
### ClearML Agent 1.6.0
**New Features and Improvements**
* Upgrade `requests` library ([ClearML Agent GitHub issue #161](https://github.com/allegroai/clearml-agent/issues/161))
* Add support for controlling PyTorch resolving mode using the `CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE` environment
variable and `agent.package_manager.pytorch_resolve configuration` setting with `none` (no resolving), `pip` (sets extra
index based on cuda and lets pip resolve) or `direct` (the previous parsing algorithm that does the matching and downloading). Default is `pip` ([ClearML Agent GitHub issue #152](https://github.com/allegroai/clearml-agent/issues/152))
* Add backwards compatibility in standalone mode using the `CLEARML_AGENT_STANDALONE_CONFIG_BC` environment variable
* Add `CLEARML_AGENT_DOCKER_AGENT_REPO` alias for the `FORCE_CLEARML_AGENT_REPO` environment variable
* Show a better message for `agent init` when an existing `clearml.conf` is found
* Add support for task field injection into container docker name using the `agent.docker_container_name_format_fields`
configuration setting
* Add support for adding additional labels to docker containers using the `CLEARML_AGENT_EXTRA_DOCKER_LABELS` environment variable
* Add support for setting file mode in files applied by the agent (using the `files` configuration option) using the
`mode` property
* Add support for skipping agent pip upgrade in the default k8s pod container bash script using the `CLEARML_AGENT_NO_UPDATE` environment variable
* Add support for additional pip install flags when installing dependencies using the `CLEARML_EXTRA_PIP_INSTALL_FLAGS`
environment variable and `agent.package_manager.extra_pip_install_flags` configuration option
* Add support for extra docker arguments referencing machines environment variables using the `agent.docker_allow_host_environ`
configuration option, allowing users to use `$ENV` in the task docker arguments (e.g. `-e HOST_NAME=$HOST_NAME`)
* Add support for k8s jobs execution (as opposed to only pods)
* Update default docker image versions
* Add Python 3.11 support
**Bug Fixes**
* Fix `git+ssh://` links inside installed packages not being properly converted to authenticated `https://` and vice versa
* Fix pip version required in the "Installed Packages" is now preserved and reinstalled
* Fix various agent paths not loaded correctly if an empty string or null is used (should be disabled, not converted to `.`)
* Fix docker container backwards compatibility for `API<2.13`
* Fix default docker match rules resolver (used incorrect field `"container" instead of "image")
* Fix task docker argument might be passed twice (might cause an error with flags such as `--network` and `--ipc`)

36
docs/release_notes/clearml_agent/ver_1_7.md Normal file
View File

@@ -0,0 +1,36 @@
---
title: Version 1.7
---
### ClearML Agent 1.7.0
**New Features**
* Add `agent.docker_args_extra_precedes_task` and `agent.protected_docker_extra_args` configuration settings to prevent
the same switch to be used by both `agent.extra_docker_args` and a Task's docker args
* Add `agent.resource_monitoring.disk_use_path` configuration option to allow monitoring a different volume than the one
containing the home folder
* Change default `agent.enable_git_ask_pass` to `true`
* Add example and support for pre-built containers including services-mode support with overrides `CLEARML_AGENT_FORCE_CODE_DIR`
and `CLEARML_AGENT_FORCE_EXEC_SCRIPT`
* Add `CLEARML_AGENT_SERVICE_TASK=1` environment variable in case of running a service task
* Add `CLEARML_AGENT_TEMP_STDOUT_FILE_DIR` to allow specifying temp dir used for storing agent log files and temporary
log files (daemon and execution)
* Update GPU stats and pynvml support
* Add git clone verbosity using `CLEARML_AGENT_GIT_CLONE_VERBOSE` environment variable
* k8s glue
* Add status reason when aborting before moving to `k8s_scheduler` queue
* When cleaning up pending pods, verify task is still aborted and pod is still pending before deleting the pod
* Set worker ID in k8s pod execution
**Bug Fixes**
* Fix `agent.package_manager.poetry_install_extra_args` are used in all Poetry commands and not just in `install` ([ClearML Agent GitHub issue #173](https://github.com/allegroai/clearml-agent/issues/173))
* Fix if process return code is SIGKILL (-9 or 137) and abort callback was called, do not mark as failed but as aborted
* Fix `agent.git_host` setting will cause git@domain URLs to not be replaced by SSH URLs since furl cannot parse them to
obtain host
* Fix an environment variable that should be set with a numerical value of 0 (i.e. end up as `"0"` or `"0.0"`) is set to
an empty string
* Fix `agent.package_manager.extra_index_url` URLs are not sanitized in configuration printout
* Fix recursion issue when deep-copying a session
* k8s glue
* Fix k8s glue configuration might be contaminated when changed during apply
* Fix `KeyError` if container does not contain the arguments field

45
docs/release_notes/clearml_agent/ver_1_8.md Normal file
View File

@@ -0,0 +1,45 @@
---
title: Version 1.8
---
### ClearML Agent 1.8.1
**New Features**
* Add option to set daemon polling interval ([ClearML Agent GitHub PR #197](https://github.com/allegroai/clearml-agent/pull/197))
* Add Python 3.12 support
**Bug Fixes**
* Fix git pulling on cached invalid git entry. On error, re-clone the entire repository again (enable using `agent.vcs_cache.clone_on_pull_fail: true`)
* Fix conda env should not be cached if installing into base conda or conda existing env
* Fix cached repositories not passing user/token when pulling
* Fix when disabling vcs cache do not add vcs mount point to container
### ClearML Agent 1.8.0
**New Features**
* Add `CLEARML_AGENT_FORCE_POETRY` environment variable to allow forcing poetry even when using pip requirements manager
* Add `CLEARML_AGENT_FORCE_TASK_INIT` environment variable to allow runtime patching of script even if no repository is
specified and the code is running a preinstalled docker
* Improve venv cache handling:
* Add `FileLock` readonly mode, default is write mode (i.e. exclusive lock, preserving behavior)
* Add venv cache now uses readonly lock when copying folders from venv cache into target folder. This enables multiple read, single write operation
* Do not lock the cache folder if we do not need to delete old entries
* Add `agent.venvs_cache.lock_timeout` to control the venv cache folder lock timeout (in seconds, default 30)
* Add protection for `truncate()` call
* Move configuration sanitization settings to the default config file
* Add queue ID report before pulling task
* Improve GPU monitoring for MIGs
**Bug Fixes**
* Use correct Python version in Poetry init ([ClearML Agent GitHub PR #179](https://github.com/allegroai/clearml-agent/pull/179))
* Fix queue handling in `K8sIntegration` and `k8s_glue_example.py` ([ClearML Agent GitHub PR #183](https://github.com/allegroai/clearml-agent/pull/183))
* Fix `FileNotFoundException` crash in `find_python_executable_for_version` ([ClearML Agent GitHub issue #164](https://github.com/allegroai/clearml-agent/issues/164))
* Fix delete temp console pipe log files after Task execution is completed (important for long-lasting services agents to avoid collecting temp files on host machine)
* Fix `agent.enable_git_ask_pass` does not show in configuration dump
* Fix `pip` is returned as a pip version if no value exists in `agent.package_manager.pip_version`
* Fix Python 3.12 support by removing `distutil` imports
* Fix `IOError` on file lock when using shared folder
* Fix torch resolver settings applied to `PytorchRequirement` instance are not used
* Fix comment lines (starting with `#`) are not ignored in docker startup bash script
* Fix dynamic GPU sometimes misses the initial print

45
docs/release_notes/clearml_agent/ver_1_9.md Normal file
View File

@@ -0,0 +1,45 @@
---
title: Version 1.9
---
### ClearML Agent 1.9.1
**New Features and Improvements**
* Add default pip version support for Python 3.12
### ClearML Agent 1.9.0
**New Features and Improvements**
* Add `NO_DOCKER` flag to `clearml-agent-services` entrypoint ([ClearML Agent GitHub PR #206](https://github.com/allegroai/clearml-agent/pull/206))
* Use `venv` module if `virtualenv` is not supported
* Find the correct python version when using a pre-installed python environment
* Add `/bin/bash` support in the task's `script.binary` property
* Add support for `.ipynb` script entry files (install nbconvert in runtime, convert file to python and execute the
python script). Includes `CLEARML_AGENT_FORCE_TASK_INIT` patching of `.ipynb` files (post-python conversion)
* Add `CLEARML_MULTI_NODE_SINGLE_TASK` (values -1, 0, 1, 2) for easier multi-node single Task workloads
* Add default docker `agent.default_docker.match_rules` configuration option (note: `matching_rules` are ignored if `--docker container` is passed in command line)
* Add `-m module args` in script entry now supports standalone script. Standalone script is placed in a file specified
by the `working_dir` setting in the `<dir>:<target_file>` format (e.g. `:standalone.py`), or in `untitled.py` if not specified
* Add `K8S_GLUE_POD_USE_IMAGE_ENTRYPOINT` env var to allow running k8s pods without overriding the image entrypoint
(useful for agents using prebuilt images in k8s)
* Add venv cache mount override for non-root containers (use: `agent.docker_internal_mounts.venvs_cache`)
* Add `/bin/bash -c "command"` support. Task binary should be set to `/bin/bash` and `entry_point` should be set to `-c command`
* Add support for tasks containing only bash script or python module command
* Add support for skipping container apt installs using `CLEARML_AGENT_SKIP_CONTAINER_APT` env var in k8s
**Bug Fixes**
* Fix git fetch did not update new tags ([ClearML Agent GitHub issue #209](https://github.com/allegroai/clearml-agent/issues/209))
* Fix file mode should be optional in configuration `files` section
* Fix `-m module $env` to support parsing `$env` before launching
* Fix setting tasks that were just marked as `aborted` to `started` - only force task to `started` after dequeueing it,
otherwise do nothing
* Fix slurm multi-node rank detection
* Fix passing only `--docker` (i.e. no default container image) when using `--dynamic-gpus` feature
* Fix logger object used even if `None`
* Fix a race condition where in rare conditions popping a task from a queue that was aborted did not set it to `started`
before the watchdog killed it (not applicable in k8s/slurm)
* Fix multi-node support to only send `pip freeze` update, only set task as `started`, and only update task status on exit for RANK 0
* Fix do not cache venv cache if venv/python skip env var was set
* Fix use same state transition in k8s if supported by the server (instead of stopping the task before re-enqueue)
* Fix failed Task in services mode logged as `User aborted` instead of `failed`. Add Task state reason string
* Fix remove task from pending queue and set to `failed` in k8s when applying the pod template fails

59
docs/release_notes/clearml_server/enterprise/ver_3_20.md Normal file
View File

@@ -0,0 +1,59 @@
---
title: Version 3.20
---
### Enterprise Server 3.20.4
**Bug Fix**
* Fix access cannot be granted to UI Application projects through access rules
### Enterprise Server 3.20.3
**Bug Fix**
* Fix batch deletes from files server sometimes does not work
### Enterprise Server 3.20.2
**Bug Fixes**
* Fix clicking on UI Metric Snapshot datapoint does not navigate to experiment
* Fix missing experiment labels in UI experiment plot comparison
* Fix starting time not updating for some tasks
* Fix S3/Minio file deletion broken in non-default region
### Enterprise Server 3.20.1
**New Features and Improvements**
* Add support for serving UI from a subpath
**Bug Fixes**
* Fix UI experiment plot comparison displays duplicates of image plots
* Fix downloading UI experiment console log sometimes returns an older file
### Enterprise Server 3.20.0
**New Features and Improvements**
* Add Administrator identity provider management UI: administrators can add and manage multiple identity providers
* New UI experiment table comparative view: compare plots and scalars of all selected experiments
* Add UI project metric snapshot support for multiple metrics
* Add UI experiment display of original python requirements along with actual packages used.
* Add compressed UI experiment table info panel mode displaying only experiment name and status
* Add "x unified" hover mode to UI plots
* Add option to view metadata of published dataset versions in UI Hyper-Dataset list view
* Add experiment hyperparameter UI section dataset IDs link to datasets page
* Make all URLs clickable in UI experiment artifacts
* Redesign WebApp tabs: Tabs now appear in UI header
* Add option to force setting original task as clone's parent when cloning a task
**Bug Fixes**
* Fix CVE-2023-45133 Web App vulnerability
* Fix "select all" in UI dataset archive counts also unarchived datasets
* Fix hyperparameter keys with dots in their names displayed incorrectly in UI experiment table and task info
* Fix UI experiment scalar comparison raising error
* Fix “Project” column redundantly appears inside a specific project's model table
* Fix UI models with many labels slowing down web app
* Fix Hyper-Dataset version list collapsed by default in UI DataView preview
* Fix when invalid source query filter is applied to UI Hyper-Dataset version, navigating to frame raises an error
* Fix can't select root project in UI report creation modal when workspace has no projects
* Fix UI plot "Show/hide" legend button sometimes disappears
* Fix "load more" button displayed unnecessarily in UI object tables

64
docs/release_notes/clearml_server/enterprise/ver_3_21.md Normal file
View File

@@ -0,0 +1,64 @@
---
title: Version 3.21
---
### Enterprise Server 3.21.4
**New Feature**
* Support setting up a supervisor company that can monitor the resource dashboards of all tenants on the server
### Enterprise Server 3.21.3
**New Feature**
* Allow hiding Dataset and Orchestration menu options for non-admins
### Enterprise Server 3.21.2
**New Feature**
* Add support for relative path in displayed server URL
**Bug Fix**
* Fix workspace name not displayed in generated credentials
### Enterprise Server 3.21.1
**Bug Fixes**
* Fix ROI bounding box appears in wrong location in UI frame viewer
* Fix `Dataset.delete_version()` is sometimes slow
### Enterprise Server 3.21.0
**New Features and Improvements**
* Service accounts - admins can define and provide API access to service accounts separate from full fledged ClearML users
* UI experiment hyperparameter comparison
* Add Parallel coordinate view support for multiple metrics
* Add scatter plot mode to UI experiment hyperparameter comparison
* Add UI Hyper-Dataset filtered frameviewer shareable link
* Add model lineage to UI model comparison
* Add copy-to-clipboard on click of Hyper-Dataset ID
* Add expanded view for UI Orchestration Dashboard console log
**Bug Fixes**
* Fix insufficient session expiration vulnerability (CWE-613)
* Fix XSS vulnerability of HTML debug samples
* Fix UI "No tags" filter raises an error
* Fix UI experiments sometimes missing plots
* Fix delete not working in batch action bar in UI Dataviews and Models tables
* Fix dequeue not working from UI experiment table batch action bar
* Fix UI dashboard search returns results from team’s work when "My Work" filter is enabled.
* Fix UI plots and scalars are available in reset experiments
* Fix data retrieval is slow in UI Orchestration Dashboard Resource Graph
* Fix slow loading speed for UI experiment comparison scalars
* Fix UI scalar value comparison sometimes displays empty page
* Fix relaunching a UI application instance launches two instances
* Fix using filter causes UI experiment console to jump
* Fix embedded scalar comparison plot does not display metric variant in maximized view in UI Reports
* Fix "copy embed code" not working properly for comparison plot
* Fix UI embedded plots missing titles and axis labels
* Fix incorrect plot name is displayed when expanding UI scalar plots
* Fix UI Project Dashboard Application wizard project selection is slow
* Fix UI experiment scalar comparison missing active filter indicator
* Fix Dataview cloned in Dataview archived created inside archive
* Fix missing status filter in UI All Models table
* Fix UI pipeline table "No value" selection does not work with version filter
* Fix task IDs not displayed in UI experiment comparison graph titles and legends

84
docs/release_notes/clearml_server/enterprise/ver_3_22.md Normal file
View File

@@ -0,0 +1,84 @@
---
title: Version 3.22
---
### Enterprise Server 3.22.6
**New Features**
* Add support for datetime values in `MultiFieldParameters` query
* Add option to show/hide specific UI applications
### Enterprise Server 3.22.5
**Bug Fix**
* Fix users in the Queue Admins user group cannot modify queues
### Enterprise Server 3.22.4
**New Feature**
* Add support for group integration in Keycloak SSO with environment variables
### Enterprise Server 3.22.3
**New Feature**
* Add links to queues in UI Resource Configuration Resource Pools details
**Bug Fix**
* Fix default value of the Callback URL field in the UI Identity Providers’ Connection Configuration modal contains two slashes (`//`)
### Enterprise Server 3.22.2
**New Feature**
* Add fractional GPU support to the UI resource dashboard
### Enterprise Server 3.22.1
**Bug Fixes**
* Fix XSS vulnerability (CWE-79)
* Fix UI scalar plot not displayed if metric name includes a slash surrounded by spaces (` / `)
* Fix S3 Credentials modal opens unnecessarily in UI Experiment Debug Samples
* Fix failure to upload files to ClearML file server
### Enterprise Server 3.22.0
**New Features and Improvements**
* New UI Resource Policy Manager - Admins can define resource quotas and reservations to prioritize resource utilization.
Please contact our sales team for more information on how to enable this feature for your account
* New UI new experiment creation
* Improve UI Hyper-Dataset version statistics
* Real-time calculation
* Calculate statistics on frame filter query results
* Calculate statistics for user choice of frame/ROI field
* Improve UI Hyper-Dataset version preview
* New sort options by any frame/ROI property
* New grouped sampling preview by any frame/ROI property
* Add "Block running user's scripts in the browser" UI setting option for added security
* Implement lexical sorting for UI Hyper-Datasets frame metadata
**Bug Fixes**
* Fix adding an annotation to a UI hyper-dataset frame and moving to the next frame creates redundant copies of the added annotation
* Fix large number of tags in UI tables' tag filter causes UI to crash
* Fix searching in UI object tables' tag filter causes UI to crash
* Fix published annotation task is editable
* Fix UI DataView label enumeration is numbered incorrectly when label mapping is used
* Fix model plots not displayed in UI model plot tab
* Fix UI experiment Debug Samples tab freezes when a lot of debug samples are reported in the experiment
* Fix sub-path does not appear in credentials generated in the UI
* Fix consistent plot colors not maintained when embedded in UI Reports
* Fix UI single-scalar plots are not ordered alphabetically by scalar name
* Fix UI scalar “Export to CSV” button returns an empty CSV file
* Fix preview of GUI application plot does not display when embedded in UI Reports
* Fix queue experiment list is not updated in the UI Orchestration's Queues page
* Fix numeric range filters of UI table columns not working
* Fix UI DataView repetition type not updated after modification
* Fix UI DataView frame filter ratio displays incorrect number
* Fix adding label enumeration while creating a UI DataView prevents inputting the rest of the data
* Fix image names change in UI task debug sample comparison when a different metric is selected
* Fix duplicate labels can be added to the same UI annotation
* Fix My work/Team work persists across UI modules (i.e. projects, datasets, pipelines, etc.)
* Fix multiple parameters can be selected for plot axes in scatter plot view in UI experiment comparison
* Add confirmation pop-up when running experiment is archived
* Fix multiple Hyper-Dataset frames with the same source are displayed only once in the UI DatasetVersion previews
* Fix GUI app instance filters (My/Running/All instances) persist across applications
* Fix UI plot legends and tooltips wrapping

43
docs/release_notes/clearml_server/open_source/ver_0_10.md Normal file
View File

@@ -0,0 +1,43 @@
---
title: Version 0.10
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.10.1
* Add simple username / password login scheme, see [GitHub trains-server Issue 3](https://github.com/allegroai/trains-server/issues/3)
* Fixed User/Password configuration (see [instructions](../../../deploying_clearml/clearml_server_config.md#web-login-authentication)).
* Replace the Python web-server with NGINX.
* Improve sub-domain support ([GitHub trains-server Issue 9](https://github.com/allegroai/trains-server/issues/9)).
* Extend configuration options
* Environment variables for overriding mongoDB and Elastic hosts URLs.
* File server disabling browser cache.
* File server compression support.
* File server CORS support.
### Trains Server 0.10.0
* Experiment code execution detection.
* Add source-control diff section.
* Add code requirements section.
* Automatically stop inactive experiments (default: 2 hours).
* Configuration details in readme.md:
* Improve visibility:
* Finer status definitions: Identify successful completion vs. user aborted.
* Experiment plot comparison: Ensure different colors for different experiments.
* Parse newline character in experiment description.
* Show experiment start time in table display.
* Add vertical guide in scalar plots.
* Move hyperparameters to the designated tab.
* "Admin" section now named "Profile".

18
docs/release_notes/clearml_server/open_source/ver_0_11.md Normal file
View File

@@ -0,0 +1,18 @@
---
title: Version 0.11
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.11.0
* Add scalar graphs smoothing (matched TensorBoard behavior).
* Add scalar graphs X-axis selection: Iteration / Relative / Wall Time.
* Add min / max / last value for all scalars (available in experiment table column selection / sorting).
* Add hyperparameter to experiment table custom columns.
* Add artifacts support (Models tab moved into artifacts as specific artifact type).
* Add full Azure storage support.
* Fix column sorting N/A values.

21
docs/release_notes/clearml_server/open_source/ver_0_12.md Normal file
View File

@@ -0,0 +1,21 @@
---
title: Version 0.12
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.12.1
Minor bug fixes and improvements
* Add resource monitoring.
* Fix Web UI compare plots ([GitHub Issue #55](https://github.com/allegroai/clearml/issues/55)).
* Improve server upgrade checks/messages.
### Trains Server 0.12.0
* Add full DevOps support.
* Add job scheduling queue management.
* Add cluster management (TRAINS Agent workers).
* Add system wide auto-refresh.

27
docs/release_notes/clearml_server/open_source/ver_0_13.md Normal file
View File

@@ -0,0 +1,27 @@
---
title: Version 0.13
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.13.0
**Features and Bug Fixes**
* Add parallel coordinates hyperparameter comparison, available under **Compare Experiments** **>** **HYPERPARAMETERS**
**>** **Parallel Coordinates** (in the drop-down) ([GitHub Issue #53](https://github.com/allegroai/trains/issues/53)).
* Add encoding of experiment table view settings in URL to allow sharing, using browser URL copy / paste.
* Add loguru (ANSI color) support ([GitHub Issue #29](https://github.com/allegroai/trains/issues/29)).
* Add support for special characters in hyperparameter keys (white-spaces, `.` and `$`) ([GitHub Issue #69](https://github.com/allegroai/trains/issues/69)).
* Add optional anonymous daily usage statistics (help us improve Trains Server):
* Disabled by default.
* Requires user opt-in.
* Single averages report per day.
* Reports average load metrics per day (CPU / memory).
* Reports average workload per day (amount and average duration of queues, agents and experiments).
* Improve experiment table filtering indication.
* Improve model view to allow navigating to its generating experiment.
* Fix experiment comparison to distinguish between experiments with the same name ([GitHub Issue #52](https://github.com/allegroai/trains/issues/52)).
* Fix Web UI compare plots bug ([GitHub Issue #55](https://github.com/allegroai/trains/issues/55) and [GitHub Issue #73](https://github.com/allegroai/trains/issues/73)).

51
docs/release_notes/clearml_server/open_source/ver_0_14.md Normal file
View File

@@ -0,0 +1,51 @@
---
title: Version 0.14
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.14.2
**User survey**
* Add survey invitation popup.
**Bug fixes**
* Fix misaligned titles in experiment results, scalars, and plots (**RESULTS** tab, **SCALARS** and **PLOTS** sub-tabs).
* Fix parallel coordinate comparison in comparing experiments. Adding dimension with no values removes all lines.
* Fix resizing the experiments tables window. The preferences wheel jumped over the titles.
* Fix the browser zoom (**CTRL +**, **CTRL -**, and mouse wheel) in experiment **RESULTS** **>** **SCALARS** tab. It broke labels ([GitHub Issue #127](https://github.com/allegroai/trains/issues/127)).
### Trains Server 0.14.1
**Features and Bug Fixes**
* Support default **Trains Server** Kubernetes Helm ports in the **Trains Web-App (UI)**.
* Fix issue switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/allegroai/trains/issues/75)).
* Do not reset view filtering when switching tabs in experiment view.
### Trains Server 0.14.0
**Features and Bug Fixes**
* Add debug image viewing for an unlimited number of iterations, filtering by metric, and comparing debug images for different iterations.
* Add debug image view zoom, fit to screen, resize, hover coordinates ([GitHub Issue #68](https://github.com/allegroai/trains/issues/68)).
* Add switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/allegroai/trains/issues/75)).
* Add scalar minimums and maximums to comparing experiments.
* Add opening HTML artifact files in the browser.
* Add Web-App notification for version updates.
* Add toggle button to switch between full screen and standard views in the **RESULTS** tab.
* Add a load more button to the global search results.
* Add automatic detection of debug video type from the file extension.
* Add `Task.get_parameters_as_dict` and `Task.set_parameters_as_dict`.
* Allow editing model description.
* Allow editing experiment uncommitted changes and installed packages separately.
* Allow deleting an input model from an experiment whose status is *Draft*.
* Fix dialog elements which are now accessible in non-100% screen scale. ([GitHub Issue #89](https://github.com/allegroai/trains/issues/89)).
* Fix custom columns in the experiments table not properly loaded from browser store ([GitHub Issue #106](https://github.com/allegroai/trains/issues/106)).
* Fix issue with showing 2D graphs containing sub-plots ([trains #slack](https://github.com/allegroai/trains/issues/106))
* Fix issues with vertical scrolling of experiment logs.
* Fix issue with image plots sometimes not visible in experiments comparison.

94
docs/release_notes/clearml_server/open_source/ver_0_15.md Normal file
View File

@@ -0,0 +1,94 @@
---
title: Version 0.15
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.15.1
**Bug Fixes**
* Fix experiment clone dialog resets while typing ([GitHub trains-server Issue 44](https://github.com/allegroai/trains-server/issues/44)).
* Fix image-only plots are not displayed ([GitHub Issue #140](https://github.com/allegroai/trains/issues/140)).
* Fix last item in plots list is hidden.
* Fix input model edit view contains a redundant save button.
* Fix credentials popup shows incorrect URL for fileserver.
* Fix experiment description does not allow multiple lines.
* Fix tags filter ("No tags" selection, display only tags available for the project).
* Fix cannot save empty model configuration.
* Fix add tag dialog does not close after adding a tag.
* Show active column filters for values even when values are unavailable in current project.
* Fix project selection resets when cloning an experiment.
* Fix several broken icons.
* Fix cannot add tags in Results view.
* Add warning when deleting a model/experiment tag.
* New icons for experiment/task types.
### Trains Server 0.15.0
**Features**
* Add support for debug media (images, audio, video) ([GitHub Issue #120](https://github.com/allegroai/trains/issues/120)).
* Add support for previewing HTML debug samples.
* Add experiment and model tag management ([GitHub Issue #46](https://github.com/allegroai/trains/issues/46)).
* Add support for more task types (see Trains v0.15.0 and Trains Agent v0.15.0 for related support and services mode).
* Experiment comparison:
* Add Last / Max (maximal) / Min (minimal) scalar values to parallel coordinates in experiment comparison ([GitHub Issue #131](https://github.com/allegroai/trains/issues/131)).
* Add deep comparison support to experiment comparison, including uncommitted changes by file with line diff and search of all text.
* Enhance table columns:
* Add dynamic column ordering to the experiments and models tables.
* Add the User column to the experiments table.
* Add the Description to experiments and models tables.
* Add double click to table column resizers for column width reset.
* Add column customization to the models table.
* Add support for saving and sharing customized model table URLs by embedding table customizations in the URL.
* Add a popup text editor for all text areas.
* Move resource monitoring plots after the metrics plots in the experiment results SCALARS tab.
* Add support for the clear_all parameter in tasks.reset to improve Trains SDK Task reuse.
* Unify experiments table and comparison auto-refresh (switching between table and comparison auto refresh setting did not change).
* Add a cancel button to all text searches.
* Improve the experiment / info panel split to 75% / 25%.
* Improve infrastructure:
* Update pymongo and mongoengine to their latest versions.
* Improve mongo performance by eliminating non-indexed queries.
* Upgrade WebApp to Angular 9.
* Use native javascript instead of date-fns.
* Add survey invitation popup.
**Bug Fixes**
* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/allegroai/trains-agent/issues/19)).
* Improve docker host-mount support, use `TRAINS_AGENT_DOCKER_HOST_MOUNT` environment variable.
* Support pip v20.1 local / http package reference in pip freeze.
* Fix detached mode to correctly use cache folder slots.
* Fix `CUDA_VISIBLE_DEVICES` should never be set to "all".
* Do not monitor GPU when running with `--cpu-only`
* Fix experiment comparison issues:
* Scalar comparison type does not reflect displayed comparison ([GitHub Issue #40](https://github.com/allegroai/trains/issues/40)).
* Comparison animations overlapping other fields.
* Cannot choose a metric after clearing it in experiment comparison parallel coordinates.
* Adding a dimension without a value removes all lines in experiment comparison parallel coordinates.
* Fix experiment view issues:
* Custom URL gets erased when clicking on any tab on experiment.
* All experiments re-fetched when navigating through tabs of an experiment.
* Input and output models configuration box sizes differs in the experiments **ARTIFACTS** tab.
* Changing experiment table / info panel split redraws scalars and plots.
* Wrong download URL shown for S3 artifact links with special characters.
* Fix experiment results view issues:
* Browser zoom breaks labels (Ctrl + / Ctrl - / mouse wheel) in experiment results **SCALARS** tab ([GitHub Issue #127](https://github.com/allegroai/trains/issues/127)).
* Debug image name not shown if file cannot be loaded.
* minio images with spaces in the path not displaying correctly
* Plots should use the full panel space.
* Plot relative time displaying incorrectly in experiment results **SCALARS** tab.
* Wall time display in local time in experiment results **SCALARS** tab.
* Fix experiment table issues:
* Hyperparameters sorted as strings, not as numbers ([GitHub Issue #124](https://github.com/allegroai/trains/issues/124)).
* Prevent auto refresh reordering when the info panel is open.
* Table resizing made the preferences wheel to jump over titles.
* Fix experiment reset when both the input and output model are the same causes model deletion.
* Fix experiment clone not working if the input model was deleted.
* Fix missing trash icon (delete) if a custom column title is too long.
* Fix display of aborted tasks in the recent experiments on the Homepage.

61
docs/release_notes/clearml_server/open_source/ver_0_16.md Normal file
View File

@@ -0,0 +1,61 @@
---
title: Version 0.16
---
:::important
**Trains** is now **ClearML**.
:::
### Trains Server 0.16.1
:::important
Upgrading to this version requires a manual [data migration](../../../deploying_clearml/clearml_server_es7_migration.md).
:::
**Bug Fixes**
* Fix model page issue causing N/A to show after switching tabs (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597672446125300).
* Removed experiments comparison limit (only 10 were allowed). Limit is now 100, configurable using `services.tasks.multi_task_histogram_limit`. (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597755746141500).
* Fix scalar plots sometimes not calculated by the server in lower iteration values (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597321653085900).
* Fix error while retrieving experiment log when only a few lines were reported ([GitHub trains-server Issue 59](https://github.com/allegroai/trains-server/issues/59)).
* Update Fixed User full-name on restart (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597906492003300).
* Fix project ordering issue.
* When loading plots, display a spinner and don't show "no data".
* Improve logging to provide more coherent ElasticSearch connection status in server log.
### Trains Server 0.16.0
:::important
Upgrading to this version requires a manual [data migration](../../../deploying_clearml/clearml_server_es7_migration.md).
:::
**Features**
* Add experiment hyperparameter grouping:
* **HYPER PARAMETERS** tab renamed to **CONFIGURATION**.
* **CONFIGURATION** tab contains the sections **USER PROPERTIES**, **HYPER PARAMETERS**, **CONFIGURATION OBJECTS**
* Add user properties group. Key-value pairs always editable (**USER PROPERTIES** section).
* Add command line options group * argparse and older experiments parameters (**CONFIGURATIONS / HYPER PARAMETERS / Args**).
* Add TensorFlow definitions group (**CONFIGURATIONS / HYPER PARAMETERS / TF_DEFINE**).
* Add environment variables group (**CONFIGURATIONS / HYPER PARAMETERS / Environment**).
* Improve experiment model configuration:
* Model design is in the **ARTIFACTS** tab.
* Experiment model description is in the **CONFIGURATION OBJECTS** section in the **CONFIGURATION** tab.
* Improve experiment comparison:
* In hyperparameter parallel coordinate comparison, hover over an experiment name to highlight it on plot ([GitHub Issue #53](https://github.com/allegroai/trains/issues/53)).
* Remove fields providing no additional information from comparison.
* Improve the model framework filter. Filter contains only frameworks used by models in the project.
* Add configurable **Trains** services examples.
* Add support for text debug samples in the **DEBUG SAMPLES** section in the **RESULTS** tab.
* Add legend on / off toggle control for every plot.
* Add clear button for text areas ([GitHub trains-server Issue #42](https://github.com/allegroai/trains-server/issues/42)).
* Reinstate the bottom bar **Archive** button.
* Add **Trains** community links to left bar.
* Add Hi-DPI display support.
* Add `debug.ping` endpoint for simple health monitoring.
* Add support for field exclusion in `*.get_all` endpoints.
* Move to ElasticSearch 7. Requires manual [data migration](../../../deploying_clearml/clearml_server_es7_migration.md).
**Bug Fixes**
* Auto-fit column width on column resize double click.
* Allow top-bar search if fewer than three characters are entered, and `Enter` is pressed.

58
docs/release_notes/clearml_server/open_source/ver_0_17.md Normal file
View File

@@ -0,0 +1,58 @@
---
title: Version 0.17
---
### Open Source ClearML Server 0.17.0
**Features**
* New profile page layout.
* Move knowledge resource to UI top bar.
* Add complete experiment full screen view.
* Add scalar results split / grouped view.
* Add experiment active duration custom column to experiment table.
* Improve comparison experiment consistency in plots for experiment name and color.
* Improve project last update time; updated whenever an experiment or model in the project is updated.
* Improve table multi-select UX: shift+mouse-click ([clearml GitHub Issue 61](https://github.com/allegroai/trains/issues/61)).
* Add experiment parent to experiment details INFO tab.
* Improve UI experiment archiving; archiving queued experiments automatically dequeues them.
* Improve experiment comparison; use available screen width.
* Add debug sample history tracking ([clearml GitHub Issue 81](https://github.com/allegroai/trains/issues/81)).
* Replace "Download" action with "Copy to Clipboard" for local files.
* Add ClearML usage tips.
* Add Task and Model, Clone and Move to allow creating new project on the fly.
* Add support for all S3 / minio special character \(%\) encoding cases.
* Add API for filter by parent.
* Improve browser-search to find data not visible in uncommitted changes / installed packages window.
* Improve Task artifacts optimization.
* Upgrade plotly to 1.57.1.
* Rename Log tab to Console.
**Bug Fixes**
* Fix HiDPI table issues.
* Fix results / plots not scaling properly ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)).
* Fix experiment comparison:
* Cannot compare regular Tasks to example Tasks.
* Cannot unhide scalars.
* Scalars wall time view and relative view will not auto-refresh.
* Fix log scrolling in Chrome.
* Fix UI adds to plots legend labels unnecessary task ID string ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)).
* Fix parallel coordinates: dimensions are not sorted by value.
* Fix scalar auto color selection provides too similar colors.
* Fix reordering a custom column does not restore column location.
* Fix uncommitted changes - not possible to copy the entire text.
* Fix queues wait time graph base should be 0, not in the middle.
* Fix plots view - "No Data" shown for a moment when changing experiments.
* Fix adding new tag during auto-refresh.
* Fix hide empty input model on non draft experiment.
* Fix experiment / model table - name column restores to default size after opening and closing info.
* Fix double click resizer should auto fit column.
### ClearML Hosted Service 0.17.0
* Launched free [ClearML Hosted Service](https://app.clear.ml/dashboard).
* Multiple workspaces.
* Workspace invitation.
* Add publicly sharing experiments.

71
docs/release_notes/clearml_server/open_source/ver_1_0.md Normal file
View File

@@ -0,0 +1,71 @@
---
title: Version 1.0
---
### ClearML Server 1.0.2
**Bug Fixes**
- Fix Task container does not accept `null` values ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1622119047293300) [ClearML GitHub issue 365](https://github.com/allegroai/clearml/issues/365))
- Fix debug images exception in Results page
- Fix a typo in Worker Setup help popup
### ClearML Server 1.0.1
**Bug Fixes**
- Fix clearing experiment requirements causes "empty" requirements (as opposed to "no requirements")
- Fix logout fails with `endpoint not found` error ([ClearML GitHub issue 349](https://github.com/allegroai/clearml/issues/349))
- Fix hamburger side menu `Manage Queues` does nothing and returns console error ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1620308724418100))
- Fix broken config dir backwards compatibility (`/opt/trains/config` should also be supported)
### ClearML Server 1.0.0
**Features**
* Add Multi-level project hierarchy - Projects can now contain sub-projects
* Add markdown editor for project overview
* Add support for multiple models per experiment
* Add Context menu batch operations ([ClearML GitHub issue 286](https://github.com/allegroai/clearml/issues/286))
* Add global search regex support ([ClearML GitHub issue 81](https://github.com/allegroai/clearml/issues/81#issuecomment-735003956))
* Add support for extended container (docker) configuration and init script
* Add special character support ("." (dot) and "$") in Hyperparameters
* Add new context menu icons
* Add deleting experiments and models through UI (ClearML GitHub issues [53](https://github.com/allegroai/clearml-server/issues/53), [61](https://github.com/allegroai/clearml-server/issues/61), and [81](https://github.com/allegroai/clearml/issues/81#issuecomment-706907718))
* Improve Project deletion: deletes project's experiments and models.
* Add experiment description access from info title
* Improve experiment table columns
* Add filter by user
* Add filters time columns
* Add filters for custom columns (metrics and hyperparameters)
* Add secondary nested sorting
* Add `worker<->queue` clickable reference in workers and queues tables
* Add presenting all metric debug samples concurrently
* Add Full project name display in project cards ([ClearML GitHub issue 81](https://github.com/allegroai/clearml/issues/81#issuecomment-823303842))
* Add option for continuing aborted tasks
**Bug Fixes**
* Fix UI storage credentials input missing for HTML artifacts
* Fix sorting custom metric columns sort treats empty fields as zeroes
* Fix UI experiments table redundant rendering on auto refresh
* Fix missing URL encoding for hyperparameters key names
* Fix UI experiment section edit opens outside viewable screen
* Fix UI missing default selection of artifacts in full screen view
* Fix UI variant name not shown in plots
* Fix UI missing tooltip for truncated column headers
* Fix UI custom columns choice does not persist per project ([ClearML GitHub issue 314](https://github.com/allegroai/clearml/issues/314))
* Fix API plot_str not returned for compressed plots
* Fix UI plots color picker consistency
* Fix API ```Tasks.reset``` marking parent id as 'deleted' in its children
* Fix UI missing queue selection on queue delete
* Fix UI debug image history slider not shown when there's only a single iteration
* Fix UI X-axis labels are being cut in plots ([ClearML GitHub issue 264](https://github.com/allegroai/clearml/issues/264))
* Fix UI scalar color choice dialog toggles between screen positions every time you open it
* Fix UI hovering on legend clears parallel coordinates graph filters ([ClearML GitHub issue 259](https://github.com/allegroai/clearml/issues/259))
* Fix UI experiment comparison 'hide identical fields' button disable doesn't work
* Fix UI auto refresh removes model ID in model panel
* Fix UI debug samples download image opens new tab instead of downloading
* Fix UI experiment/model tables: Right most column is not fixed to right edge of screen
* Fix UI 'Add experiment' modal in compare scalar comparison intertwines with comparison display
* Fix UI experiment configurations showing redundant "General" section

47
docs/release_notes/clearml_server/open_source/ver_1_1.md Normal file
View File

@@ -0,0 +1,47 @@
---
title: Version 1.1
---
### ClearML Server 1.1.0
**New Features and Improvements**
- Add metric snapshot plot to project overview UI - Show an aggregated view of all project experiments value for a leading metric
- Add logical `AND` option to UI experiment table tag filter
- Add Task runtime properties to experiment INFO UI tab
- Add full screen view for any experiment result plot
- Add extended version information to UI profile page
- Stop using special characters in secrets
- Allow setting status_message in `tasks.update`
- Improve UI table view configuration persistence - User table-view configuration is saved per project:
- Visible columns
- Column order
- Column width
- Active sort
- Active filters
**Bug Fixes**
- Fix experiment details UI failure opening hyperparameter sections beginning with `#` ([ClearML Server GitHub issue #79](https://github.com/allegroai/clearml-server/issues/79))
- Fix performance issues with UI comparison of large experiments ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1621698235159800))
- Fix filtering on hyperparameters ([ClearML GitHub issue #385](https://github.com/allegroai/clearml/issues/385) [Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1626600582284700))
- Fix profile page user options toggle control area of effect
- Fix browser resizing affecting plot zoom
- Fix UI dataview filter resetting version on filter addition
- Fix UI project overview:
- Fix links in project overview not working
- Allow editing empty overview for legacy projects
- Fix Table plots using fraction of available space
- Fix scalars color assignment broken by `.` in scalar name
- Fix tasks cannot be moved between queues
- Fix UI Docker argument input - Force arguments in designated field
- Fix extraneous "tag" string prefixing Commit ID in Task execution information UI
- Fix missing 'no value' option in hyperparameters table filters
- Fix queued task is not dequeued on `tasks.stop`
- General aesthetic fixes:
- Fix input title alignment in UI clone experiment window
- Fix UI empty experiment table message alignment
- Fix UI table filter menu proportions
- Fix debug sample dropdown menu coloring
- Fix dashboard card alignment
- Fix redundant separator in plot control when maximized

56
docs/release_notes/clearml_server/open_source/ver_1_10.md Normal file
View File

@@ -0,0 +1,56 @@
---
title: Version 1.10
---
### ClearML Server 1.10.1
**Bug Fix**
* Fix WebApp does not connect to the server when using non-default ports ([ClearML Server GitHub issue #191](https://github.com/allegroai/clearml-server/issues/191))
### ClearML Server 1.10.0
**New Features and Improvements**
* Additional model information
* Add Lineage tab to UI model page, which displays experiment that created model and lists experiments that use the model ([ClearML GitHub issue #810](https://github.com/allegroai/clearml/issues/810))
* Add Plots and Scalars tabs to UI Model page
* Add project hierarchy view for ClearML resources (pipelines, datasets, hyper-datasets, reports) ([ClearML GitHub issue #724](https://github.com/allegroai/clearml/issues/724)):
* Flat view - all objects displayed side-by-side
* Nested view - objects grouped by containing projects.
* Add archived experiment inclusion in UI experiment comparison ([ClearML GitHub issue #795](https://github.com/allegroai/clearml/issues/795))
* Update ElasticSearch image to 7.17.7 ([ClearML Server GitHub issue #183](https://github.com/allegroai/clearml-server/issues/183))
* Add reported plot support for multiple color formats
* Add running tasks to UI project overview metric snapshot
* Improve UI Reports:
* Add option to upload and embed images from PC
* Add option to embed experiment comparison parallel coordinates plots and full screen debug sample
* Add UI indicator to hidden resources when "Show hidden projects" is enabled
* Add Experiment comparison UI parallel coordinate plots full screen view
**Bug Fixes**
* Fix UI scalar plot smoothing is biased towards initial value ([ClearML Server GitHub issue #180](https://github.com/allegroai/clearml-server/issues/180))
* Fix UI experiment plot comparison not merging plots
* Fix UI reports search bar doesn't search report descriptions
* Fix UI reports search bar does not close after result is selected
* Fix UI user preferences don't persist in duplicate tab or after logout/login
* Fix UI pipeline step names are cropped with no available tooltips
* Fix refreshing UI console output scrolls away from latest entries
* Fix UI experiment artifact list not updated upon artifact deletion
* Fix UI experiment comparison Hyperparameters tab scroll erratic behavior in Firefox
* Fix UI console log partially showing pipeline log entries
* Fix can't move UI Report to root project
* Fix queue isn't selected when name is clicked in the UI enqueue modal
* Fix turning off filter in UI experiment table does not persist
* Fix UI plot lines appearing in wrong places in plot
* Fix newly created dataset tags missing from dataset filter tag list
* Fix unclear text in UI plots downloaded as PDF/PNG
* Fix UI global search incorrectly returns 0 Dataset/Reports count
* Fix long named reports flash upon hover
* Fix disappearing UI scalar graph when "Group by" is set to "None"
* Fix UI experiment plots horizontal axis units change to "Iterations" when refreshed
* Fix UI scalar graphs do not update when horizontal axis units are changed
* Fix tooltip does not appear when hovering over name in UI breadcrumbs
* Fix UI experiment scalar smoothing does not persist in plot fullscreen
* Fix newly created report tags missing from report filter tag list
* Fix reports names can contain a "/"
* Fix My work / Team work filter selection does not persist upon app refresh

40
docs/release_notes/ver_1_11.md → docs/release_notes/clearml_server/open_source/ver_1_11.md
View File

@@ -2,48 +2,8 @@
title: Version 1.11
---
### ClearML SDK 1.11.1
**New Features and Improvements**
* Support `lightning>=2.0` ([ClearML GitHub issue #1032](https://github.com/allegroai/clearml/issues/1032))
* Add `clearml-init` support for standard HTTP and HTTPS ports in webapp in conjunction with non-default api/files server ports
([ClearML Server GitHub issue #198](https://github.com/allegroai/clearml-server/issues/198))
* Improve docstrings
**Bug Fixes**
* Fix remote execution does not log configuration values from modified `jsonargparse` configuration files
* Fix `ValueError` when setting task properties at the start of a pipeline component
* Fix OS exceptions on passing `Path` where `str` expected
* Fix infinite recursion when remotely updating a `dict` added to task with `Task.connect()`
* Fix `Task.connect()` race condition overwriting task description with connected input model
### ClearML SDK 1.11.0
**New Features and Improvements**
* Add offline mode for ClearML Datasets ([ClearML GitHub issue #854](https://github.com/allegroai/clearml/issues/854))
* Improve performance of `Dataset.add_external_files()` ([ClearML GitHub PR #962](https://github.com/allegroai/clearml/pull/962))
* Add `Task.launch_multi_node()` for distributed experiment execution
* Add `Task.get_all_reported_scalars()` to fetch all available scalar data
* Add `Task.remove_input_models()` to disassociate input models from a task
* Improve `Dataset.list_datasets()` to include dataset version in the result
* Add support for PyTorch Lightning CLI >=2.0
* Add support for automatic Dataset content preview for JSON files
**Bug Fixes**
* Fix multiple local parallel experiments conflicting on storage access validation ([ClearML GitHub issue #897](https://github.com/allegroai/clearml/issues/897))
* Fix code running with `joblib` hangs ([ClearML GitHub issue #1009](https://github.com/allegroai/clearml/issues/1009))
* Fix comment for storage scheme `gs://` ([ClearML GitHub issue #986](https://github.com/allegroai/clearml/issues/986))
* Fix typo "Categories per column" ([ClearML GitHub PR #1021](https://github.com/allegroai/clearml/pull/1021))
* Fix downloading artifacts from GCP when the download URI includes unsafe characters
* Fix pipeline step cache is not cleared when the only change to the step is that the docker image changed
* Fix `Task.report_text()` sometimes reporting to an incorrect task when multiple tasks run simultaneously
* Fix` Task.set_offline(offline_mode=False)` raising an `UnboundLocalError`
* Fix `scikit-image` package (`skimage`) is sometimes not detected as a dependency
* Fix `PipelineDecorator` sometimes causing a race condition when starting a remote execution
### ClearML Server 1.11.0
**New Features and Improvements**
* New model comparison UI: compare model configuration, metrics, and plots
* Add Preview tab to UI pipeline for viewing debug samples and plots attached to a pipeline

46
docs/release_notes/ver_1_12.md → docs/release_notes/clearml_server/open_source/ver_1_12.md
View File

@@ -2,18 +2,6 @@
title: Version 1.12
---
### ClearML SDK 1.12.2
**New Features and Improvements**
* Add `always_create_from_code` argument to `PipelineController` (default `True`) to allow remote execution to create
the Pipeline DAG at runtime (as opposed to adhering to the existing structure stored on the task when running locally)
**Bug Fixes**
* Fix using `OutputModel` reporting and other methods fail if `OutputModel.update_weights()` is not called before that ([ClearML GitHub issue #1078](https://github.com/allegroai/clearml/issues/1078))
* Fix hash for Pandas objects artifacts
* Fix Hydra/OmegaConf `--overrides` support
* Fix Pipeline metrics not forwarded to pipeline controller at the end of a pipeline step
### ClearML Server 1.12.1
**Bug Fixes**
@@ -21,15 +9,6 @@ the Pipeline DAG at runtime (as opposed to adhering to the existing structure st
* Fix pipeline parameters not displayed in UI pipeline "New Run" modal
* Fix UI pipeline project cards display incorrect pipeline run counts
### ClearML SDK 1.12.1
**New Features and Improvements**
* Raise error if dataset name is empty ([ClearML GitHub PR #1083](https://github.com/allegroai/clearml/pull/1083))
* Add support for `spawn` start method for Python `multiprocess`. This should help circumvent issues like [this](https://github.com/python/cpython/issues/84559).
**Bug Fixes**
* Fix S3 uploads fail with `LazyEvalWrapper` type error ([ClearML GitHub issue #1081](https://github.com/allegroai/clearml/issues/1081))
### ClearML Server 1.12.0
**New Features and Improvements**
@@ -97,28 +76,3 @@ the Pipeline DAG at runtime (as opposed to adhering to the existing structure st
* Fix UI generated resource embed codes do not maintain plot's x-axis units
* Fix long model names obscure model page menu
* Fix some model details missing in UI experiment artifacts
### ClearML SDK 1.12.0
:::info BREAKING CHANGES
As a result of fixing [issue #1051](https://github.com/allegroai/clearml/issues/1051) and similar issues with retrieving
task data/models/artifacts/samples stored in GCS, we introduced some breaking changes. To access resources registered to
GCS from ClearML SDK versions 1.11.0 to 1.11.1, see
the instructions [here](https://github.com/allegroai/clearml/tree/master/docs/errata_breaking_change_gcs_sdk_1_11_x.md).
:::
**New Features and Improvements**
* Add `include_archive` parameter to `Dataset.list_datasets()`: include archived datasets in list ([ClearML GitHub issue #1067](https://github.com/allegroai/clearml/issues/1067))
* Add support to specify the multipart chunk size and threshold using the `aws.boto3.multipart_chunksize` and
`aws.boto3.multipart_threshold` configuration options in the clearml.conf ([ClearML GitHub issue #1058](https://github.com/allegroai/clearml/issues/1058))
* Add `PipelineController.get_pipeline()` for retrieving previously run pipelines.
**Bug Fixes**
* Fix AWS driver issues ([ClearML GitHub PR #1000](https://github.com/allegroai/clearml/pull/1000)):
* Fix credential authentication failure when attempting to use token
* Fix instantiation within VPC without AvailabilityZones
* Fix `continue_last_task=0` is ignored in pipelines run with `retry_on_failure` ([ClearML GitHub issue #1054](https://github.com/allegroai/clearml/issues/1054))
* Fix `Task.connect_configuration()` doesn't handle dictionaries with special characters
* Fix pipeline steps created with `PipelineDecorator` aren't cached
* Fix `Task.get_by_name()` doesn't return the most recent task when multiple tasks have same name
* Fix Boto3 certificate verification ignores custom self-signed certificates

58
docs/release_notes/clearml_server/open_source/ver_1_13.md Normal file
View File

@@ -0,0 +1,58 @@
---
title: Version 1.13
---
### ClearML Server 1.13.1
**Bug Fixes**
* Fix GCS credentials keep popping up
* Fix long name overflows onto show/hide button in scalar metric list
### ClearML Server 1.13.0
**New Features and Improvements**
* Improve UI comparison
* Add visual comparison of single value scalars ([ClearML Server GitHub issue #202](https://github.com/allegroai/clearml-server/issues/202))
* Scalar value comparison: metrics are presented in a single table ([ClearML GitHub issue #760](https://github.com/allegroai/clearml/issues/760))
* Add global experiment legend: Experiment name, tags and color ([ClearML GitHub issue #81](https://github.com/allegroai/clearml/issues/81))
* Clickable URLs/object IDs in experiment details comparison ([ClearML GitHub issue #1073](https://github.com/allegroai/clearml/issues/1073))
* UI Search improvements
* Dashboard search results ordered by update time
* Top bar search query saved in URL
* Improve UI experiment debug sample iteration fetching speed ([ClearML Server GitHub issue #170](https://github.com/allegroai/clearml-server/issues/170))
* Add UI cloud credentials support for GCS signed URLs with HMAC authentication
* Add Experiment plot UI support for displaying subplots
**Bug Fixes**
* Fix UI single-value scalars table does not display all single-value scalars ([ClearML GitHub issue #1111](https://github.com/allegroai/clearml/issues/1111))
* Fix UI pipeline step duration does not update during runtime ([ClearML GitHub issue #1090](https://github.com/allegroai/clearml/issues/1090))
* Fix UI experiment comparison parallel coordinates plot crops the metric and parameter names ([ClearML GitHub issue #1111](https://github.com/allegroai/clearml/issues/1111))
* Fix hidden experiments appear in UI All Experiments search results ([ClearML GitHub issue #1139](https://github.com/allegroai/clearml/issues/1139))
* Fix disabled graphs in UI scalars reappear in full-screen mode
* Fix UI auto-refresh sometimes causes many error message to be raised
* Fix "All" tag filter not working in UI object pages
* Fix tag filter in UI dataset page does not persist on refresh
* Fix scalars and plots are not displayed in UI model page
* Fix UI pipeline step status does not update
* Fix UI experiment comparison plots don't display all experiments in legend
* Fix UI Model Lineage tab sometimes doesn't display models
* Fix some plots present partial data in UI experiment comparison plots
* Fix clicking URL of deleted UI report navigates to empty page
* Fix applying UI tag in bulk raises an error
* Fix task IDs appear multiple times in UI task comparison plot legends
* Fix UI project filter in All Experiments table takes a long time load
* Fix custom columns in UI tables do not persist on refresh
* Fix UI report card missing creating user's name
* Fix new tag is not displayed in "Add Tag" table action menu until page refresh
* Fix clicking on worker name in Orchestration page's Queue info panel does not navigate to the worker's info panel
* Fix clicking on older empty UI datasets redirects to 404 page
* Fix experiment names don't appear in the plot legend in UI experiment comparison scalar plots grouped by metric
* Fix task ID does not appear in UI task comparison plots of tasks with the same name
* Fix UI scalar comparison's Horizontal Axis, Smoothing, and Group by selections do not persist upon experiment selection modification
* Fix UI scalars sometimes missing plots in Chrome
* Fix values can't be input into the UI scalar smoothing field
* Fix UI experiment configuration is sometimes not editable
* Fix UI project name tooltips don't appear for projects with long names
* Fix debug samples appear in UI experiment only after refresh
* Fix cropped names in tag list in "Add Tag" action in UI table action menu
* Fix UI experiment configuration tooltip obscure the "Edit" button

44
docs/release_notes/clearml_server/open_source/ver_1_14.md Normal file
View File

@@ -0,0 +1,44 @@
---
title: Version 1.14
---
### ClearML Server 1.14.1
**New Features and Improvements**
* Add support for serving UI from a subpath
* Change default cookie `SameSite` setting to `Lax`
**Bug Fixes**
* Fix clicking on UI Metric Snapshot datapoint does not navigate to experiment ([ClearML Server GitHub issue #228](https://github.com/allegroai/clearml-server/issues/228))
* Fix some APIs not sanitizing query parameters
* Fix downloading UI experiment console log sometimes returns an older file
* Fix UI experiment plot comparison displays duplicates of image plots
* Fix missing experiment labels in UI experiment plot comparison
* Fix starting time not updating for some tasks
### ClearML Server 1.14.0
**New Features and Improvements**
* New UI experiment table comparative view: compare plots and scalars of all selected experiments
* Add UI experiment display of original python requirements along with actual packages used ([ClearML GitHub issue #793](https://github.com/allegroai/clearml/issues/793))
* Add UI project metric snapshot support for multiple metrics
* Add compressed UI experiment table info panel mode displaying only experiment name and status
* Add "x unified" hover mode to UI plots
* Add experiment hyperparameter UI section dataset IDs link to datasets page
* Make all URLs clickable in UI experiment artifacts
* Redesign WebApp tabs: tabs now appear in UI header
* Add option to force setting original task as clone's parent when cloning a task
**Bug Fixes**
* Fix CVE-2023-45133 Web App vulnerability
* Fix "select all" in UI dataset archive displays count of all archived and unarchived experiments
* Fix hyperparameter keys with dots in their names displayed incorrectly in UI experiment table and task info
* Fix UI experiment scalar comparison raising error
* Fix "Project" column redundantly appears inside a specific project's model table
* Fix UI models with many labels slowing down web app
* Fix simple UI plots (scatter, bar, etc.) in experiment comparisons not merging by experiment
* Fix UI pipeline details panel does not display information
* Fix UI plot "Show/hide" legend button sometimes disappears
* Fix can't select root project in UI report creation modal when workspace has no projects
* Fix "load more" button displayed unnecessarily in UI object tables
* Fix archive action in UI dataset table archives unselected datasets

32
docs/release_notes/ver_1_15.md → docs/release_notes/clearml_server/open_source/ver_1_15.md
View File

@@ -7,38 +7,6 @@ title: Version 1.15
**Bug Fix**
* Fix restarting ClearML Server deletes app credentials ([ClearML Server GitHub issue #240](https://github.com/allegroai/clearml-server/issues/240) and ClearML GitHub issues [#1247](https://github.com/allegroai/clearml/issues/1257) & [#1262](https://github.com/allegroai/clearml/issues/1262))
### ClearML SDK 1.15.1
**Bug Fixes**
* Fix autoscaler should recheck that the worker is still idle before shutting it down ([ClearML GitHub issue #1202](https://github.com/allegroai/clearml/issues/1202))
* Fix resource monitor fails to get GPU stats in some edge cases
### ClearML SDK 1.15.0
**New Features**
* Add draft option to pipeline steps ([ClearML GitHub PR #1226](https://github.com/allegroai/clearml/pull/1226))
* Add support for custom working directory for pipelines ([ClearML GitHub issue #1194](https://github.com/allegroai/clearml/issues/1194))
* Add `Task.get_requirements()` method that returns the task's requirements
* Allow controlling the number of threads used by `StorageManager.download_folder()` using the `max_workers` argument
* Update examples dependencies
* Improve auto populate in `Task.init()`
**Bug Fixes**
* Fix UTF-8 script code encoding issue ([ClearML GitHub issue #1207](https://github.com/allegroai/clearml/issues/1207))
* Fix Colab docs ([ClearML GitHub issue #1204](https://github.com/allegroai/clearml/issues/1204))
* Fix metrics reporting with `OutputModel` while in offline mode results in an error ([ClearML GitHub issue #1172](https://github.com/allegroai/clearml/issues/1172))
* Fix task running in Google Colab doesn't properly get the notebook diff ([ClearML GitHub issue #1204](https://github.com/allegroai/clearml/issues/1204))
* Fix hydra binds break in offline mode ([ClearML GitHub issue #1215](https://github.com/allegroai/clearml/issues/1215))
* Fix HPO crashes when optimizing for single value scalars ([ClearML GitHub issue #1221](https://github.com/allegroai/clearml/issues/1221))
* Fix GPU info such as `gpu_memory` and `gpu_type` is not being collected in some cases
* Fix `clearml-data` CLI tool moves non-dataset tasks to a `.dataset` project when the respective task is not a dataset
* Fix pandas `DataFrame` artifacts with hierarchical indices get mangled by the CSV round-trip
* Fix `urllib3` sends deprecation warning when setting `ssl_version`
* Fix `Task.connect(dict)` return value is not dict-compatible
* Fix `jsonargparse` sub-command config parsing
* Fix Lightning integration crashes when a config entry contains `.` in its name
* Fix Python 3.5 compatibility
### ClearML Server 1.15.0
**New Features and Improvements**

37
docs/release_notes/clearml_server/open_source/ver_1_16.md Normal file
View File

@@ -0,0 +1,37 @@
---
title: Version 1.16
---
### ClearML Server 1.16.2
**Bug Fix**
* Fix no graphs are shown in Workers and Queues screens
### ClearML Server 1.16.1
**Bug Fix**
* Fix user credentials reset on API server restart ([ClearML Server GitHub issue #250](https://github.com/allegroai/clearml-server/issues/250))
### ClearML Server 1.16.0
**New Features and Improvements**
* New UI "new experiment creation"
* ClearML File Server now uses authentication by default. This can be turned off using the `fileserver.auth.enabled: false`
configuration setting
* Add confirmation pop-up when running experiment is archived
**Bug Fixes**
* Fix model plots not displayed in UI model plot tab
* Fix UI experiment Debug Samples tab freezes when a lot of debug samples are reported in the experiment
* Fix UI scalar “Export to CSV” button returns an empty CSV file
* Fix large number of tags in UI tables' tag filter causes UI to crash
* Fix searching in UI object tables' tag filter causes UI to crash
* Fix consistent plot colors not maintained when embedded in UI Reports ([ClearML GitHub issue #1239](https://github.com/allegroai/clearml/issues/1239))
* Fix UI plot legends and tooltips wrapping ([ClearML Server GitHub issue #241](https://github.com/allegroai/clearml-server/issues/241))
* Fix multiple parameters can be selected for plot axes in scatter plot view in UI experiment comparison
* Fix image names change in UI task debug sample comparison when a different metric is selected
* Fix numeric range filters of UI table columns not working
* Fix queue experiment list is not updated in the UI Orchestration's Queues page
* Fix My work/Team work persists across UI modules (i.e. projects, datasets, pipelines, etc.)
* Fix UI single-scalar plots are not ordered alphabetically by scalar name
* Fix limit on file uploads to the fileserver

63
docs/release_notes/clearml_server/open_source/ver_1_2.md Normal file
View File

@@ -0,0 +1,63 @@
---
title: Version 1.2
---
### ClearML Server 1.2.0
**New Features and Improvements**
* Add functionality to "Abort all children" UI action for controller and optimizer tasks
* Add parameter search to task configuration UI section ([ClearML GitHub issue #467](https://github.com/allegroai/clearml/issues/467))
* Add tag exclusion filters in UI experiment and model tables
* Add "Clear Filters" functionality button to UI table filters
* Improve full screen scalar graph to display all data points
* Add UI experiment table option for mass selection of out-of-screen items
* Improve experiment comparison UI:
* New experiment addition modal with table filtering and sorting
* Fix scalar selection being reset upon comparison contents change
* Add Task ID suffix to debug samples with same-name experiments
* Add task ID suffix to experiment name for same-experiment-name series in scalar experiment comparison
* New Settings pages
* User profile
* Webapp configuration
* Workspace settings
* Add "Updated" column to models table
* Add refresh button to full screen scalar display
* Improve display of float values in UI experiment and model tables. Tables will display rounded up float values
according to space allotted to a column. View precise value by hovering over value.
* Add capability to edit username in UI profile page
* Add build number to version information in UI Settings page
**Bug Fixes**
* Fix UI debug sample viewer ignoring metric filter on auto-refresh ([ClearML GitHub issue #512](https://github.com/allegroai/clearml/issues/512))
* Fix "Wall time" x-axis option in UI plots slowing down web app ([ClearML GitHub issue #441](https://github.com/allegroai/clearml/issues/441))
* Fix UI table plot titles don't display their iteration number ([ClearML GitHub issue #474](https://github.com/allegroai/clearml/issues/474))
* Fix auto-refresh resets vertical scroll in UI debug samples ([ClearML GitHub issue #474](https://github.com/allegroai/clearml/issues/474))
* Fix manually run experiments cannot be edited in UI after being reset ([ClearML GitHub issue #449](https://github.com/allegroai/clearml/issues/449))
* Fix UI Workers & Queues and Experiment Table pages display mismatching experiment runtime values ([ClearML Server GitHub issue #100](https://github.com/allegroai/clearml-server/issues/100))
* Fix UI plots sometimes disappear after task execution completion
* Fix UI experiment plots don't display some plots on a http server
* Fix failure to add Azure credentials through UI Settings page by new users
* Fix UI experiment table failing to clear selection of off-screen experiments
* Fix UI experiment table column resizing malfunctioning
* Fix UI table filters being reset after page is refreshed
* Fix failure to create a sub-project on-the-fly when cloning an experiment through the UI
* Fix UI project overview editing enabled when project is public
* Fix model deletion from UI "All Experiments" page navigates to model project's experiment table
* Fix missing "Project" custom column in "All Experiments" views of non-root projects
* Fix UI project cards' task enumeration summaries displaying negative number
* Fix experiment bar graph colors cannot be changed sometimes
* Fix fail to filter experiments on tags beginning with "-"
* Fix tag list order in UI experiment table filter after tag selection
* Fix long hyperparameter names are obscured in UI comparison screen
* Fix add tag context menu operation fails when experiment selection includes a system example
* Fix size of low value points in UI metric snapshot plot
* Fix long queue names obscures copy button in UI queue list and Worker & Queue page plot titles
* Fix UI breadcrumb menus missing values
* Fix UI scalar plot titles are unreadable
* Fix UI buttons not working after closing "Add Experiment" modal in UI experiment comparison page
* Fix underscores disappear while editing UI text blocks on Chromium
* Fix UI Workers & Queues page's queue panel does not display experiment information
* Fix long queue names obscure copy button in UI queue list
* Fix UI debug image viewer momentarily displays previously viewed image
* Fix "Load More" button in UI experiment table presents duplicates of already displayed experiments

40
docs/release_notes/clearml_server/open_source/ver_1_3.md Normal file
View File

@@ -0,0 +1,40 @@
---
title: Version 1.3
---
### ClearML Server 1.3.1
**Bug Fixes**
* Fix scalar graph smoothing ([ClearML Server GitHub issue #127](https://github.com/allegroai/clearml-server/issues/127))
### ClearML Server 1.3.0
**New Features and Improvements**
* New UI pipelines page:
* Pipelines statistics dashboard
* Pipeline run history: View pipeline structure, run configuration and outputs
* Pipeline control: Abort/Run pipelines from UI
* Add support for exporting UI table plots in CSV formats ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400))
* Improve UI breadcrumbs use of screen real-estate ([ClearML GitHub issue #529](https://github.com/allegroai/clearml/issues/529))
* Add UI adherence to user specification of plot colors and labels ([ClearML GitHub issue #518](https://github.com/allegroai/clearml/issues/518))
* Add Model metadata to UI model information (Model table and details page)
* Add multi-selection in UI experiment info-panel mode
* Add labels to workspace credentials
* Add UI Queues "Clear" action
* Add "Show/Hide" controls to UI experiments scalars tab.
* Add Previous/Next controls to UI text area search bars
**Bug Fixes**
* Fix UI plots do not display Plotly `imshow texttemplate` ([ClearML GitHub issue #538](https://github.com/allegroai/clearml/issues/538))
* Fix clicking F5 clears storage credentials from UI settings page
* Fix UI plots "Wall time" horizontal axis option is not working
* Fix disappearing UI full-screen scalar plot
* Fix disappearing object names in table info-panel mode
* Fix clicking "group by" menu in UI experiment scalars page opens the incorrect menu
* Fix disappearing scalar plots in UI experiment comparison page
* Fix hidden custom UI table column is not removed from main column list
* Fix UI plot toolbar missing tools
* Fix UI table plot tools only appear when hovering over the end of the plot
* Fix resizing UI table column causes subsequent columns to thin
* Fix "Tip of the day" window continues appearing after user-requested removal

33
docs/release_notes/clearml_server/open_source/ver_1_4.md Normal file
View File

@@ -0,0 +1,33 @@
---
title: Version 1.4
---
### ClearML Server 1.4.0
**New Features and Improvements**
- Allow serving with a URL path prefix ([ClearML Server GitHub issue #119](https://github.com/allegroai/clearml-server/issues/119))
- Allow overriding clearml web git url on build ([ClearML Server GitHub Pull Request #122](https://github.com/allegroai/clearml-server/pull/122))
- Ensure agent-services waits for API server to be ready ([ClearML Server GitHub issue #113](https://github.com/allegroai/clearml-server/issues/113))
- Add filter to UI experiment table "Project" column ([ClearML GitHub issue #529](https://github.com/allegroai/clearml/issues/529))
- Add filters to UI model table custom metadata columns
- Improve UI table object selection. Add buttons and context menu actions to switch between table and info panel mode
- Add task detail header to UI experiment comparison debug image section addressed in ([ClearML GitHub issue #81](https://github.com/allegroai/clearml/issues/81) ([comment](https://github.com/allegroai/clearml/issues/81#issuecomment-996841658)))
- Improve series name truncation behavior in UI scalar plots ([ClearML GitHub issue #562](https://github.com/allegroai/clearml/issues/562))
**Bug Fixes**
- Fix server limiting image and artifact upload size ([ClearML GitHub issue #606](https://github.com/allegroai/clearml/issues/606))
- Fix server unnecessarily displays deletion error message ([ClearML Server GitHub issue #112](https://github.com/allegroai/clearml-server/issues/112))
- Fix UI experiment comparison sections overly wide for many tag experiments ([ClearML GitHub issue #594](https://github.com/allegroai/clearml/issues/594))
- Fix model name display in UI pipeline run info panel
- Fix UI pipeline run info panel missing artifacts and models sections
- Fix UI pipeline run info panel displays unneeded resource utilization metrics
- Fix UI project cards missing task summary values
- Fix pipeline tag colors can't be changed in UI pipeline page
- Fix pipeline run UI page not displaying pipeline steps ([ClearML GitHub issue #618](https://github.com/allegroai/clearml/issues/618))
- Fix breadcrumb links don't navigate anywhere
- Fix horizontal scroll in UI experiment table causes column headers to jump
- Fix removing UI table column creates a blank column
- Fix internal server error (500) on `events.get_scalar_metrics_and_variants` ([ClearML Server GitHub issue #120](https://github.com/allegroai/clearml-server/issues/120))

36
docs/release_notes/clearml_server/open_source/ver_1_5.md Normal file
View File

@@ -0,0 +1,36 @@
---
title: Version 1.5
---
### ClearML Server 1.5.0
**New Features and Improvements**
* Redesign UI tables info panel:
* Redesign experiment status banner
* Display all experiment tabs in single line
* Move experiment action buttons to the top right
* Add UI adherence to user specified plot dimensions ([ClearML GitHub issue #587](https://github.com/allegroai/clearml/issues/587))
* Add row highlight to experiment textual comparison UI ([ClearML GitHub issue #581](https://github.com/allegroai/clearml/issues/581))
* Add UI "Delete" action for Pipelines
* Add UI indicator for task and pipeline progress
* Add UI API credentials label "Edit" action
* Add option to create queues on-the-fly when enqueuing tasks in UI
* Add "Number of Workers" column to UI queues table
* Add pipeline results to UI dashboard search
**Bug Fixes**
* Fix UI experiment debug samples disappearing after refresh ([ClearML Server GitHub issue #136](https://github.com/allegroai/clearml-server/issues/136))
* Fix deleting tasks sometimes raises errors ([ClearML GitHub issue #632](https://github.com/allegroai/clearml/issues/632))
* Fix only partial task log shown when running on ES with multiple shards
* Fix move task to trash is not thread-safe
* Fix UI Project overview metric snapshot not showing
* Fix no progress indicator when performing off-screen selection in UI experiments table
* Fix removing long pipeline tags doesn't work
* Fix UI experiment scalar comparison graph titles are cropped with no available tooltips
* Fix UI model table status column filters incorrectly when multiple statuses are selected.
* Fix unreadable plot data causes UI to crash
* Fix UI project cards displaying incorrect task counts
* Fix hover over system tags doesn't display full tag name
* Fix `.txt` file debug sample previews are unreadable
* Fix UI table/info-panel toggle doesn't update when clicking project breadcrumbs
* Fix UI experiment debug samples leave artifacts when empty

66
docs/release_notes/clearml_server/open_source/ver_1_6.md Normal file
View File

@@ -0,0 +1,66 @@
---
title: Version 1.6
---
### ClearML Server 1.6.0
**New Features and Improvements**
* New ClearML Datasets UI pages for tracking dataset versions and exploring version lineage and contents
* Add history navigation to experiments plots UI page ([ClearML GitHub issues #81](https://github.com/allegroai/clearml/issues/81) and [#255](https://github.com/allegroai/clearml/issues/255)):
* Plots page shows last reported plot for each metric/variation combination
* Single plot view provides history navigation slider
* Add single value scalar reporting: Single value scalars are aggregated into a summary table in the experiment's scalars
UI page ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400))
* Add "show hidden projects" control ([ClearML GitHub issue #694](https://github.com/allegroai/clearml/issues/694))
* Improve UI projects page setting persistence - User's last chosen settings for recent/name sorting and team/personal
filter is saved ([ClearML GitHub issue #353](https://github.com/allegroai/clearml/issues/353))
* Improve UI presentation for object IDs:
* Display shortened ID next to task name
* Display full ID in tooltip
* Copy-to-clipboard on click
* Add "Clear all active filters" button to Projects and Pipelines details view
* Add notification for UI session expiration
* Add ID columns to UI object tables (experiments, models, etc.)
* Add "Info" section to "Details" tab of UI experiment comparison
* Add "loading" indicator for HTML debug samples
* Improve UI text editor behavior: when applicable, cursor returns to most recent edit, otherwise, cursor goes to start of text
* Maintain UI viewing mode when restoring archived items
**Bug Fixes**
* Fix experiment selection with 'Shift' key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/allegroai/clearml-server/issues/139))
* Fix UI search error message interferes with inputting search query ([ClearML Server GitHub issue #138](https://github.com/allegroai/clearml-server/issues/138))
* Fix refresh breaks UI breadcrumbs ClearML Server ([ClearML Server GitHub issue #142](https://github.com/allegroai/clearml-server/issues/142))
* Fix UI Workers and Queues page displays incorrect queue metrics
* Fix failure to publish models whose generating task was reset
* Fix listed models in UI pipeline run info panel doesn't link to model
* Fix "Load more" button disappears from UI experiment page
* Fix breadcrumb link to parent project does not navigate to the parent's project page
* Fix spaces deleted while typing query in UI search bars
* Fix UI plots not loading in experiments
* Fix UI experiment debug sample full screen failing to display multiple metrics
* Fix using search in UI tables removes custom columns
* Fix experiment debug samples remain in file server after experiment deletion
* Fix UI pipeline page sorting not working
* Fix UI scalar comparison does not display task tags
* Fix smoothed UI plots obscured by original graphs
* Fix broken task name links in UI scalar and hyperparameter comparison pages
* Fix "My work" filter not filtering out sub-projects
* Fix generic project link navigates to a blank page
* Fix UI experiment's model link inefficient use of screen real estate
* Fix network error causes UI models and experiments auto-refresh to stop working
* Fix Empty path menu appearing in UI breadcrumbs when there are no intermediary projects
* Fix long loading time in UI experiment comparison
* Fix slow performance of UI experiment plots and scalars in Chrome
* Fix Delete action not appearing in bottom bar of UI pipeline runs table archive
* Fix UI experiments not displaying dataset tasks created with `clearml` version lower than 1.6
* Fix UI projects with dataset tasks created with `clearml` version lower than 1.6 display incorrect statistics
* Fix maximizing image-plot chart in UI causes CORS error
* Fix UI experiment's "Select A Public Model" modal missing "Updated" and "Description" column data
* Fix UI experiments table's project filter displays duplicates
* Fix UI projects display incorrect statistics
* Fix clicking project with only hidden sub-projects doesn't navigate to the project's UI page
* Fix passing empty string parameter name to `Task.set_parameters` causes a 500 error
* Fix UI model page General tab's layout
* Fix using UI experiment plot navigation menu pushes WebApp's top navigation bar
* Fix example pipeline project created with "Generate Example" is empty
* Fix example pipeline created with "Generate Example" is not displayed in project dashboard when "Show hidden projects" is enabled
* Fix UI table/info-panel toggle doesn't update when switching viewing modes

33
docs/release_notes/clearml_server/open_source/ver_1_7.md Normal file
View File

@@ -0,0 +1,33 @@
---
title: Version 1.7
---
### ClearML Server 1.7.0
**New Features and Improvements**
* Add “Sync comparison” to UI experiment debug samples comparison: Control metric/iteration for all compared experiments ([ClearML GitHub issue #691](https://github.com/allegroai/clearml/issues/691))
* Support serving UI from a non-root path of the ClearML Server ([ClearML Helm Charts issue #101](https://github.com/allegroai/clearml-helm-charts/issues/101) and [ClearML Server issue #135](https://github.com/allegroai/clearml-server/issues/135))
* Add UI option for hiding “secret” experiment container arguments ([ClearML Server GitHub issue #146](https://github.com/allegroai/clearml-server/issues/146))
* Add UI tables switch to detail mode through double-click ([ClearML Server GitHub issue #134](https://github.com/allegroai/clearml-server/issues/134))
* Add customizable user activity timeout for UI logout
* Add UI navigation from experiment comparison back to originating experiment table
* Improve UI scalar comparison graph configuration - Persist user's choice for viewing graph data.
* Add model IDs display in UI experiment artifact tab
* Add dataset description to UI dataset information display
**Bug Fixes**
* Fix UI experiment Hyperparameter tab's inefficient use of screen real estate ([ClearML GitHub issue #705](https://github.com/allegroai/clearml/issues/705))
* Fix navigating to an archived experiment's configuration causes UI to return to non-archived view ([ClearML Server GitHub issue #148](https://github.com/allegroai/clearml-server/issues/148))
* Fix metric storage failure when large amount of metrics logged
* Fix UI plots downloaded as PNGs don't contain legends
* Fix UI plot colors can't be changed when colors are specified in code
* Fix experiment table hyperparameter column sorting fails when hyperparameter includes "."
* Fix artifacts aren't deleted when experiment is reset
* Fix UI project cards displaying incorrect task counts when "Show Hidden Projects" is enabled
* Fix changing UI filter to "My Work" within a project page causes main project to become undefined
* Fix clicking project with only hidden sub-projects doesn't navigate to the project's UI page
* Fix multiple UI projects can have same name
* Fix UI experiment plot legends disappear when plots take up whole row
* Fix cloning an experiment into a different project does not navigate to the new project
* Fix queue menu automatically opens when UI enqueue modal is opened
* Fix can't dequeue experiments from deleted queue.
* Fix UI Dataset content previews displays "No preview" message while previews are loading

28
docs/release_notes/clearml_server/open_source/ver_1_8.md Normal file
View File

@@ -0,0 +1,28 @@
---
title: Version 1.8
---
### ClearML Server 1.8.0
**New Features and Improvements**
* Support `/` delimited project names for on-the-fly nested project creation in UI operations ([ClearML Server GitHub issue #151](https://github.com/allegroai/clearml-server/issues/151))
* Add keyboard shortcut iteration navigation in full screen view of debug samples and plots
* Add "Don't show again" option for UI notification when archiving tasks ([ClearML Server GitHub issue #149](https://github.com/allegroai/clearml-server/issues/149))
* Add task reset/delete modal automatically closes when no additional information is provided
* Add parent project name in UI dataset and pipeline cards
* Move UI plot legends to bottom of plot
**Bug Fixes**
* Fix task artifacts not deleted from file server when task deleted via UI ([ClearML GitHub issue #801](https://github.com/allegroai/clearml/issues/801))
* Fix deleting pipeline projects via UI fails ([ClearML GitHub issue #798](https://github.com/allegroai/clearml/issues/798))
* Fix multiple selection with `Shift` key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/allegroai/clearml-server/issues/139))
* Fix disappearing user filter in UI "All Experiments" table ([ClearML Web GitHub issue #33](https://github.com/allegroai/clearml-web/issues/33))
* Fix UI Dashboard missing My Work / Team Work filter
* Fix navigating to Scalars and Plots tabs in UI experiment comparison freezes browser
* Fix project stats do not exclude dataset and annotation tasks
* Fix subproject with no experiments displayed as "undefined" in parent project page
* Fix maximizing UI combined scalar plots displays split scalar plots
* Fix adding description to UI dataset does not work
* Fix empty "parent" field in task cloned in UI
* Fix running UI application instances can be deleted
* Fix queue option list does not open in UI enqueue modal

46
docs/release_notes/clearml_server/open_source/ver_1_9.md Normal file
View File

@@ -0,0 +1,46 @@
---
title: Version 1.9
---
### ClearML Server 1.9.2
**Bug Fixes**
* Fix "Unable to load image" error in UI experiment debug samples after entering storage credentials ([ClearML Web GitHub issue #46](https://github.com/allegroai/clearml-web/issues/46))
* Fix incorrect default project name displayed in UI "Clone Experiment" modal ([ClearML Server GitHub issue #171](https://github.com/allegroai/clearml-server/issues/171))
* Fix UI experiment debug sample viewer iteration slider not working
* Fix UI experiment console log jumps backwards unnecessarily on refresh
* Fix UI published report is not expandable
* Fix UI published report's description is not editable
* Fix UI experiment-reset popup displays incorrect experiment count
### ClearML Server 1.9.1
**New Features and Improvements**
* Add collapsible description panel for UI reports
* Add in-app markdown guide for UI reports
**Bug Fixes**
* Fix can't generate new pipeline runs from UI ([ClearML Server GitHub issue #169](https://github.com/allegroai/clearml-server/issues/169))
* Fix plot legend texts overlap in UI reports
* Fix UI embedded plot colors are sometimes obscure
* Fix `ctrl z` doesn't undo codeblocks in UI reports
* Fix "Getting Started" popup wrongfully showing when enqueuing tasks
### ClearML Server 1.9.0
**New Features and Improvements**
* New ClearML Reports UI: Create and share rich MarkDown documents supporting embeddable online content ([ClearML GitHub issue #839](https://github.com/allegroai/clearml/issues/839))
* Add user option to not show example content in the UI ([ClearML GitHub issue #774](https://github.com/allegroai/clearml/issues/774))
* Add tag filter to UI Pipeline, Dataset, and Hyper-Dataset pages
* Extend UI experiment comparison up to 100 experiments
* Allow project default output destination S3 URLs to include dots (`.`) and hyphens (`-`)
**Bug Fixes**
* Fix long experiment names break UI experiment comparison scalar plot display ([ClearML GitHub issue #166](https://github.com/allegroai/clearml-server/issues/166))
* Fix plot legend definitions set by SDK don't override UI default settings
* Fix comparing experiments with read-only components causes error
* Fix UI model link doesn't preserve double spaces, breaking the URI
* Fix tasks/models/datasets can't be moved to UI root project
* Fix UI tag color list doesn't display tags
* Fix UI dashboard search returns results from team's work when "My Work" filter is enabled
* Fix UI experiment console does not display complete text when large text is reported

24
docs/release_notes/clearml_serving/ver_1_0.md Normal file
View File

@@ -0,0 +1,24 @@
---
title: Version 1.0
---
### ClearML Serving 1.0.0
:::warning Backwards Compatibility
This release is not backwards compatible
:::
**Breaking Changes**
* `preprocess` and `postprocess` class functions get 3 arguments
* Add support for per-request state storage, passing information between the pre/post-processing functions
**Features and Bug Fixes**
* Optimize serving latency while collecting statistics
* Fix metric statistics collecting auto-refresh issue
* Fix live update of model preprocessing code
* Add `pandas` to the default serving container
* Add per endpoint/variable statistics collection control
* Add `CLEARML_EXTRA_PYTHON_PACKAGES` for easier additional python package support (serving inference container)
* Upgrade Nvidia Triton base container image to 22.04 (requires Nvidia drivers 510+)
* Add Kubernetes Helm chart

20
docs/release_notes/clearml_serving/ver_1_1.md Normal file
View File

@@ -0,0 +1,20 @@
---
title: Version 1.1
---
### ClearML Serving 1.1.0
:::warning Backwards Compatibility
This release is not backwards compatible - see notes below on upgrading
:::
**Breaking Changes**
* Triton engine size supports variable request size (-1)
**Features and Bug Fixes**
* Add version number of serving session task
* Triton engine support for variable request (matrix) sizes
* Triton support, fix `--aux-config` to support more configurations elements
* Hugging Face Transformer support
* `Preprocess` class as module

15
docs/release_notes/clearml_serving/ver_1_2.md Normal file
View File

@@ -0,0 +1,15 @@
---
title: Version 1.2
---
### ClearML Serving 1.2.0
**New Features and Improvements**
* Improve GPU Performance, 50%-300% improvement over vanilla Triton
* Improve performance on CPU, optimize uvloop + multi-processing
* Add Hugging Face Transformer example
* Add binary input support ([ClearML Serving PR #37](https://github.com/allegroai/clearml-serving/pull/37) )
**Bug Fix**
* stdout/stderr in inference service not logging to dedicated Task

27
docs/release_notes/clearml_serving/ver_1_3.md Normal file
View File

@@ -0,0 +1,27 @@
---
title: Version 1.3
---
### ClearML Serving 1.3.1
**New Features and Bug Fixes**
* Add missing `await` ([ClearML Serving GitHub PR #55](https://github.com/allegroai/clearml-serving/pull/55))
* Add traceback for failing to load `preprocess` class ([ClearML Serving GitHub issue #57](https://github.com/allegroai/clearml-serving/issues/57))
* Fix Triton `config.pbtxt` for missing values or colliding specifications ([ClearML Serving GitHub issue #62](https://github.com/allegroai/clearml-serving/issues/62))
* Add safer code for pulling from Kafka
* Add `str` type to Triton type conversion
* Fix ignore auto-detected platform when passing `config.pbtxt` with platform entry
* Fix Triton engine model with multiple versions was not properly supported
* Fix serving session keep alive is also sent on idle
* Log preprocess exceptions with full stack trace to serving session console output
### ClearML Serving 1.3.0
**New Features and Improvements**
* 20% Overall performance increase
* gRPC channel configuration ([ClearML Serving GitHub issue #49](https://github.com/allegroai/clearml-serving/issues/49))
* Hugging Face Transformer example
**Bug fixes**
* Fix NumPy compatibility ([ClearML Serving GitHub issue #47](https://github.com/allegroai/clearml-serving/issues/47) and [#46](https://github.com/allegroai/clearml-serving/issues/46))
* Fix Triton examples ([ClearML Serving GitHub issue #48](https://github.com/allegroai/clearml-serving/issues/48))
* Add storage environment variables ([ClearML Serving GitHub PR #45](https://github.com/allegroai/clearml-serving/pull/45))

31
docs/release_notes/sdk/enterprise/ver_3_10.md Normal file
View File

@@ -0,0 +1,31 @@
---
title: Version 3.10
---
### AllegroAI 3.10.2
**Bug Fixes**
* Fix duplicate frame registration changes the source URI for HyperDatasets
* Fix incorrect mask ID is provided when creating an annotation with an RoiMask
* Fix passing Iterable objects raises a subscript exception when being used in `_to_api_object`
* Fix SingleFrame annotation of type mask_rgb does not contain the correct mask ID
### AllegroAI 3.10.1
**New Features**
* Use ClearML 1.13.0 SDK
**Bug Fixes**
* Fix requirements conflict with ClearML SDK requirements (upper limit constraints removed)
* Fix deprecated login service support causes exception when loading `APIClient()`
### AllegroAI 3.10.0
**New Feature**
* `Dataset.project` property for Hyperdataset project ID.
**Bug Fixes**
* Fix `allegroai.CachedStorageHelper` and `clearml.StorageHelper` compatibility
* Fix `DatasetVersion.update_frames()` doesn't update metadata when keys are removed
* Fix `DataView.Store()` raises errors
* Fix UI DataView frame query values are not applied at runtime.

20
docs/release_notes/sdk/enterprise/ver_3_11.md Normal file
View File

@@ -0,0 +1,20 @@
---
title: Version 3.11
---
### AllegroAI 3.11.1
**Bug Fix**
* Fix updating frames in `BulkContext` sometimes causes infinite recursion.
### AllegroAI 3.11.0
**New Features and Improvements**
* Add `DataView.get_current_iteration()` that returns current iteration
* Add `DataView.set_initial_iteration()` to set initial iteration, instead of zero.
* Allow dataset version filtering by tags with `tags` parameter in `DatasetVersion.get_datasets()`
* Add `Task.remove_dataviews()` to delete dataviews from a task
**Bug Fixes**
* Fix frames that failed to upload still upload their metadata
* Fix SingleFrame registration to a DatasetVersion failing
* Fix `DataView.add_query`'s `queries` parameter does not accept the returned results of `DataView.get_queries`

13
docs/release_notes/sdk/enterprise/ver_3_12.md Normal file
View File

@@ -0,0 +1,13 @@
---
title: Version 3.12
---
### AllegroAI 3.12.0
**New Features**
* Add `max_request_size_mb` parameter to `DatasetVersion.add_frames()` to set the maximum request size sent to the
server when adding frames.
* Add `status` property to `DataView`
**Bug Fix**
* Improve `DatasetVersion.add_frames()` error message clarity

100
docs/release_notes/ver_0_10.md → docs/release_notes/sdk/open_source/ver_0_10.md
View File

@@ -6,9 +6,7 @@ title: Version 0.10
**Trains** is now **ClearML**.
:::
## Version 0.10.7
### Trains
### Trains 0.10.7
**Features and Bug Fixes**
@@ -16,49 +14,45 @@ title: Version 0.10
* Removed apache-libcloud from requirements.
* `trains-init` now verifies credentials against the trains-server installation.
## Version 0.10.6
### Trains
### Trains 0.10.6
**Features and Bug Fixes**
* Fix broken (v0.10.5) Keras Binding support.
## Version 0.10.5
### Trains
### Trains 0.10.5
**Features and Bug Fixes**
* Add GPU monitoring support (add gpustat package to extras_require).
* Install with GPU monitoring support:
pip install trains[gpu]
```
pip install trains[gpu]
```
* Move all cloud storage package requirements to extras_require. Install with specific cloud provider support:
* Microsoft Azure support:
pip install trains[azure]
```
pip install trains[azure]
```
* Google Storage support:
pip install trains[gs]
```
pip install trains[gs]
```
* Amazon S3 support:
pip install trains[s3]
```
pip install trains[s3]
```
* Combine Cloud support with GPU monitoring. For example, install S3 and GPU using the following command:
pip install trains[s3,gpu]
```
pip install trains[s3,gpu]
```
* Improve stability with intermittent network connection.
* Support upgrading trains-server while running training jobs without losing log data.
## Version 0.10.4
### Trains
### Trains 0.10.4
**Features and Bug Fixes**
@@ -66,9 +60,8 @@ title: Version 0.10
* Improve matplotlib support (custom axis ticks).
* Improve Python package detection.
## Version 0.10.3
### Trains
### Trains 0.10.3
**Features and Bug Fixes**
@@ -79,9 +72,7 @@ title: Version 0.10
* Fix sub-process support.
* Fix multiple TensorBoard writers ([GitHub Issue #26](https://github.com/allegroai/trains/issues/26)).
## Version 0.10.2
### Trains
### Trains 0.10.2
**Features and Bug Fixes**
@@ -92,34 +83,14 @@ title: Version 0.10
* Add Google Colab support.
* Fix TensorBoard RGB channel order.
## Version 0.10.1
### Trains
### Trains 0.10.1
**Features and Bug Fixes**
* Fix Jenkins CI/CD support.
### Trains Server
* Add simple username / password login scheme, see [GitHub trains-server Issue 3](https://github.com/allegroai/trains-server/issues/3)
* Fixed User/Password configuration (see [instructions](../deploying_clearml/clearml_server_config.md#web-login-authentication)).
* Replace the Python web-server with NGINX.
* Improve sub-domain support ([GitHub trains-server Issue 9](https://github.com/allegroai/trains-server/issues/9)).
* Extend configuration options
* Environment variables for overriding mongoDB and Elastic hosts URLs.
* File server disabling browser cache.
* File server compression support.
* File server CORS support.
## Version 0.10.0
### Trains
### Trains 0.10.0
* Experiment code execution detection
@@ -135,9 +106,9 @@ title: Version 0.10
* CPU, network, I/O, memory, and other resources.
* For GPU support please install gpustat.
(currently not part of the requirements due to gpustat compatibility issues with Windows).
pip install gpustat
```
pip install gpustat
```
* Automatically stop inactive experiments (default: 2 hours)
* Improve visibility
@@ -149,21 +120,4 @@ title: Version 0.10
* Add vertical guide in scalar plots.
* Move hyperparameters to the designated tab.
* "Admin" section now named "Profile".
### Trains Server
* Experiment code execution detection.
* Add source-control diff section.
* Add code requirements section.
* Automatically stop inactive experiments (default: 2 hours).
* Configuration details in readme.md:
* Improve visibility:
* Finer status definitions: Identify successful completion vs. user aborted.
* Experiment plot comparison: Ensure different colors for different experiments.
* Parse newline character in experiment description.
* Show experiment start time in table display.
* Add vertical guide in scalar plots.
* Move hyperparameters to the designated tab.
* "Admin" section now named "Profile".

22
docs/release_notes/ver_0_11.md → docs/release_notes/sdk/open_source/ver_0_11.md
View File

@@ -7,27 +7,23 @@ title: Version 0.11
:::
## Version 0.11.3
### Trains
### Trains 0.11.3
**Features and Bug Fixes**
* Resource-Monitor will only monitor active GPU devices based on environment variables: `NVIDIA_VISIBLE_DEVICES` or `CUDA_VISIBLE_DEVICES`.
* Fix issue ([GitHub Issue #48](https://github.com/allegroai/trains/issues/48)).
## Version 0.11.2
### Trains
### Trains 0.11.2
**Features and Bug Fixes**
* Fix Python 2.7 support.
* Improve sample code Windows support.
## Version 0.11.1
### Trains
### Trains 0.11.1
**Features and Bug Fixes**
@@ -39,9 +35,8 @@ title: Version 0.11
* Fix support for tuples in hyperparameters.
* Fix multi processing issues with different task types.
## Version 0.11.0
### Trains
### Trains 0.11.0
**Features and Bug Fixes**
@@ -56,12 +51,3 @@ title: Version 0.11
* TensorBoard scalars are not grouped into one graph, but are stored on individual graphs (to match TensorBoard behavior).
To restore previous behavior call: `Logger.tensorboard_auto_group_scalars(group_scalars=True)`.
### Trains Server
* Add scalar graphs smoothing (matched TensorBoard behavior).
* Add scalar graphs X-axis selection: Iteration / Relative / Wall Time.
* Add min / max / last value for all scalars (available in experiment table column selection / sorting).
* Add hyperparameter to experiment table custom columns.
* Add artifacts support (Models tab moved into artifacts as specific artifact type).
* Add full Azure storage support.
* Fix column sorting N/A values.

56
docs/release_notes/ver_0_12.md → docs/release_notes/sdk/open_source/ver_0_12.md
View File

@@ -5,9 +5,8 @@ title: Version 0.12
**Trains** is now **ClearML**.
:::
## Version 0.12.2
### Trains
### Trains 0.12.2
**Features and Bug Fixes**
@@ -23,19 +22,8 @@ title: Version 0.12
Do not reuse an experiment with artifacts.
:::
### Trains Agent
**Features and Bug Fixes**
* Improve configuration wizard.
* Improve Docker and Kubernetes support.
* Add `docker build` command.
* Add initial Poetry support.
* Fix Docker CUDA support.
## Version 0.12.1
### Trains
### Trains 0.12.1
**Features and Bug Fixes**
@@ -51,27 +39,8 @@ Do not reuse an experiment with artifacts.
* Fix Python2 support.
* Fix Issue ([GitHub Issue #56](https://github.com/allegroai/trains/issues/56)).
### Trains Server
Minor bug fixes and improvements
* Add resource monitoring.
* Fix Web UI compare plots ([GitHub Issue #55](https://github.com/allegroai/clearml/issues/55)).
* Improve server upgrade checks/messages.
### Trains Agent
**Features and Bug Fixes**
* Windows support! YES, you can now have Windows machines as part of your cluster (notice --docker is not supported on Windows).
* Add initial Conda package manager support (still in beta).
* Add --gpus and --cpu-only for easier GPU control when running multiple `trains-agent` instances on the same machine.
* [python_binary](https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf#L35) can now be specified (the default is the same Python binary executing the `trains-agent`).
* Fix Issue ([GitHub Issue #2](https://github.com/allegroai/trains-agent/issues/2)).
## Version 0.12.0
### Trains
### Trains 0.12.0
**Features and Bug Fixes**
@@ -84,22 +53,3 @@ Minor bug fixes and improvements
* Fix artifacts support on Windows.
* Fix example code Windows support.
### Trains Server
* Add full DevOps support.
* Add job scheduling queue management.
* Add cluster management (TRAINS Agent workers).
* Add system wide auto-refresh.
### Trains Agent
TRAINS Agent - Initial Release
* TRAINS Agent currently supports Linux and Mac.
* Python packaging system supported: pip.
* Containers supported: Docker v19.03 and above.
* Version Control support: git
* Support for standalone scripts and Jupyter Notebooks.
* TRAINS-server v0.12 and above.
* TRAINS v0.11.3 and above.
* Sample configuration file available [here](https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf).

59
docs/release_notes/ver_0_13.md → docs/release_notes/sdk/open_source/ver_0_13.md
View File

@@ -6,9 +6,8 @@ title: Version 0.13
**Trains** is now **ClearML**.
:::
## Version 0.13.3
### Trains
### Trains 0.13.3
**Features and Bug Fixes**
@@ -27,26 +26,8 @@ title: Version 0.13
* Fix incorrect printouts in initialization wizard and upgrade notifications.
* Fix debug images URL for uploaded files with `%` in their name.
### Trains Agent
**Features and Bug Fixes**
* Allow providing queue names instead of queue IDs in daemon mode.
* Improve Docker mode:
* Support running as a specific user inside a docker using the `TRAINS_AGENT_EXEC_USER` environment flag.
* Pass the correct GPU limit when skipping gpus flag.
* Add the `--force-current-version` daemon command-line flag.
* Add K8s/trains glue service example.
* Add K8s support in daemon mode.
* Running inside a K8s pod.
* Mounting dockerized experiment folders to host.
* Allow a specific network for the docker.
* Add default storage environment vars (for AWS, GS and Azure) to generated agent configuration.
* Improve Unicode/UTF stdout handling.
## Version 0.13.2
### Trains
### Trains 0.13.2
**Features and Bug Fixes**
@@ -58,9 +39,7 @@ title: Version 0.13
* Fix [savefig](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.savefig.html) patching in matplotlib binding.
* Fix logs, events and Jupyter Notebook flushing on exit.
## Version 0.13.1
### Trains
### Trains 0.13.1
**Features and Bug Fixes**
@@ -71,9 +50,8 @@ title: Version 0.13
* Do not store keras model network design if it cannot be serialized ([GitHub Issue #72](https://github.com/allegroai/trains/issues/72)).
* Fix matplotlib binding support.
## Version 0.13.0
### Trains
### Trains 0.13.0
**Features and Bug Fixes**
@@ -95,32 +73,3 @@ title: Version 0.13
* Fix argparser binding sub-parser and type casting support ([GitHub Issue #74](https://github.com/allegroai/trains/issues/74)).
* Fix argparser binding Python 2.7 unicode handling.
* Fix unsynced connected hyperparameters overridden during remote execution.
### Trains Server
**Features and Bug Fixes**
* Add parallel coordinates hyperparameter comparison, available under **Compare Experiments** **>** **HYPERPARAMETERS**
**>** **Parallel Coordinates** (in the drop-down) ([GitHub Issue #53](https://github.com/allegroai/trains/issues/53)).
* Add encoding of experiment table view settings in URL to allow sharing, using browser URL copy / paste.
* Add loguru (ANSI color) support ([GitHub Issue #29](https://github.com/allegroai/trains/issues/29)).
* Add support for special characters in hyperparameter keys (white-spaces, `.` and `$`) ([GitHub Issue #69](https://github.com/allegroai/trains/issues/69)).
* Add optional anonymous daily usage statistics (help us improve Trains Server):
* Disabled by default.
* Requires user opt-in.
* Single averages report per day.
* Reports average load metrics per day (CPU / memory).
* Reports average workload per day (amount and average duration of queues, agents and experiments).
* Improve experiment table filtering indication.
* Improve model view to allow navigating to its generating experiment.
* Fix experiment comparison to distinguish between experiments with the same name ([GitHub Issue #52](https://github.com/allegroai/trains/issues/52)).
* Fix Web UI compare plots bug ([GitHub Issue #55](https://github.com/allegroai/trains/issues/55) and [GitHub Issue #73](https://github.com/allegroai/trains/issues/73)).
### Trains Agent
**Features**
* Add support for Docker pre-installed pytorch versions that do not exist on PyPI/PyTorch.org.
* Add AWS dynamic cluster management service.
* Add support for various event query endpoints in APIClient.
* Improve the configuration wizard.

99
docs/release_notes/sdk/open_source/ver_0_14.md Normal file
View File

@@ -0,0 +1,99 @@
---
title: Version 0.14
---
:::important
**Trains** is now **ClearML**.
:::
### Trains 0.14.3
**Features**
* Add the `Logger.report_histogram` method `mode` parameter.
The values are `group` (the default), `stack`, and `relative`.
* Add [PEP610](https://www.python.org/dev/peps/pep-0610/) support for the new `pip` version 20.1 and newer Git reference
feature (see [pypa/pip#609](https://github.com/pypa/pip/issues/609)) ([GitHub Issue #62](https://github.com/allegroai/trains/issues/62)).
* Improve support for PyTorch Ignite integration.
* Support `#` and `?` in uploaded image file names.
**Bug Fixes**
* Support `ArgumentParser.parse_arg` use before initializing a Task when `auto_connect_arg_parser=False`.
* Fix `upload_object_via_stream` in the Azure storage driver.
* Fix the `Task.get_tasks` method return type.
### Trains 0.14.2
**Features**
* Add media (audio) support for both `Logger` and TensorBoard binding using `Logger.report_media` ([GitHub Issue #120](https://github.com/allegroai/trains/issues/120)).
* Add [autokeras example](https://github.com/allegroai/trains/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) ([GitHub Issue #125](https://github.com/allegroai/trains/issues/125)).
* Add caching of extracted zip artifacts.
* Add `Task.get_tasks` filtering support.
* Add `trains.storage.StorageManager`.
* Add nicer stdout log flush.
* Add text logging to TF V1 example.
* Improve SSL retries and error handling.
* Improve shutdown behavior in case of a user abort (Ctrl-C).
* Improve repository and module detection:
* Improve resource monitor.
* Report memory usage for process (and sub-processes).
* Report GPU memory usage for process (and sub-processes).
* Stability
* Improve Jupyter support:
* Make sure `trains` is included in Jupyter requirements.
* Ignore IPython directives in converted Python script (like `%` and `!` lines).
* Update PyTorch / TensorboardX examples.
**Bug Fixes**
* Fix renaming / deleting model file right after saving will break asynchronous upload ([GitHub Issue #123](https://github.com/allegroai/trains/issues/123)).
* Fix shutdown behavior:
* Logger flush delay and termination.
* Repository / package detection termination.
* Threads termination.
* Fix text encoding `utf-8` and `pr_curve` broken in Tensorboard support.
* Fix single log request exceeds max packet size.
* Fix upload debug assets with series / title including `,` or `/` did not add iteration index to the filename.
* Fix `scikit-learn` module detection.
* Use UID in message if username cannot be retrieved.
* Fix cast in Tensorflow v2 example.
### Trains 0.14.1
**Features and Bug Fixes**
* Add OpenMPI / Slurm support, support `TRAINS_FORCE_MASTER_NODE` environment variable to allow forcefully specifying a node is master (not based on the MPI Rank).
* Add `Task.get_models` and `Task.models` for retrieving stored models for previously executed tasks
* Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) ([GitHub Issue #26](https://github.com/allegroai/trains/issues/26)).
* Support reusing models.
* Fix TensorFlow not a number (`NaN`) and infinity (`Inf`)
values support ([GitHub Issue #118](https://github.com/allegroai/trains/issues/118)).
* Fix joblib binding.
* Fix metrics / files / models upload on shutdown.
* Fix support for Task init / close multiple times.
* Update `manual_reporting.py` example with `Logger.report_table` example.
* Deprecate `Task.set_model_config`, `Task.get_model_config_text`, and `Task.get_model_config_dict`.
### Trains 0.14.0
**Features and Bug Fixes**
* Add support for reporting tables (pandas.DataFrame, CSV files, URL to CSV file) using the `Logger.report_table` method.
* Add support for setting a Task's initial iteration to allow continuing previous runs using the `Task.set_initial_iteration` and `Task.get_initial_iteration` methods.
* Add multiprocess task support ([GitHub Issue #96](https://github.com/allegroai/trains/issues/96)).
* Add `trains-init` support for config file override using the `TRAINS_CONFIG_FILE` environment variable or `--file` command line argument ([GitHub Issue #105](https://github.com/allegroai/trains/issues/105)).
* Add initial slurm support.
* Allow disabling repository detection when calling `Task.init`.
* Add Jupyter Notebook server execution example.
* Update examples.
* Improve warning messages for storage errors and missing storage packages.
* Fix HTTP link quoting in stored links.

74
docs/release_notes/sdk/open_source/ver_0_15.md Normal file
View File

@@ -0,0 +1,74 @@
---
title: Version 0.15
---
:::important
**Trains** is now **ClearML**.
:::
### Trains 0.15.1
**Features**
* Add direct plotly figure reporting ([GitHub Issue #136](https://github.com/allegroai/trains/issues/136)).
* Improve matplotlib integration ([GitHub Issue #140](https://github.com/allegroai/trains/issues/140)).
* Add `Task.set_parent` ([GitHub Issue #144](https://github.com/allegroai/trains/issues/144)).
* Add `Task.wait_for_status` ([GitHub Issue #145](https://github.com/allegroai/trains/issues/145)).
* Enhance framework `save` / `load` binding callbacks ([pytorch/ignite Issue #1056](https://github.com/pytorch/ignite/issues/1056)).
* Add `TrainsJob.is_failed`.
* Add APIClient support using `trains.backend_api.session.APIClient`.
* Add URL tracking when using `StorageManager.get_local_copy` (log remote URL instead of local file copy).
* Improve and expand examples.
**Bug Fixes**
* Fix `pytorch_tensorboard.py` example ([GitHub Issue #148](https://github.com/allegroai/trains/issues/148)).
* Fix `ssh://user@domain@server.com/` in Git repository ([GitHub Issue #149](https://github.com/allegroai/trains/issues/149)).
* Fix do not call `make_deterministic` on import.
* Fix initialization wizard (allow at most two verification retries, then print error) ([GitHub trains-agent Issue #16](https://github.com/allegroai/trains-agent/issues/16)).
* Fix Google CoLab code / package detection.
* Improve pytorch-ignite integration.
* Fix TensorFlow version 2 and later histogram binding.
* Fix `Logger.tensorboard_single_series_per_graph`.
* Fix anonymous named models.
* Fix incorrect entry point detection when called from Trains wrapper (e.g. `TrainsLogger` in PyTorch Ignite / Lightning).
### Trains 0.15.0
**Features**
* Add automation support including hyperparameter optimization (see example [here](https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py))
* `Task.init` `auto_connect_arg_parser` argument can accept a dictionary disabling specific keys from the argparser (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589355878206700))
* Allow `worker_id` override using `TRAINS_WORKER_NAME` environment variable (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589920581251600?thread_ts=1589920220.250100&cid=CTK20V944))
* Support layout configuration for plotly objects using `extra_layout` argument in all `Logger` reporting methods ([GitHub Issue #136](https://github.com/allegroai/trains/issues/136)).
* Add `Task.execute_remotely` to allow cloning and enqueuing a locally executed task (or stopping and re-enqueuing a remotely executed task) ([GitHub Issue #128](https://github.com/allegroai/trains/issues/128)).
* Add Parquet framework and model type.
* Support recursive model folder packaging.
* Add `Task.get_reported_console_output` and `Task.get_reported_scalars` to allow retrieval of reported output and scalar metrics.
* Add `Task.add_requirements` to force requirement package into "installed packages."
* Improve task reuse responsiveness.
* Add `raise_on_error` (default False) argument to `Model.get_local_copy` and `Artifact.get_local_copy` ([GitHub trains-agent Issue #17](https://github.com/allegroai/trains-agent/issues/17)).
* Support `Task.get_task` without project name (i.e. all projects)
* Support using the file calling `Task.init` as the task's script in case sys.argv doesn't point to a git repository
* Support detecting and remotely executing code running from a module (i.e. `-m module`)
* Add callback for framework `save` / `load` binding for better integration with pytorch / ignite ([GitHub pytorch/ignite Issue #1056](https://github.com/pytorch/ignite/issues/1056))
* Support new task types provided in Trains Server v0.15.0
* Add automation and distributed examples
* Upgrade default pip version to `<20.2`
**Bug Fixes**
* Fix `exact_match_regex` in case of empty pattern ([GitHub Issue #138](https://github.com/allegroai/trains/issues/138)).
* Address deprecation warning and newer `attrs` versions in MetricsEventAdapter ([GitHub Issue #134](https://github.com/allegroai/trains/issues/134)).
* Fix issues with plotly support (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589871253243600) and [thread](https://clearml.slack.com/archives/CTK20V944/p1589279340195000)).
* Fix default argument behavior to match argparse behavior.
* Fix `OutputModel` with `task=None` should use current task, if exists.
* Fix `Task.get_task()` to raise proper error on incorrect `task_id`.
* Fix `Task.enqueue()` to use an exact queue name match.
* Fix `NaN`, `Inf` and `-Inf` values display in reported table (not supported by JSON).
* Limit max requirement size to 0.5 MB.
* Fix issues with repository analysis.
* Fix `StorageManager` so it should only try to extract .zip files, `Model` should not auto extract package ([GitHub trains-agent Issue #17](https://github.com/allegroai/trains-agent/issues/17)).

158
docs/release_notes/ver_0_16.md → docs/release_notes/sdk/open_source/ver_0_16.md
View File

@@ -5,9 +5,8 @@ title: Version 0.16
**Trains** is now **ClearML**.
:::
## Version 0.16.4
### Trains
### Trains 0.16.4
**Features**
@@ -54,9 +53,8 @@ title: Version 0.16
- Fix connect dict containing `None` default values, blocked the remote execution from passing string instead of None.
- Fix `Task.upload_artifact()` argument `delete_after_upload=True` used in conjunction with `wait_for_upload=True` was not supported.
## Version 0.16.3
### Trains
### Trains 0.16.3
**Features**
@@ -96,9 +94,8 @@ title: Version 0.16
* Do not add specific setuptools version to requirements (pip can't install it anyway).
* Fix image URL quoting when uploading from a file path.
## Version 0.16.2
### Trains
### Trains 0.16.2
**Features**
@@ -136,54 +133,8 @@ title: Version 0.16
* Fix `TrainsJob` hyper-params overriding order was not guaranteed.
* Fix ArgParse auto-connect to support default function type.
### Trains-Agent
**Features**
- conda:
- Add `agent.package_manager.conda_env_as_base_docker` allowing "docker_cmd" to contain link to a full pre-packaged conda environment (`tar.gz` created by `conda-pack`). Use `TRAINS_CONDA_ENV_PACKAGE` environment variable to specify `conda tar.gz` file.
- Add conda support for read-only pre-built environment (pass conda folder as `docker_cmd` on Task).
- Improve trying to find conda executable.
- k8s glue:
- Add support for limited number of services exposing ports.
- Add support for k8s pod custom user properties.
- Allow selecting external `trains.conf` file for the pod itself.
- Allow providing pod template, extra bash init script, alternate SSH server port, gateway address (k8s ingress / ELB).
- Allow specifying `cudatoolkit` version in the "installed packages" section when using conda as package manager (<a href="https://github.com/allegroai/trains/issues/229" target="_blank">GitHub trains Issue 229</a>).
- Add `agent.package_manager.force_repo_requirements_txt`. If True, "Installed Packages" on Task are ignored, and only repository `requirements.txt` is used.
- Pass `TRAINS_DOCKER_IMAGE` into docker for interactive sessions.
- Add `torchcsprng` and `torchtext` to PyTorch resolving.
**Bug Fixes**
- When logging suppress "\r" when reading a current chunk of a file / stream. Add `agent.suppress_carriage_return` (default True) to support previous behavior.
- Make sure `TRAINS_AGENT_K8S_HOST_MOUNT` is used only once per mount.
- Fix k8s glue script to trains-agent default docker script.
- Fix apply git diff from submodule only.
- conda:
- Fix conda pip freeze to be consistent with trains 0.16.3.
- Fix conda environment support for trains 0.16.3 full env. Add `agent.package_manager.conda_full_env_update` to allow conda to update back the requirements (default False, to preserve previous behavior).
- Fix running from conda environment - `conda.sh` not found in first conda PATH match.
- Fix docker mode ubuntu / debian support by making sure not to ask for input (fix `tzdata` install).
- Fix repository detection - ignore environment `SSH_AUTH_SOCK`, only check if git user/pass are configured.
- git diff:
- Fix support for non-ascii diff.
- Fix diff with empty line at the end will cause corrupt diff apply message.
- Allow zero context diffs (useful when blind patching repository).
- Fix `daemon --stop` when agent UID cannot be located.
- Fix nvidia docker support on some linux distros (SUSE).
- Fix nvidia pytorch dockers support.
- Fix torch CUDA 11.1 support.
- Fix requirements dict with null entry in `pip` should be considered None install from repository's `requirements.txt`.
## Version 0.16.1
### Trains
### Trains 0.16.1
**Features**
@@ -199,45 +150,8 @@ title: Version 0.16
* Fix artifact upload to only use file stream when not uploading a locally stored file (multipart upload is not supported on stream upload) (<a href="https://github.com/allegroai/trains/issues/189" target="_blank">GitHub trains Issue 189</a>).
* Fix double-escaped model design text when connecting OutputModel.
### Trains Server
:::important
Upgrading to this version requires a manual [data migration](../deploying_clearml/clearml_server_es7_migration.md).
:::
**Bug Fixes**
* Fix model page issue causing N/A to show after switching tabs (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597672446125300).
* Removed experiments comparison limit (only 10 were allowed). Limit is now 100, configurable using `services.tasks.multi_task_histogram_limit`. (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597755746141500).
* Fix scalar plots sometimes not calculated by the server in lower iteration values (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597321653085900).
* Fix error while retrieving experiment log when only a few lines were reported ([GitHub trains-server Issue 59](https://github.com/allegroai/trains-server/issues/59)).
* Update Fixed User full-name on restart (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597906492003300).
* Fix project ordering issue.
* When loading plots, display a spinner and don't show "no data".
* Improve logging to provide more coherent ElasticSearch connection status in server log.
### Trains Agent
**Features**
* Add `sdk.metrics.plot_max_num_digits` configuration option to reduce plot storage size.
* Add `agent.package_manager.post_packages` and `agent.package_manager.post_optional_packages` configuration options to control packages install order (e.g. horovod).
* Add `agent.git_host` configuration option for limiting git credential usage for a specific host (overridable using `TRAINS_AGENT_GIT_HOST` environment variable).
* Add `agent.force_git_ssh_port` configuration option to control HTTPS to SSH link conversion for non-standard SSH ports.
* Add requirements detection features. Improve support for detecting new pip version (20+) supporting `package @ scheme://link`.
**Bug Fixes**
* Fix pre-installed packages are ignored when installing a git package wheel. Reinstalling a `git+http` link is enough
to make sure all requirements are met / installed ([GitHub Issue #196](https://github.com/allegroai/trains/issues/196)).
* Fix incorrect check for spaces in current execution folder.
* Fix requirements detection:
* Update torch version after using downloaded / system pre-installed version.
* Do not install git packages twice when a new pip version is used (pip freeze will detect the correct git link version).
## Version 0.16.0
### Trains
### Trains 0.16.0
**Features**
@@ -287,65 +201,3 @@ Upgrading to this version requires a manual [data migration](../deploying_clearm
* Fix multiple `-e` packages were not detected (only the first one).
* Fix running with Trains in `PYTHONPATH` resulted in double entry of trains.
* Fix `Task.set_base_docker()` on main task to do nothing when running remotely.
### Trains Server
:::important
Upgrading to this version requires a manual [data migration](../deploying_clearml/clearml_server_es7_migration.md).
:::
**Features**
* Add experiment hyperparameter grouping:
* **HYPER PARAMETERS** tab renamed to **CONFIGURATION**.
* **CONFIGURATION** tab contains the sections **USER PROPERTIES**, **HYPER PARAMETERS**, **CONFIGURATION OBJECTS**
* Add user properties group. Key-value pairs always editable (**USER PROPERTIES** section).
* Add command line options group * argparse and older experiments parameters (**CONFIGURATIONS / HYPER PARAMETERS / Args**).
* Add TensorFlow definitions group (**CONFIGURATIONS / HYPER PARAMETERS / TF_DEFINE**).
* Add environment variables group (**CONFIGURATIONS / HYPER PARAMETERS / Environment**).
* Improve experiment model configuration:
* Model design is in the **ARTIFACTS** tab.
* Experiment model description is in the **CONFIGURATION OBJECTS** section in the **CONFIGURATION** tab.
* Improve experiment comparison:
* In hyperparameter parallel coordinate comparison, hover over an experiment name to highlight it on plot ([GitHub Issue #53](https://github.com/allegroai/trains/issues/53)).
* Remove fields providing no additional information from comparison.
* Improve the model framework filter. Filter contains only frameworks used by models in the project.
* Add configurable **Trains** services examples.
* Add support for text debug samples in the **DEBUG SAMPLES** section in the **RESULTS** tab.
* Add legend on / off toggle control for every plot.
* Add clear button for text areas ([GitHub trains-server Issue #42](https://github.com/allegroai/trains-server/issues/42)).
* Reinstate the bottom bar **Archive** button.
* Add **Trains** community links to left bar.
* Add Hi-DPI display support.
* Add `debug.ping` endpoint for simple health monitoring.
* Add support for field exclusion in `*.get_all` endpoints.
* Move to ElasticSearch 7. Requires manual [data migration](../deploying_clearml/clearml_server_es7_migration.md).
**Bug Fixes**
* Auto-fit column width on column resize double click.
* Allow top-bar search if fewer than three characters are entered, and `Enter` is pressed.
### Trains Agent
**Features**
* Add `agent.docker_init_bash_script` configuration section to allow finer control over Docker startup script.
* Changed default Docker image from `nvidia/cuda` to `nvidia/cuda:10.1-runtime-ubuntu18.04` to support `cudnn` frameworks (e.g. TF).
* Improve support for Dockers with preinstalled `conda` environment.
* Improve trains-agent-docker spinning.
* Add `daemon --order-fairness` for round-robin queue pulling.
* Add `daemon --stop` to terminate a running agent (assuming other arguments are the same). If no additional arguments, Agents are terminated in lexicographical order.
* Support cleanup of all log files on termination unless executed with `--debug`.
* Add error message when Trains API Server is not accessible on startup.
**Bug Fixes**
* Fix GPU Windows monitoring support ([GitHub Issue #177](https://github.com/allegroai/trains/issues/177)).
* Fix `.git-credentials` and `.gitconfig` mapping into docker.
* Fix non-root docker image usage.
* Fix docker to use `UTF-8` encoding, so prints won't break it.
* Fix `--debug` to set all loggers to `DEBUG`.
* Fix task status change to `queued` should never happen during Task runtime.
* Fix `requirement_parser` to support `package @ git+http` lines.
* Fix GIT user/password in requirements and support for `-e git+http` lines.
* Fix configuration wizard to generate `trains.conf` matching latest Trains definitions.

112
docs/release_notes/ver_0_17.md → docs/release_notes/sdk/open_source/ver_0_17.md
View File

@@ -81,48 +81,7 @@ title: Version 0.17
- Fix Python 2.7 support.
## ClearML Agent 0.17.2
**Features**
- Add virtual environment caching:
* Supports venv caching both in standard and docker mode.
* Configurable using the `agent.venvs_cache` configuration section.
* Disabled by default, enable [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L101).
- Add support for `--services-mode` with venvs.
- Add `agent.force_git_ssh_user` configuration value (default `git`, see [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L29))
([clearml-agent GitHub Issue 42](https://github.com/allegroai/clearml-agent/issues/42)).
- Add `agent.ignore_requested_python_version` configuration option for multi Python environments (default `false`).
- Add `agent.enable_task_env` configuration option to set the OS environment based on the Environment section of the Task (default `false`, see
[here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L151)).
- K8s glue:
* Add support for detecting and deleting k8s pods that fail to start.
* Allow providing namespace in k8s glue and k8s glue example.
* Add base-pod-number parameter to k8s glue and example.
- Change `agent.default_docker.image` to `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` (see [here](https://github.com/allegroai/clearml-agent/blob/8a46dc6b03860bc1cc022fd19e86bf4524986a0e/docs/clearml.conf#L144)).
- Use shared git cache for multiple agents on the same machine.
- Upgrade pynvml add detect CUDA version from driver level.
- Update agent and services docker files.
- Update documentation.
**Bug Fixes**
- Fix `docker --network` returns `None`.
- Fix docker mode without venvs cache dir.
- Fix applying git diff on a newly added file.
- Fix environment variables `CLEARML_WEB_HOST`/`CLEARML_FILES_HOST` not passed to running tasks (or updated on the config object).
- Fix `--detached` command line option not supported on Windows (ignore and issue warning).
- Fix file not found error (errno 2) interpreted as aborted (i.e. `Ctrl-C`).
- Fix `from clearml` runtime diff patching.
- Fix cache to take cuda version into account.
- Fix CPU mode.
- Fix multi instances on Windows.
- Fix conda support for `git+http` links.
- Fix k8s glue does not pass docker environment variables, remove deprecated flags.
## Version 0.17.4
### ClearML
### ClearML 0.17.4
**Features**
@@ -139,9 +98,8 @@ title: Version 0.17
- Fix PyJWT v2.0 token parsing.
- Fix Python package detection it should not list `file://` links.
## Version 0.17.3
### ClearML
### ClearML 0.17.3
**Features**
@@ -174,25 +132,22 @@ title: Version 0.17
* Fix dataset genealogy, graph and restoring data.
* Fix numpy dependency for Python versions.
## Version 0.17.2
### ClearML
### ClearML 0.17.2
**Bug Fixes**
* Fix broken `clearml-task` CLI.
## Version 0.17.1
### ClearML
### ClearML 0.17.1
**Bug Fixes**
* `clearml-task` CLI link fixed.
## Version 0.17.0
### ClearML
### ClearML 0.17.0
**Features**
@@ -205,60 +160,3 @@ title: Version 0.17
* PyJWT v2.0 breaks the interface.
### Open Source ClearML Server
**Features**
* New profile page layout.
* Move knowledge resource to UI top bar.
* Add complete experiment full screen view.
* Add scalar results split / grouped view.
* Add experiment active duration custom column to experiment table.
* Improve comparison experiment consistency in plots for experiment name and color.
* Improve project last update time; updated whenever an experiment or model in the project is updated.
* Improve table multi-select UX: shift+mouse-click ([clearml GitHub Issue 61](https://github.com/allegroai/trains/issues/61)).
* Add experiment parent to experiment details INFO tab.
* Improve UI experiment archiving; archiving queued experiments automatically dequeues them.
* Improve experiment comparison; use available screen width.
* Add debug sample history tracking ([clearml GitHub Issue 81](https://github.com/allegroai/trains/issues/81)).
* Replace "Download" action with "Copy to Clipboard" for local files.
* Add ClearML usage tips.
* Add Task and Model, Clone and Move to allow creating new project on the fly.
* Add support for all S3 / minio special character \(%\) encoding cases.
* Add API for filter by parent.
* Improve browser-search to find data not visible in uncommitted changes / installed packages window.
* Improve Task artifacts optimization.
* Upgrade plotly to 1.57.1.
* Rename Log tab to Console.
**Bug Fixes**
* Fix HiDPI table issues.
* Fix results / plots not scaling properly ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)).
* Fix experiment comparison:
* Cannot compare regular Tasks to example Tasks.
* Cannot unhide scalars.
* Scalars wall time view and relative view will not auto-refresh.
* Fix log scrolling in Chrome.
* Fix UI adds to plots legend labels unnecessary task ID string ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)).
* Fix parallel coordinates: dimensions are not sorted by value.
* Fix scalar auto color selection provides too similar colors.
* Fix reordering a custom column does not restore column location.
* Fix uncommitted changes - not possible to copy the entire text.
* Fix queues wait time graph base should be 0, not in the middle.
* Fix plots view - "No Data" shown for a moment when changing experiments.
* Fix adding new tag during auto-refresh.
* Fix hide empty input model on non draft experiment.
* Fix experiment / model table - name column restores to default size after opening and closing info.
* Fix double click resizer should auto fit column.
### ClearML Hosted Service Only
* Launched free [ClearML Hosted Service](https://app.clear.ml/dashboard).
* Multiple workspaces.
* Workspace invitation.
* Add publicly sharing experiments.
### ClearML Agent
* Renamed `trains-agent` as `clearml-agent` (see [clearml-agent](https://github.com/allegroai/clearml-agent) in GitHub).

12
docs/release_notes/ver_0_9.md → docs/release_notes/sdk/open_source/ver_0_9.md
View File

@@ -6,9 +6,7 @@ title: Version 0.9
**Trains** is now **ClearML**.
:::
## Version 0.9.3
### Trains
### Trains 0.9.3
**Features and Bug Fixes**
@@ -18,17 +16,13 @@ title: Version 0.9
* Fix support for local git branches (i.e. no matching remote branch).
* Verify support for Official TensorFlow 1.14 release and PyTorch with TensorBoard.
## Version 0.9.2
### Trains
### Trains 0.9.2
**Features and Bug Fixes**
* Improve support for Windows.
## Version 0.9.1
### Trains
### Trains 0.9.1
**Features and Bug Fixes**

125
docs/release_notes/ver_1_0.md → docs/release_notes/sdk/open_source/ver_1_0.md
View File

@@ -1,29 +1,6 @@
---
title: Version 1.0
---
### ClearML Serving 1.0.0
:::warning Backwards Compatibility
This release is not backwards compatible
:::
**Breaking Changes**
* `preprocess` and `postprocess` class functions get 3 arguments
* Add support for per-request state storage, passing information between the pre/post-processing functions
**Features and Bug Fixes**
* Optimize serving latency while collecting statistics
* Fix metric statistics collecting auto-refresh issue
* Fix live update of model preprocessing code
* Add `pandas` to the default serving container
* Add per endpoint/variable statistics collection control
* Add `CLEARML_EXTRA_PYTHON_PACKAGES` for easier additional python package support (serving inference container)
* Upgrade Nvidia Triton base container image to 22.04 (requires Nvidia drivers 510+)
* Add Kubernetes Helm chart
### ClearML 1.0.5
**Features**
@@ -116,23 +93,6 @@ This release is not backwards compatible
- Fix matplotlib date convertor
- Fix automation controller overrides nodes clone
### ClearML Server 1.0.2
**Bug Fixes**
- Fix Task container does not accept `null` values ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1622119047293300) [ClearML GitHub issue 365](https://github.com/allegroai/clearml/issues/365))
- Fix debug images exception in Results page
- Fix a typo in Worker Setup help popup
### ClearML Server 1.0.1
**Bug Fixes**
- Fix clearing experiment requirements causes "empty" requirements (as opposed to "no requirements")
- Fix logout fails with `endpoint not found` error ([ClearML GitHub issue 349](https://github.com/allegroai/clearml/issues/349))
- Fix hamburger side menu `Manage Queues` does nothing and returns console error ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1620308724418100))
- Fix broken config dir backwards compatibility (`/opt/trains/config` should also be supported)
### ClearML 1.0.2
**Bug Fixes**
@@ -205,88 +165,3 @@ Arguments order changed in `Logger.report_line_plot()`, `Logger.report_plotly()`
* Improve flush behaviour on `KeyboardInterrupt` exception
* Wait for subprocesses to flush when terminating
* Fix `Task.set_credentials()` with `store_conf_file=True`
### ClearML Agent 1.0.0
**Features**
* Add conda and pip environment debug prints (using `--debug`)
* Add support for PyJWT v2
* Change the default conda channel order, so it pulls the correct `pytorch` package
* Improve k8s glue support
* Support k8s glue container env vars merging
* Add number of pods limit to k8s glue using the `max_pods_limit` argument (use `--max-pods` switch in the k8s glue example)
* Add k8s glue default `restartPolicy=Never` to template to prevent pods from restarting
* Add `--stop` switch support for dynamic gpus
* Verify `docker` command exists when running in docker mode
* Add support for terminating dockers on `sig_term` in dynamic mode
* Add stopping message on Task process termination
* Add `agent.docker_install_opencv_libs` configuration option to enable automatic opencv libs install for faster docker spin-up (default: `true`, see [here](https://github.com/allegroai/clearml-agent/blob/4f18bb7ea0600db6ee63ecb5ad0ea8d048a272ca/docs/clearml.conf#L139))
* Add support for new container base setup script feature
* Bump virtualenv dependency version (support `v>=16,<21`)
* Add support for dynamic gpus opportunistic scheduling (with min/max gpus per queue)
* Deprecate `venv_update` in configuration (replaced by the more robust `venvs_cache`)
* Add Python 3.9 to the support table
**Bug Fixes**
* Fix agent can return non-zero error code and pods will end up restarting forever ([clearml-agent GitHub Issue 56](https://github.com/allegroai/clearml-agent/issues/56))
* Fix poetry support ([clearml-agent GitHub Issue 57](https://github.com/allegroai/clearml-agent/issues/57))
* Fix cuda version from driver does not return minor version
* Fix requirements local path replace back when using cache
* Fix k8s glue
* Fix broken k8s glue docker args parsing
* Fix empty env prevents override when merging template
* Fix venv cache crash on bad symbolic links
* Fix no docker arguments provided
### ClearML Server 1.0.0
**Features**
* Add Multi-level project hierarchy - Projects can now contain sub-projects
* Add markdown editor for project overview
* Add support for multiple models per experiment
* Add Context menu batch operations ([ClearML GitHub issue 286](https://github.com/allegroai/clearml/issues/286))
* Add global search regex support ([ClearML GitHub issue 81](https://github.com/allegroai/clearml/issues/81#issuecomment-735003956))
* Add support for extended container (docker) configuration and init script
* Add special character support ("." (dot) and "$") in Hyperparameters
* Add new context menu icons
* Add deleting experiments and models through UI (ClearML GitHub issues [53](https://github.com/allegroai/clearml-server/issues/53), [61](https://github.com/allegroai/clearml-server/issues/61), and [81](https://github.com/allegroai/clearml/issues/81#issuecomment-706907718))
* Improve Project deletion: deletes project's experiments and models.
* Add experiment description access from info title
* Improve experiment table columns
* Add filter by user
* Add filters time columns
* Add filters for custom columns (metrics and hyperparameters)
* Add secondary nested sorting
* Add `worker<->queue` clickable reference in workers and queues tables
* Add presenting all metric debug samples concurrently
* Add Full project name display in project cards ([ClearML GitHub issue 81](https://github.com/allegroai/clearml/issues/81#issuecomment-823303842))
* Add option for continuing aborted tasks
**Bug Fixes**
* Fix UI storage credentials input missing for HTML artifacts
* Fix sorting custom metric columns sort treats empty fields as zeroes
* Fix UI experiments table redundant rendering on auto refresh
* Fix missing URL encoding for hyperparameters key names
* Fix UI experiment section edit opens outside viewable screen
* Fix UI missing default selection of artifacts in full screen view
* Fix UI variant name not shown in plots
* Fix UI missing tooltip for truncated column headers
* Fix UI custom columns choice does not persist per project ([ClearML GitHub issue 314](https://github.com/allegroai/clearml/issues/314))
* Fix API plot_str not returned for compressed plots
* Fix UI plots color picker consistency
* Fix API ```Tasks.reset``` marking parent id as 'deleted' in its children
* Fix UI missing queue selection on queue delete
* Fix UI debug image history slider not shown when there's only a single iteration
* Fix UI X-axis labels are being cut in plots ([ClearML GitHub issue 264](https://github.com/allegroai/clearml/issues/264))
* Fix UI scalar color choice dialog toggles between screen positions every time you open it
* Fix UI hovering on legend clears parallel coordinates graph filters ([ClearML GitHub issue 259](https://github.com/allegroai/clearml/issues/259))
* Fix UI experiment comparison 'hide identical fields' button disable doesn't work
* Fix UI auto refresh removes model ID in model panel
* Fix UI debug samples download image opens new tab instead of downloading
* Fix UI experiment/model tables: Right most column is not fixed to right edge of screen
* Fix UI 'Add experiment' modal in compare scalar comparison intertwines with comparison display
* Fix UI experiment configurations showing redundant "General" section

160
docs/release_notes/ver_1_1.md → docs/release_notes/sdk/open_source/ver_1_1.md
View File

@@ -2,32 +2,7 @@
title: Version 1.1
---
### ClearML Serving 1.1.0
:::warning Backwards Compatibility
This release is not backwards compatible - see notes below on upgrading
:::
**Breaking Changes**
* Triton engine size supports variable request size (-1)
**Features and Bug Fixes**
* Add version number of serving session task
* Triton engine support for variable request (matrix) sizes
* Triton support, fix `--aux-config` to support more configurations elements
* Hugging Face Transformer support
* `Preprocess` class as module
### ClearML Agent 1.1.2
**Bug Fixes**
- Fix the `six` conflict with the new `pathlib2` version 2.3.7 and up.
### ClearML SDK 1.1.6
### ClearML 1.1.6
**Features**
@@ -54,7 +29,7 @@ This release is not backwards compatible - see notes below on upgrading
- Fix always store session cache in `~/.clearml` (regardless of the cache folder)
- Fix `StorageManager.download_folder()` fails on Windows path
### ClearML SDK 1.1.5
### ClearML 1.1.5
**Features**
* Add support for `jsonargpraser` ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403))
@@ -121,7 +96,7 @@ This release is not backwards compatible - see notes below on upgrading
* Fix TF 2.7 support (get `logdir` on with multiple TB writers)
### ClearML SDK 1.1.4
### ClearML 1.1.4
**Bug Fixes**
@@ -132,18 +107,7 @@ This release is not backwards compatible - see notes below on upgrading
- Fix `tf.saved_model.load()` binding for `TensorFlow>=2.0`
- Fix hyperparams with `None` value converted to empty string causes inferred type to change to `str` in consecutive `Task.connect()` calls
### ClearML Agent 1.1.1
**Features and Bug Fixes**
- Add support for truncating task log file after reporting to server using `agent.truncate_task_output_files` configuration setting
- Fix PyJWT resiliency support
- Fix `--stop` checking default queue tag ([ClearML Agent GitHub issue #80](https://github.com/allegroai/clearml-agent/issues/80))
- Fix queue tag `default` does not exist and `--queue` not specified (try queue named `"default"`)
- Fix Python 3.5 compatibility
- Fix PY2.7 support for PyTorch
### ClearML SDK 1.1.3
### ClearML 1.1.3
**Features**
@@ -205,20 +169,20 @@ This release is not backwards compatible - see notes below on upgrading
- Fix broken `Dataset.finalize()`
- Fix Python 3.5 compatibility
### ClearML SDK 1.1.2
### ClearML 1.1.2
**Bug Fix**
- Fix PyJWT issue (limit dependency to `<2.2.0`)
### ClearML SDK 1.1.1
### ClearML 1.1.1
**Bug Fixes**
- Fix `Logger.report_image()` throws warning
- Fix TensorBoard `add_image()` not being reported
### ClearML SDK 1.1.0
### ClearML 1.1.0
:::info Breaking Changes
- New PipelineController v2 was introduced. New constructor is not backwards compatible!
@@ -279,113 +243,3 @@ This release is not backwards compatible - see notes below on upgrading
- Fix `task.set_parent()` fails when passing Task object
- Fix `PipelineController` skipping queued Tasks
- Remove `humanfriendly` dependency (unused)
### ClearML Agent 1.1.0
:::info Breaking Changes
ClearML Agent will no longer try to use the demo server by default (change this by setting the `CLEARML_NO_DEFAULT_SERVER=0` environment variable)
ClearML k8s glue default pod label was changed to `CLEARML=agent` (instead of `TRAINS=agent`)
:::
**Features**
- Add poetry cache into docker mapping ([ClearML Agent GitHub issue #74](https://github.com/allegroai/clearml-agent/issues/74))
- Allow rewriting SSH URLs (see [here](https://github.com/allegroai/clearml-agent/commit/9456e493ac6d6495310ee084db906f9cdca8218c)), refers to [ClearML Agent GitHub PR #72](https://github.com/allegroai/clearml-agent/pull/72) ([ClearML Agent GitHub issue #42](https://github.com/allegroai/clearml-agent/issues/42))
- Add docker environment arguments log masking support, customizable using the `agent.hide_docker_command_env_vars` configuration value (see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L172)) ([ClearML Agent GitHub issue #67](https://github.com/allegroai/clearml-agent/issues/67))
- Add support for naming docker containers using the `agent.docker_container_name_format` configuration option to set a name format (disabled by default) ([ClearML issue #412](https://github.com/allegroai/clearml/issues/412))
- k8s glue
- Remove queue name from pod name, add queue name and ID to pod labels ([ClearML Agent GitHub issue #64](https://github.com/allegroai/clearml-agent/issues/64))
- Update task `status_message` for non-responsive or hanging pods
- Support the `agent.docker_force_pull` configuration option for scheduled pods
- Add docker example for running the k8s glue as a pod in a k8s cluster
- Add `agent.ignore_requested_python_version` configuration option to ignore any requested python version (default false, see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L45))
- Add `agent.docker_internal_mounts` configuration option to control containers internal mounts (non-root containers, see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L184))
- Add support for `-r requirements.txt` in the Installed Packages section
- Add support for `CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE` environment variable to override initial server connection behavior (defaults to true, allows boolean value or an explicit number specifying the number of connect retries)
- Add support for `CLEARML_AGENT_DISABLE_SSH_MOUNT` environment variable allowing to disable the auto `.ssh` mount into the docker
- Add support for `CLEARML_AGENT_SKIP_PIP_VENV_INSTALL` environment variable to skip Python virtual env installation on execute and allow providing a custom venv binary
- Add support for `CLEARML_AGENT_VENV_CACHE_PATH` environment variable to allow overriding venv cache folder configuration
- Add support for `CLEARML_AGENT_EXTRA_DOCKER_ARGS` environment variable to allow overriding extra docker args configuration
- Add support for environment variables containing bash-style string lists using shlex
- Add printout when using ClearML key/secret from environment variables
- Increase worker keep-alive timeout to 10 minutes instead of 1 minute
- Update documentation
**Bug Fixes**
- Fix auto mount `SSH_AUTH_SOCK` into docker ([ClearML Agent GitHub issue #45](https://github.com/allegroai/clearml-agent/issues/45))
- Fix package manager configuration documentation ([ClearML Agent GitHub issue #78](https://github.com/allegroai/clearml-agent/issues/78))
- Fix support for spaces in docker arguments ([ClearML GitHub issue #358](https://github.com/allegroai/clearml/issues/358))
- Fix standalone script with pre-exiting conda venv
- Fix PyYAML v5.4, v5.4.1 versions not supported
- Fix parsing VCS links starting with `git+git@` (notice `git+git://` was already supported)
- Fix Python package with `git+git://` links or `git+ssh://` conversion
- Fix `--services-mode` if the execute agent fails when starting to run with error code 0
- Fix `--stop` with dynamic gpus
- Fix support for unicode standalone scripts, changing default `ascii` encoding to `UTF-8`
- Fix venv cache cannot reinstall package from git with http credentials
- Fix `PYTHONIOENCODING` environment variable is overwritten when already defined
- k8s glue
- Fix support for multiple k8s glue instances with pod limits
- Fix task container handling fails parsing docker image
- Fix task container is not set when using default image/arguments
- Fix task container image arguments are used when no image is specified
- Fix task container arguments not supported in when template is not provided
- Fix `agent.extra_docker_bash_script` not applied correctly
- Fix task runtime properties are removed when re-enqueuing task
- Fix error is not thrown when failing to push task to queue
### ClearML Server 1.1.1
**Bug Fixes**
- Fix experiment plots override reported color-scale ([ClearML GitHub issue #373](https://github.com/allegroai/clearml/issues/373), [Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1627310354415200))
- Fix file server `GET` response for gzipped data-files contains `Content-Encoding: gz` header, causing clients to automatically decompress the file ([ClearML GitHub issue #411](https://github.com/allegroai/clearml/issues/411))
- Fix server error when running with non-migrated v0.16 ElasticSearch data ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1627911579075600))
### ClearML Server 1.1.0
**New Features and Improvements**
- Add metric snapshot plot to project overview UI - Show an aggregated view of all project experiments value for a leading metric
- Add logical `AND` option to UI experiment table tag filter
- Add Task runtime properties to experiment INFO UI tab
- Add full screen view for any experiment result plot
- Add extended version information to UI profile page
- Stop using special characters in secrets
- Allow setting status_message in `tasks.update`
- Improve UI table view configuration persistence - User table-view configuration is saved per project:
- Visible columns
- Column order
- Column width
- Active sort
- Active filters
**Bug Fixes**
- Fix experiment details UI failure opening hyperparameter sections beginning with `#` ([ClearML Server GitHub issue #79](https://github.com/allegroai/clearml-server/issues/79))
- Fix performance issues with UI comparison of large experiments ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1621698235159800))
- Fix filtering on hyperparameters ([ClearML GitHub issue #385](https://github.com/allegroai/clearml/issues/385) [Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1626600582284700))
- Fix profile page user options toggle control area of effect
- Fix browser resizing affecting plot zoom
- Fix UI dataview filter resetting version on filter addition
- Fix UI project overview:
- Fix links in project overview not working
- Allow editing empty overview for legacy projects
- Fix Table plots using fraction of available space
- Fix scalars color assignment broken by `.` in scalar name
- Fix tasks cannot be moved between queues
- Fix UI Docker argument input - Force arguments in designated field
- Fix extraneous "tag" string prefixing Commit ID in Task execution information UI
- Fix missing 'no value' option in hyperparameters table filters
- Fix queued task is not dequeued on `tasks.stop`
- General aesthetic fixes:
- Fix input title alignment in UI clone experiment window
- Fix UI empty experiment table message alignment
- Fix UI table filter menu proportions
- Fix debug sample dropdown menu coloring
- Fix dashboard card alignment
- Fix redundant separator in plot control when maximized

65
docs/release_notes/ver_1_10.md → docs/release_notes/sdk/open_source/ver_1_10.md
View File

@@ -2,7 +2,7 @@
title: Version 1.10
---
### ClearML SDK 1.10.4
### ClearML 1.10.4
**New Features and Improvements**
* Update references to `GetAllRequest` in Task docstrings ([ClearML GitHub issue #981](https://github.com/allegroai/clearml/issues/981))
@@ -17,17 +17,12 @@ and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent
* Fix `Task.init(continue_last_task=0)` being ignored when running remotely
* Fix `urllib3` import error when using `urllib3>=2.0.0`
### ClearML Server 1.10.1
**Bug Fix**
* Fix WebApp does not connect to the server when using non-default ports ([ClearML Server GitHub issue #191](https://github.com/allegroai/clearml-server/issues/191))
### ClearML SDK 1.10.3
### ClearML 1.10.3
**Bug Fix**
* Fix mixed-up logs when using pipelines. Now pipeline controller logs won't be visible in pipeline steps' logs.
### ClearML SDK 1.10.2
### ClearML 1.10.2
**New Features and Improvements**
* Allow task filtering with `Task.query_tasks()` using `TaskTypes` ([ClearML GitHub issue #938](https://github.com/allegroai/clearml/issues/938))
@@ -42,7 +37,7 @@ and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent
* Fix Pipeline version auto-bumping
* Fix `APIClient.events.debug_images()` for latest server API version
### ClearML SDK 1.10.1
### ClearML 1.10.1
**New Features and Improvements**
* Add cast parameter to Task.get_parameter method ([ClearML GitHub issue #951](https://github.com/allegroai/clearml/issues/951))
@@ -51,7 +46,7 @@ and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent
* Fix backward compatibility issue when using datetime filtering with `APIClient().models.get_all` and `APIClient().tasks.get_all` requests
* Fix docstrings explaining tag filtering on `Model` and `Task`
### ClearML SDK 1.10.0
### ClearML 1.10.0
**Breaking Changes**
* `StorageManager.download_file()` and `StorageManager.download_folder()` from MinIO and Azure no longer include bucket name
@@ -93,53 +88,3 @@ in target local path ([ClearML GitHub issue #709](https://github.com/allegroai/c
* Fix `silence_errors` parameter in `StorageManager.get_file_size_bytes()` not working
* Fix setting `Dataset` metadata name to a string prefixed with `data_` breaks `Dataset.get()`
* Fix Pipeline fails creating tasks from multi-argument functions
### ClearML Server 1.10.0
**New Features and Improvements**
* Additional model information
* Add Lineage tab to UI model page, which displays experiment that created model and lists experiments that use the model ([ClearML GitHub issue #810](https://github.com/allegroai/clearml/issues/810))
* Add Plots and Scalars tabs to UI Model page
* Add project hierarchy view for ClearML resources (pipelines, datasets, hyper-datasets, reports) ([ClearML GitHub issue #724](https://github.com/allegroai/clearml/issues/724)):
* Flat view - all objects displayed side-by-side
* Nested view - objects grouped by containing projects.
* Add archived experiment inclusion in UI experiment comparison ([ClearML GitHub issue #795](https://github.com/allegroai/clearml/issues/795))
* Update ElasticSearch image to 7.17.7 ([ClearML Server GitHub issue #183](https://github.com/allegroai/clearml-server/issues/183))
* Add reported plot support for multiple color formats
* Add running tasks to UI project overview metric snapshot
* Improve UI Reports:
* Add option to upload and embed images from PC
* Add option to embed experiment comparison parallel coordinates plots and full screen debug sample
* Add UI indicator to hidden resources when "Show hidden projects" is enabled
* Add Experiment comparison UI parallel coordinate plots full screen view
**Bug Fixes**
* Fix UI scalar plot smoothing is biased towards initial value ([ClearML Server GitHub issue #180](https://github.com/allegroai/clearml-server/issues/180))
* Fix UI experiment plot comparison not merging plots
* Fix UI reports search bar doesn't search report descriptions
* Fix UI reports search bar does not close after result is selected
* Fix UI user preferences don't persist in duplicate tab or after logout/login
* Fix UI pipeline step names are cropped with no available tooltips
* Fix refreshing UI console output scrolls away from latest entries
* Fix UI experiment artifact list not updated upon artifact deletion
* Fix UI experiment comparison Hyperparameters tab scroll erratic behavior in Firefox
* Fix UI console log partially showing pipeline log entries
* Fix can't move UI Report to root project
* Fix queue isn't selected when name is clicked in the UI enqueue modal
* Fix turning off filter in UI experiment table does not persist
* Fix UI plot lines appearing in wrong places in plot
* Fix newly created dataset tags missing from dataset filter tag list
* Fix unclear text in UI plots downloaded as PDF/PNG
* Fix UI global search incorrectly returns 0 Dataset/Reports count
* Fix long named reports flash upon hover
* Fix disappearing UI scalar graph when "Group by" is set to "None"
* Fix UI experiment plots horizontal axis units change to "Iterations" when refreshed
* Fix UI scalar graphs do not update when horizontal axis units are changed
* Fix tooltip does not appear when hovering over name in UI breadcrumbs
* Fix UI experiment scalar smoothing does not persist in plot fullscreen
* Fix newly created report tags missing from report filter tag list
* Fix reports names can contain a "/"
* Fix My work / Team work filter selection does not persist upon app refresh

42
docs/release_notes/sdk/open_source/ver_1_11.md Normal file
View File

@@ -0,0 +1,42 @@
---
title: Version 1.11
---
### ClearML 1.11.1
**New Features and Improvements**
* Support `lightning>=2.0` ([ClearML GitHub issue #1032](https://github.com/allegroai/clearml/issues/1032))
* Add `clearml-init` support for standard HTTP and HTTPS ports in webapp in conjunction with non-default api/files server ports
([ClearML Server GitHub issue #198](https://github.com/allegroai/clearml-server/issues/198))
* Improve docstrings
**Bug Fixes**
* Fix remote execution does not log configuration values from modified `jsonargparse` configuration files
* Fix `ValueError` when setting task properties at the start of a pipeline component
* Fix OS exceptions on passing `Path` where `str` expected
* Fix infinite recursion when remotely updating a `dict` added to task with `Task.connect()`
* Fix `Task.connect()` race condition overwriting task description with connected input model
### ClearML 1.11.0
**New Features and Improvements**
* Add offline mode for ClearML Datasets ([ClearML GitHub issue #854](https://github.com/allegroai/clearml/issues/854))
* Improve performance of `Dataset.add_external_files()` ([ClearML GitHub PR #962](https://github.com/allegroai/clearml/pull/962))
* Add `Task.launch_multi_node()` for distributed experiment execution
* Add `Task.get_all_reported_scalars()` to fetch all available scalar data
* Add `Task.remove_input_models()` to disassociate input models from a task
* Improve `Dataset.list_datasets()` to include dataset version in the result
* Add support for PyTorch Lightning CLI >=2.0
* Add support for automatic Dataset content preview for JSON files
**Bug Fixes**
* Fix multiple local parallel experiments conflicting on storage access validation ([ClearML GitHub issue #897](https://github.com/allegroai/clearml/issues/897))
* Fix code running with `joblib` hangs ([ClearML GitHub issue #1009](https://github.com/allegroai/clearml/issues/1009))
* Fix comment for storage scheme `gs://` ([ClearML GitHub issue #986](https://github.com/allegroai/clearml/issues/986))
* Fix typo "Categories per column" ([ClearML GitHub PR #1021](https://github.com/allegroai/clearml/pull/1021))
* Fix downloading artifacts from GCP when the download URI includes unsafe characters
* Fix pipeline step cache is not cleared when the only change to the step is that the docker image changed
* Fix `Task.report_text()` sometimes reporting to an incorrect task when multiple tasks run simultaneously
* Fix` Task.set_offline(offline_mode=False)` raising an `UnboundLocalError`
* Fix `scikit-image` package (`skimage`) is sometimes not detected as a dependency
* Fix `PipelineDecorator` sometimes causing a race condition when starting a remote execution

49
docs/release_notes/sdk/open_source/ver_1_12.md Normal file
View File

@@ -0,0 +1,49 @@
---
title: Version 1.12
---
### ClearML 1.12.2
**New Features and Improvements**
* Add `always_create_from_code` argument to `PipelineController` (default `True`) to allow remote execution to create
the Pipeline DAG at runtime (as opposed to adhering to the existing structure stored on the task when running locally)
**Bug Fixes**
* Fix using `OutputModel` reporting and other methods fail if `OutputModel.update_weights()` is not called before that ([ClearML GitHub issue #1078](https://github.com/allegroai/clearml/issues/1078))
* Fix hash for Pandas objects artifacts
* Fix Hydra/OmegaConf `--overrides` support
* Fix Pipeline metrics not forwarded to pipeline controller at the end of a pipeline step
### ClearML 1.12.1
**New Features and Improvements**
* Raise error if dataset name is empty ([ClearML GitHub PR #1083](https://github.com/allegroai/clearml/pull/1083))
* Add support for `spawn` start method for Python `multiprocess`. This should help circumvent issues like [this](https://github.com/python/cpython/issues/84559).
**Bug Fixes**
* Fix S3 uploads fail with `LazyEvalWrapper` type error ([ClearML GitHub issue #1081](https://github.com/allegroai/clearml/issues/1081))
### ClearML 1.12.0
:::info BREAKING CHANGES
As a result of fixing [issue #1051](https://github.com/allegroai/clearml/issues/1051) and similar issues with retrieving
task data/models/artifacts/samples stored in GCS, we introduced some breaking changes. To access resources registered to
GCS from ClearML SDK versions 1.11.0 to 1.11.1, see
the instructions [here](https://github.com/allegroai/clearml/tree/master/docs/errata_breaking_change_gcs_sdk_1_11_x.md).
:::
**New Features and Improvements**
* Add `include_archive` parameter to `Dataset.list_datasets()`: include archived datasets in list ([ClearML GitHub issue #1067](https://github.com/allegroai/clearml/issues/1067))
* Add support to specify the multipart chunk size and threshold using the `aws.boto3.multipart_chunksize` and
`aws.boto3.multipart_threshold` configuration options in the clearml.conf ([ClearML GitHub issue #1058](https://github.com/allegroai/clearml/issues/1058))
* Add `PipelineController.get_pipeline()` for retrieving previously run pipelines.
**Bug Fixes**
* Fix AWS driver issues ([ClearML GitHub PR #1000](https://github.com/allegroai/clearml/pull/1000)):
* Fix credential authentication failure when attempting to use token
* Fix instantiation within VPC without AvailabilityZones
* Fix `continue_last_task=0` is ignored in pipelines run with `retry_on_failure` ([ClearML GitHub issue #1054](https://github.com/allegroai/clearml/issues/1054))
* Fix `Task.connect_configuration()` doesn't handle dictionaries with special characters
* Fix pipeline steps created with `PipelineDecorator` aren't cached
* Fix `Task.get_by_name()` doesn't return the most recent task when multiple tasks have same name
* Fix Boto3 certificate verification ignores custom self-signed certificates

61
docs/release_notes/ver_1_13.md → docs/release_notes/sdk/open_source/ver_1_13.md
View File

@@ -2,62 +2,7 @@
title: Version 1.13
---
### ClearML Server 1.13.1
**Bug Fixes**
* Fix GCS credentials keep popping up
* Fix long name overflows onto show/hide button in scalar metric list
### ClearML Server 1.13.0
**New Features and Improvements**
* Improve UI comparison
* Add visual comparison of single value scalars ([ClearML Server GitHub issue #202](https://github.com/allegroai/clearml-server/issues/202))
* Scalar value comparison: metrics are presented in a single table ([ClearML GitHub issue #760](https://github.com/allegroai/clearml/issues/760))
* Add global experiment legend: Experiment name, tags and color ([ClearML GitHub issue #81](https://github.com/allegroai/clearml/issues/81))
* Clickable URLs/object IDs in experiment details comparison ([ClearML GitHub issue #1073](https://github.com/allegroai/clearml/issues/1073))
* UI Search improvements
* Dashboard search results ordered by update time
* Top bar search query saved in URL
* Improve UI experiment debug sample iteration fetching speed ([ClearML Server GitHub issue #170](https://github.com/allegroai/clearml-server/issues/170))
* Add UI cloud credentials support for GCS signed URLs with HMAC authentication
* Add Experiment plot UI support for displaying subplots
**Bug Fixes**
* Fix UI single-value scalars table does not display all single-value scalars ([ClearML GitHub issue #1111](https://github.com/allegroai/clearml/issues/1111))
* Fix UI pipeline step duration does not update during runtime ([ClearML GitHub issue #1090](https://github.com/allegroai/clearml/issues/1090))
* Fix UI experiment comparison parallel coordinates plot crops the metric and parameter names ([ClearML GitHub issue #1111](https://github.com/allegroai/clearml/issues/1111))
* Fix hidden experiments appear in UI All Experiments search results ([ClearML GitHub issue #1139](https://github.com/allegroai/clearml/issues/1139))
* Fix disabled graphs in UI scalars reappear in full-screen mode
* Fix UI auto-refresh sometimes causes many error message to be raised
* Fix "All" tag filter not working in UI object pages
* Fix tag filter in UI dataset page does not persist on refresh
* Fix scalars and plots are not displayed in UI model page
* Fix UI pipeline step status does not update
* Fix UI experiment comparison plots don't display all experiments in legend
* Fix UI Model Lineage tab sometimes doesn't display models
* Fix some plots present partial data in UI experiment comparison plots
* Fix clicking URL of deleted UI report navigates to empty page
* Fix applying UI tag in bulk raises an error
* Fix task IDs appear multiple times in UI task comparison plot legends
* Fix UI project filter in All Experiments table takes a long time load
* Fix custom columns in UI tables do not persist on refresh
* Fix UI report card missing creating user's name
* Fix new tag is not displayed in "Add Tag" table action menu until page refresh
* Fix clicking on worker name in Orchestration page's Queue info panel does not navigate to the worker's info panel
* Fix clicking on older empty UI datasets redirects to 404 page
* Fix experiment names don't appear in the plot legend in UI experiment comparison scalar plots grouped by metric
* Fix task ID does not appear in UI task comparison plots of tasks with the same name
* Fix UI scalar comparison's Horizontal Axis, Smoothing, and Group by selections do not persist upon experiment selection modification
* Fix UI scalars sometimes missing plots in Chrome
* Fix values can't be input into the UI scalar smoothing field
* Fix UI experiment configuration is sometimes not editable
* Fix UI project name tooltips don't appear for projects with long names
* Fix debug samples appear in UI experiment only after refresh
* Fix cropped names in tag list in "Add Tag" action in UI table action menu
* Fix UI experiment configuration tooltip obscure the "Edit" button
### ClearML SDK v1.13.2
### ClearML v1.13.2
**New Features and Improvements**
@@ -93,13 +38,13 @@ title: Version 1.13
* Fix `pyjwt` requirements for Python < 2.9
* Fix pipeline examples
### ClearML SDK 1.13.1
### ClearML 1.13.1
**Bug Fixes**
* Fix Hydra overrides integration ([ClearML GitHub issue #1120](https://github.com/allegroai/clearml/issues/1120))
* Fix GPU memory used reports 0 when memory can not be queried per process (edge case)
### ClearML SDK 1.13.0
### ClearML 1.13.0
**New Features**
* Add support for recursive list, dict, and tuple ref parsing for `PipelineController.add step()` parameter overrides ([ClearML GitHub issue #1089](https://github.com/allegroai/clearml/issues/1089))

51
docs/release_notes/ver_1_14.md → docs/release_notes/sdk/open_source/ver_1_14.md
View File

@@ -2,7 +2,7 @@
title: Version 1.14
---
### ClearML SDK 1.14.4
### ClearML 1.14.4
**Bug Fixes**
* Fix AWS S3 configuration `secure` property not forwarded in `StorageHelper` ([ClearML GitHub PR #1205](https://github.com/allegroai/clearml/pull/1205))
@@ -10,28 +10,14 @@ title: Version 1.14
* Fix setting `output_uri` when an AWS S3 config section with `secure=false` exists raises an error
* Fix `CacheManager` attempts to clear cache when file limit has not been reached resulting in slow unnecessary checks
### ClearML Server 1.14.1
**New Features and Improvements**
* Add support for serving UI from a subpath
* Change default cookie `SameSite` setting to `Lax`
**Bug Fixes**
* Fix clicking on UI Metric Snapshot datapoint does not navigate to experiment ([ClearML Server GitHub issue #228](https://github.com/allegroai/clearml-server/issues/228))
* Fix some APIs not sanitizing query parameters
* Fix downloading UI experiment console log sometimes returns an older file
* Fix UI experiment plot comparison displays duplicates of image plots
* Fix missing experiment labels in UI experiment plot comparison
* Fix starting time not updating for some tasks
### ClearML SDK 1.14.3
### ClearML 1.14.3
**New Features and Bug Fixes**
* Add hash check for pickle files (resolves [CVE-2024-24590](https://github.com/advisories/GHSA-cpcw-9h9m-wqw9))
* Fix `clearml-task` relative paths calculation when `--cwd`, `--folder`, and `--script` are provided ([ClearML GitHub issue #1160](https://github.com/allegroai/clearml/issues/1160))
* Fix regression causing an error when `secure` is provided in AWS S3 bucket settings
### ClearML SDK 1.14.2
### ClearML 1.14.2
**New Features and Improvements**
* Report upload/download with `tqdm` if installed
@@ -53,12 +39,12 @@ title: Version 1.14
step has been aborted
* Fix HPO crashes when its corresponding task is initialized with `continue_last_task=True`
### ClearML SDK 1.14.1
### ClearML 1.14.1
**Bug Fixes**
* Fix created pipelines do not appear in the UI pipelines section
### ClearML SDK 1.14.0
### ClearML 1.14.0
**New Features and Improvements**
* Add support for decorated pipeline steps ([ClearML GitHub issue #1154](https://github.com/allegroai/clearml/issues/1154))
* Improve `torch.distributed` behavior and auto-detect Huggingface accelerate distributed execution
@@ -85,30 +71,3 @@ and can be imported globally instead
* Fix misleading error raised when using the SDK with no configuration file or an invalid configuration file
* Fix process sometimes hangs issue by improving exit and exception handlers, unregistering signal handling in child
processes except for the `BackgroundMonitor`
### ClearML Server 1.14.0
**New Features and Improvements**
* New UI experiment table comparative view: compare plots and scalars of all selected experiments
* Add UI experiment display of original python requirements along with actual packages used ([ClearML GitHub issue #793](https://github.com/allegroai/clearml/issues/793))
* Add UI project metric snapshot support for multiple metrics
* Add compressed UI experiment table info panel mode displaying only experiment name and status
* Add "x unified" hover mode to UI plots
* Add experiment hyperparameter UI section dataset IDs link to datasets page
* Make all URLs clickable in UI experiment artifacts
* Redesign WebApp tabs: tabs now appear in UI header
* Add option to force setting original task as clone's parent when cloning a task
**Bug Fixes**
* Fix CVE-2023-45133 Web App vulnerability
* Fix "select all" in UI dataset archive displays count of all archived and unarchived experiments
* Fix hyperparameter keys with dots in their names displayed incorrectly in UI experiment table and task info
* Fix UI experiment scalar comparison raising error
* Fix "Project" column redundantly appears inside a specific project's model table
* Fix UI models with many labels slowing down web app
* Fix simple UI plots (scatter, bar, etc.) in experiment comparisons not merging by experiment
* Fix UI pipeline details panel does not display information
* Fix UI plot "Show/hide" legend button sometimes disappears
* Fix can't select root project in UI report creation modal when workspace has no projects
* Fix "load more" button displayed unnecessarily in UI object tables
* Fix archive action in UI dataset table archives unselected datasets

35
docs/release_notes/sdk/open_source/ver_1_15.md Normal file
View File

@@ -0,0 +1,35 @@
---
title: Version 1.15
---
### ClearML 1.15.1
**Bug Fixes**
* Fix autoscaler should recheck that the worker is still idle before shutting it down ([ClearML GitHub issue #1202](https://github.com/allegroai/clearml/issues/1202))
* Fix resource monitor fails to get GPU stats in some edge cases
### ClearML 1.15.0
**New Features**
* Add draft option to pipeline steps ([ClearML GitHub PR #1226](https://github.com/allegroai/clearml/pull/1226))
* Add support for custom working directory for pipelines ([ClearML GitHub issue #1194](https://github.com/allegroai/clearml/issues/1194))
* Add `Task.get_requirements()` method that returns the task's requirements
* Allow controlling the number of threads used by `StorageManager.download_folder()` using the `max_workers` argument
* Update examples dependencies
* Improve auto populate in `Task.init()`
**Bug Fixes**
* Fix UTF-8 script code encoding issue ([ClearML GitHub issue #1207](https://github.com/allegroai/clearml/issues/1207))
* Fix Colab docs ([ClearML GitHub issue #1204](https://github.com/allegroai/clearml/issues/1204))
* Fix metrics reporting with `OutputModel` while in offline mode results in an error ([ClearML GitHub issue #1172](https://github.com/allegroai/clearml/issues/1172))
* Fix task running in Google Colab doesn't properly get the notebook diff ([ClearML GitHub issue #1204](https://github.com/allegroai/clearml/issues/1204))
* Fix hydra binds break in offline mode ([ClearML GitHub issue #1215](https://github.com/allegroai/clearml/issues/1215))
* Fix HPO crashes when optimizing for single value scalars ([ClearML GitHub issue #1221](https://github.com/allegroai/clearml/issues/1221))
* Fix GPU info such as `gpu_memory` and `gpu_type` is not being collected in some cases
* Fix `clearml-data` CLI tool moves non-dataset tasks to a `.dataset` project when the respective task is not a dataset
* Fix pandas `DataFrame` artifacts with hierarchical indices get mangled by the CSV round-trip
* Fix `urllib3` sends deprecation warning when setting `ssl_version`
* Fix `Task.connect(dict)` return value is not dict-compatible
* Fix `jsonargparse` sub-command config parsing
* Fix Lightning integration crashes when a config entry contains `.` in its name
* Fix Python 3.5 compatibility

44
docs/release_notes/ver_1_16.md → docs/release_notes/sdk/open_source/ver_1_16.md
View File

@@ -2,7 +2,7 @@
title: Version 1.16
---
### ClearML SDK 1.16.4
### ClearML 1.16.4
**New Features**
* Add custom task binary support to `clearml-task` and `CreateAndPopulate` (allows bash script execution, requires agent version `>=1.9.0`)
@@ -13,12 +13,7 @@ configuration option)
**Bug Fix**
* Fix valid model URL might be overridden by an invalid one in case the upload failed
### ClearML Server 1.16.2
**Bug Fix**
* Fix no graphs are shown in Workers and Queues screens
### ClearML SDK 1.16.3
### ClearML 1.16.3
**New Features and Improvements**
* Add `--tags` option to `clearml-task` ([ClearML GitHub issue #1284](https://github.com/allegroai/clearml/issues/1284))
@@ -45,36 +40,7 @@ configuration option)
* Fix `jsonschema` / `referencing` import to include `TypeError` protection
* Fix Dataset offline behavior
### ClearML Server 1.16.1
**Bug Fix**
* Fix user credentials reset on API server restart ([ClearML Server GitHub issue #250](https://github.com/allegroai/clearml-server/issues/250))
### ClearML Server 1.16.0
**New Features and Improvements**
* New UI "new experiment creation"
* ClearML File Server now uses authentication by default. This can be turned off using the `fileserver.auth.enabled: false`
configuration setting
* Add confirmation pop-up when running experiment is archived
**Bug Fixes**
* Fix model plots not displayed in UI model plot tab
* Fix UI experiment Debug Samples tab freezes when a lot of debug samples are reported in the experiment
* Fix UI scalar “Export to CSV” button returns an empty CSV file
* Fix large number of tags in UI tables' tag filter causes UI to crash
* Fix searching in UI object tables' tag filter causes UI to crash
* Fix consistent plot colors not maintained when embedded in UI Reports ([ClearML GitHub issue #1239](https://github.com/allegroai/clearml/issues/1239))
* Fix UI plot legends and tooltips wrapping ([ClearML Server GitHub issue #241](https://github.com/allegroai/clearml-server/issues/241))
* Fix multiple parameters can be selected for plot axes in scatter plot view in UI experiment comparison
* Fix image names change in UI task debug sample comparison when a different metric is selected
* Fix numeric range filters of UI table columns not working
* Fix queue experiment list is not updated in the UI Orchestration's Queues page
* Fix My work/Team work persists across UI modules (i.e. projects, datasets, pipelines, etc.)
* Fix UI single-scalar plots are not ordered alphabetically by scalar name
* Fix limit on file uploads to the fileserver
### ClearML SDK 1.16.2
### ClearML 1.16.2
**New Features and Improvements**
* Make dataset preview optional ([ClearML GitHub issue #1227](https://github.com/allegroai/clearml/issues/1227))
@@ -89,13 +55,13 @@ and [#1233](https://github.com/allegroai/clearml/issues/1233))
* Fix no need to recreate reporter if forking and reporting in subprocess
* Fix forked detection mechanism
### ClearML SDK 1.16.1
### ClearML 1.16.1
**Bug Fixes**
* Fix pipeline breaks when `continue_on_abort` is set to `true`
* Fix Pycharm Plugin Windows/Linux interoperability
### ClearML SDK 1.16.0
### ClearML 1.16.0
**New Features**
* Add additional warning instructing on how to install in case we failed detecting a Jupyter notebook with an import error

56
docs/release_notes/sdk/open_source/ver_1_2.md Normal file
View File

@@ -0,0 +1,56 @@
---
title: Version 1.2
---
### ClearML 1.2.1
**Bug Fixes**
- Fix HTTP download fails constructing URL ([ClearML GitHub issue #593](https://github.com/allegroai/clearml/issues/593))
### ClearML 1.2.0
**Features**
- Add fastai v2 support ([ClearML GitHub PR #571](https://github.com/allegroai/clearml/pull/571))
- Add catboost support ([ClearML GitHub PR #542](https://github.com/allegroai/clearml/pull/542))
- Add Python Fire support ([ClearML GitHub PR #550](https://github.com/allegroai/clearml/pull/550))
- Add new Azure Storage driver support ([ClearML GitHub PR #548](https://github.com/allegroai/clearml/pull/548))
- Add requirements file support in `Task.add_requirements` ([ClearML GitHub PR #575](https://github.com/allegroai/clearml/pull/575))
- Allow overriding `auto_delete_file` in `Task.update_output_model()` ([ClearML GitHub issue #554](https://github.com/allegroai/clearml/issues/554))
- Support `artifact_object` empty string
- Add `skip_zero_size_check` to `StorageManager.download_folder()`
- Add support for extra HTTP retry codes (see [here](https://github.com/allegroai/clearml/blob/2c916181b90c784fe0bd267cd67ea915e53e36e4/clearml/backend_api/config/default/api.conf#L29) or use `CLEARML_API_EXTRA_RETRY_CODES`)
- Add `Task.get_parameters()` cast back to original type
- Add callback support to `Task.delete()`
- Add autoscaler CPU-only support
- Add AWS autoscaler IAM instance profile support
- Update examples
- Edit HTML reporting examples ([ClearML GitHub PR #546](https://github.com/allegroai/clearml/pull/546))
- Add model reporting examples ([ClearML GitHub PR #553](https://github.com/allegroai/clearml/pull/553))
**Bug Fixes**
- Fix `nargs="?"` without type does not properly cast the default value ([ClearML GitHub issue #531](https://github.com/allegroai/clearml/issues/531))
- Fix using invalid configurations ([ClearML GitHub issue #544](https://github.com/allegroai/clearml/issues/544))
- Fix extra_layout not passed to report_matrix ([ClearML GitHub issue #559](https://github.com/allegroai/clearml/issues/559))
- Fix group arguments in click ([ClearML GitHub PR #561](https://github.com/allegroai/clearml/pull/561))
- Fix no warning when failing to patch argparse ([ClearML GitHub PR #576](https://github.com/allegroai/clearml/pull/576))
- Fix crash in `Dataset.upload()` when there is nothing to upload ([ClearML GitHub PR #579](https://github.com/allegroai/clearml/pull/579))
- Fix requirements, refactor and reformat examples ([ClearML GitHub PRs #567](https://github.com/allegroai/clearml/pull/567), [#573](https://github.com/allegroai/clearml/pull/573), [#582](https://github.com/allegroai/clearml/pull/582))
- Auto-scaler
- Change confusing log message
- Fix AWS tags support
- Fix instance startup script fails on any command (should only fail on the agent failing to launch)
- Fix spin down stuck machine, ignore unknown stale workers
- Fix pandas object passed as `Task.upload_artifact()` preview object
- Fix incorrect timeout used for stale workers
- Fix `clearml-task` calls `Task.init()` in the wrong place when a single local file is used
- Fix ArgumentParser `SUPPRESS` as default should be resolved at remote execution in the same way (i.e. empty string equals `SUPPRESS`)
- Upgrade six version (in case `pathlib2>2.3.7` is installed)
- Fix connected object base class members are not used
- Fix `clearml-init` changing web host after pasting full credentials
- Fix fileserver upload does not support path in URL
- Fix crash on semaphore acquire error
- Fix docs and docstrings ([ClearML GitHub PRs #558](https://github.com/allegroai/clearml/pull/558), [#560](https://github.com/allegroai/clearml/pull/560))

60
docs/release_notes/sdk/open_source/ver_1_3.md Normal file
View File

@@ -0,0 +1,60 @@
---
title: Version 1.3
---
### ClearML 1.3.2
**New Features and Improvements**
* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/allegroai/clearml/issues/604))
* Add reserved OS environments warning
* Add git credentials to colab example ([ClearML GitHub PR #621](https://github.com/allegroai/clearml/pull/621))
* Add jsonargparse support ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403))
* Update autokeras example
**Bug Fixes**
* Fix sub-project separators are incorrectly quoted in generated URLs ([ClearML GitHub PR #584](https://github.com/allegroai/clearml/pull/584))
* Revert Optuna deprecation fix ([ClearML GitHub PR #613](https://github.com/allegroai/clearml/pull/613))
* Fix HPO randomly aborts running tasks before the time limit
* Fix cloud driver overwrites `agent.extra_docker_arguments`
* Fix Pipeline Controller auto-magic framework connect
* Fix unused scroll is not cleared in `Task.get_reported_plots()`
### ClearML 1.3.1
**New Features and Improvements**
* Add Python 3.10 support
**Bug Fixes**
* Update Slack SDK requirement ([ClearML GitHub issue #597](https://github.com/allegroai/clearml/issues/597))
* Fix fork after `task.close()` is called ([ClearML GitHub issue #605](https://github.com/allegroai/clearml/issues/605))
* Fix Azure storage upload ([ClearML GitHub issue #598](https://github.com/allegroai/clearml/issues/598))
* Fix offline mode crash
* Fix task delete response not checked
* Fix pipeline controller kwargs with list
* Fix `PipelineDecorator.debug_pipeline()`
* Fix PipelineDecorator example
* Fix Python 3.10 issues
* Fix handling of legacy fileserver (files.community.clear.ml)
* Fix cloud driver may use None credentials
* Fix APIClient worker raises exception when accessing .name attribute
* Fix minimum/default API version setting
### ClearML 1.3.0
**New Features and Improvements**
* Add new pipeline visualization support (requires ClearML Server v1.3)
* Support IAM Instance Profile in AWS auto-scaler
* Remove old server API versions support (pre-ClearML Server)
* Restructure FastAI examples
**Bug Fixes**
* Fix failed catboost bind on GPU ([ClearML GitHub issue #592](https://github.com/allegroai/clearml/issues/592))
* Fix Optuna `n_jobs` deprecation warning
* Fix invalid method called on `delete()` error

61
docs/release_notes/sdk/open_source/ver_1_4.md Normal file
View File

@@ -0,0 +1,61 @@
---
title: Version 1.4
---
### ClearML 1.4.1
**Bug Fixes**
* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674))
### ClearML 1.4.0
**New Features**
* Add OpenMMLab example ([ClearML GitHub issue #654](https://github.com/allegroai/clearml/issues/654))
* Add support for saving artifacts with different formats ([ClearML GitHub issue #634](https://github.com/allegroai/clearml/issues/634))
* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/allegroai/clearml/issues/604))
* Support more than 500 results in `Task.get_tasks()` using the `fetch_only_first_page` argument ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612))
* Support links in `clearml-data` ([ClearML GitHub issue #585](https://github.com/allegroai/clearml/issues/585))
* Support deferred task initialization using `Task.init()` argument `deferred_init` (beta feature)
* Support resuming experiments when importing an Offline session
* Add `--import-offline-session` command line option to `clearml-task`
* Support automatically logging Tensorboard Hparams
* Add wildcard support for model auto-logging, see [`Task.init()`](../../../references/sdk/task.md#taskinit)
* Add support for Lightning CLI
* Support None values in `Task.connect()`
* Add `Model.project` getter/setter
* Add support for Task progress indication
* Datasets
* Improve Dataset version table
* Add warning to Dataset creation on current Task
* Examples and documentation
* Add manual seaborn logging example ([ClearML GitHub PR #628](https://github.com/allegroai/clearml/pull/628))
* Change package author
* Change pipeline example to run locally ([ClearML GitHub PR #642](https://github.com/allegroai/clearml/pull/642))
* Update PyTorch Lightning example for `pytorch-lightning>=v1.6.0` ([ClearML GitHub PR #650](https://github.com/allegroai/clearml/pull/650))
**Bug Fixes**
* Fix Keras model config serialization in `PatchKerasModelIO` ([ClearML GitHub issue #614](https://github.com/allegroai/clearml/issues/614))
* Fix `task.get_parameters_as_dict(cast=True)` casts `False` to `True` ([ClearML GitHub PR #622](https://github.com/allegroai/clearml/pull/622))
* Fix Fire integration is not compatible with typing library ([ClearML GitHub issue #610](https://github.com/allegroai/clearml/issues/610))
* Fix remote execution with `argparse` mutually exclusive groups raises "required" error even when no argument is required
* Fix Hydra tasks never fail and are only set to completed (fix handling return code)
* Fix `clearml-data` wildcard support
* Fix HPO randomly aborts running tasks before the time limit
* Fix matplotlib capture
* Fix issue with accessing images in projects containing `/`
* AutoScaler
* Fix resource name with a prefix matching a resource type may cause the auto-scaler to avoid spinning down idle instances
* Fix Idle workers should contain resource name and not instance type
* Fix backwards compatibility issue when using `abstractmethod`
* Matplotlib
* Fix uploading 3D plots with matplotlib plt shows 2D plot on task results page
* Fix wrong Histogram plotting using when matplotlib
* Fix PyTorch `ScriptModule` autobind
* Fix PyTorch auto-magic logging TorchScript models
* Fix forked process will not call `_at_exit` and flush all outstanding reports
* Fix matplotlib to plotly conversion fails on subplots (convert as image if figure has subplots)
* Fix Windows sub process might end up waiting forever for uploads to finish if subprocess is very shot lived
* Fix `StorageManager.get_local_copy()` returning None for a valid path in Windows
* Fix Jupyter notebook cannot be detected
* Fix PipelineController does not change node Task name, only pipeline step name
* Fix `Task.query_tasks()` specifying page size or page number

31
docs/release_notes/sdk/open_source/ver_1_5.md Normal file
View File

@@ -0,0 +1,31 @@
---
title: Version 1.5
---
### ClearML 1.5.0
**New Features and Improvements**
* Add support for single value metric reporting ClearML GitHub issue ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400))
* Add support for specifying parameter sections in `PipelineDecorator` ([ClearML GitHub issue #629](https://github.com/allegroai/clearml/issues/629))
* Add support for parallel uploads and downloads (upload / download and zip / unzip of artifacts)
* Add support for specifying execution details (repository, branch, commit, packages, image) in `PipelineDecorator`
* Bump PyJWT version due to "*Key confusion through non-blocklisted public key formats*" vulnerability
* Add support for AWS Session Token (using boto3's `aws_session_token` argument)
**Bug Fixes**
* Fix `Task.get_projects()` retrieves only the first 500 results ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612))
* Fix failure to delete artifacts stored in Azure ([ClearML GitHub issue #660](https://github.com/allegroai/clearml/issues/660))
* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674))
* Fix number of unpacked values when syncing a dataset ([ClearML GitHub issue #682](https://github.com/allegroai/clearml/issues/682))
* Fix FastAI DeprecationWarning ([ClearML GitHub PR #683](https://github.com/allegroai/clearml/issues/683))
* Fix `StorageManager.download_folder()` crash
* Fix pipelines can't handle `None` return value
* Fix pre-existing pipeline raises an exception
* Fix deprecation warning in the `image_reporting` example
* Fix patches are kept binded after `Task.close()` is called
* Fix running pipeline code remotely without first running it locally (i.e. no configuration on the Task)
* Fix local task execution with empty working directory
* Fix permission check fails when using local storage folder that does not exist
* Fix pipeline `add_function_step` breaks in remote execution
* Fix wrong mimetype used for any file or folder uploaded to S3 using `StorageManager`
* Add missing default `default_cache_manager_size` in configuration files

96
docs/release_notes/sdk/open_source/ver_1_6.md Normal file
View File

@@ -0,0 +1,96 @@
---
title: Version 1.6
---
### ClearML 1.6.4
**Bug Fix**
* Fix `APIClient` fails when calling `get_all` endpoints with API 2.20 (affects CLI tools such as `clearml-session`)
### ClearML 1.6.3
**New Features and Improvements**
* Add option to specify an endpoint URL when creating S3 resource service ([ClearML GitHub issue #673](https://github.com/allegroai/clearml/issues/673))
* Add support for providing `ExtraArgs` to boto3 when uploading files using the `sdk.aws.s3.extra_args` configuration option
* Add support for Server API 2.20
* Add `Task.get_num_enqueued_tasks()` to get the number of tasks enqueued in a specific queue
* Add support for updating model metadata using `Model.set_metadata()`, `Model.get_metadata()`, `Model.get_all_metadata()`,
`Model.get_all_metadata_casted()`, and `Model.set_all_metadata()`
* Add `Task.get_reported_single_value()`
* Add a retry mechanism for models and artifacts upload
* Pipelines with empty configuration takes it from code
* Add support for running pipeline steps on preemptible instances
* Datasets
* Add description to Datasets
* Add wild-card support in `clearml-data`
**Bug Fixes**
* Fix dataset download ([ClearML GitHub issue #713](https://github.com/allegroai/clearml/issues/713))
* Fix lock is not released after dataset cache is downloaded ([ClearML GitHub issue #671](https://github.com/allegroai/clearml/issues/671))
* Fix deadlock might occur when using process pool large number processes ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674))
* Fix 'series' not appearing on UI when using `logger.report_table()` ([ClearML GitHub issue #684](https://github.com/allegroai/clearml/issues/684))
* Fix `Task.init()` docstring to include behavior when executing remotely ([ClearML GitHub PR #737](https://github.com/allegroai/clearml/pull/737))
* Fix `KeyError` when running remotely and no params were passed to click ([ClearML Agent GitHub issue #111](https://github.com/allegroai/clearml-agent/issues/111))
* Fix full path is stored when uploading a single artifact file
* Fix passing non-alphanumeric filename in `sdk.development.detect_with_pip_freeze`
* Fix Python 3.6 and 3.10 support
* Fix mimetype cannot be `None` when uploading to S3
* Pipelines
* Fix pipeline DAG
* Add support for pipelines with spot instances
* Fix pipeline proxy object is always resolved in main pipeline logic
* Fix pipeline steps with empty configuration should try and take it from code
* Fix wait for jobs based on local/remote pool frequency
* Fix `UniformIntegerParameterRange.to_list()` ignores min value
* Fix pipeline component returning a list of length 1
* Datasets
* Fix `Dataset.get()` does not respect `auto_create`
* Fix getting datasets fails with new ClearML Server v1.6
* Fix datasets can't be queried by project/name alone
* Fix adding child dataset to older parent dataset without stats
* Fix error when connecting an input model
* Fix deadlocks, including:
* Change thread Event/Lock to a process fork safe threading objects
* Use file lock instead of process lock to avoid future deadlocks since python process lock is not process safe
(killing a process holding a lock will Not release the lock)
* Fix `StorageManager.list()` on a local Windows path
* Fix model not created in the current project
* Fix `keras_tuner_cifar` example raises DeprecationWarning and ValueError
### ClearML 1.6.2
**Bug Fix**
* Fix format string construction sometimes causing delayed evaluation errors ([ClearML GitHub issue #706](https://github.com/allegroai/clearml/issues/706))
### ClearML 1.6.1
**Bug Fixes**
* Fix `Task.get_tasks()` fails when sending `search_hidden=False`
* Fix LightGBM example shows UserWarning
### ClearML 1.6.0
**New Features and Improvements**
* New Hyperparameter Optimization CLI `clearml-param-search`
* Improvements to ClearML Data
* Add support for a new ClearML Data UI in the ClearML WebApp
* Add `clearml-data` new options set-description and rename
* Add random seed control using `Task.set_random_seed()` allowing to set a new random seed for task initialization or
to disable it
* Improve error messages when failing to download an artifact
* Improve error messages when testing for permissions
**Bug Fixes**
* Fix axis range settings when logging plots
* Fix `Task.get_project()` to return more than 500 entries ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612))
* Fix pipeline progress calculation
* Fix `StorageManager.upload_folder()` returns `None` for both successful and unsuccessful uploads
* Fix script path capturing stores a relative path and not an absolute path
* Fix HTML debug samples are saved incorrectly on S3
* Fix Hydra deprecation warning in examples
* Fix missing requirement for TensorBoardX example
**Known Issues**
* When removing an image from a Dataset, its preview image won't be removed
* Moving Datasets between projects still shows the Dataset in the old project

71
docs/release_notes/sdk/open_source/ver_1_7.md Normal file
View File

@@ -0,0 +1,71 @@
---
title: Version 1.7
---
### ClearML 1.7.2
**New Features and Improvements**
* Support running Jupyter Notebook inside a git repository (repository will be referenced without uncommitted changes
and Jupyter Notebook will be stored on plain code as uncommitted changes)
* Add Jupyter Notebook fail warning
* Allow pipeline steps to return string paths without them being treated as a folder artifact and zipped ([ClearML GitHub issue #780](https://github.com/allegroai/clearml/issues/780))
* Remove `future` from Python 3 requirements
**Bug Fixes**
* Fix exception raised when using `ThreadPool` ([ClearML GitHub issue #790](https://github.com/allegroai/clearml/issues/790))
* Fix Pyplot/Matplotlib binding reports incorrect line labels and colors ([ClearML GitHub issue #791](https://github.com/allegroai/clearml/issues/791))
* Pipelines
* Fix crash when running cloned pipeline that invokes a step twice ([ClearML GitHub issue #769](https://github.com/allegroai/clearml/issues/769))
* Fix pipeline argument becomes `None` if default value is not set
* Fix `retry_on_failure` callback does nothing when specified on `PipelineController.add_step()`
* Fix pipeline clone logic
* Jupyter Notebook
* Fix support for multiple Jupyter servers running on the same machine
* Fix issue with old/new notebook packages installed
* Fix local cache with access rules disabling partial local access
* Fix `Task.upload_artifact()` fails uploading pandas `DataFrame`
* Fix relative paths in examples ([ClearML GitHub PR #787](https://github.com/allegroai/clearml/issues/787)
### ClearML 1.7.1
**New Features and Improvements**
* Add callback option for pipeline step retry
**Bug Fixes**
* Fix Python Fire binding
* Fix Dataset failing to load helper packages should not crash
* Fix `Dataset.get_local_copy()` is allowed for a non-finalized dataset
* Fix `Task.upload_artifact()` does not upload empty lists/tuples
* Fix pipeline retry mechanism interface
* Fix `Python <3.5` compatibility
* Fix local cache warning (should be a debug message)
### ClearML 1.7.0
**New Features and Improvements**
* ClearML Data: Support providing list of links
* Upload artifacts with a custom serializer ([ClearML GitHub issue #689](https://github.com/allegroai/clearml/issues/689))
* Allow user to specify extension when using custom serializer functions (for artifacts)
* Skip server URL verification in `clearml-init` wizard process
* When calling `Dataset.get()` without "alias" field, tell user that he can use alias to log it in the UI
* Add support for mmcv for logging models
* Add support for Azure and GCP storage in `task.setup_upload()`
* Support pipeline retrying tasks which are failing on suspected non-stable failures
* Better storage (AWS, GCP) internal load balancing and configurations
* Add `Task.register_abort_callback`
**Bug Fixes**
* Allow getting datasets with non-semantic versioning ([ClearML GitHub issue #776](https://github.com/allegroai/clearml/issues/776))
* Fix interactive plots (instead of a generated png)
* Fix Python 2.7 support
* Fix clearml datasets `list` functionality
* Fix `Dataset.init` to not modify task (to `Dataset.create`)
* Fix failure with large files upload on HTTPS
* Fix 3d plots with plt shows 2d plot on task results page
* Fix uploading files with project's `default_upload_destination` ([ClearML GitHub issue #734](https://github.com/allegroai/clearml/issues/734))
* Fix broken Matplotlib reporting - using logarithmic scale breaks reporting
* Fix wildcard support in `clearml-data` CLI
* Fix `report_histogram` - does not show "horizontal" orientation ([ClearML GitHub issue 699](https://github.com/allegroai/clearml/issues/699))
* Fix table reporting - when using `logger.report_table(title, series, iteration, etc)`, the `series` arg does not appear in UI ([ClearML GitHub issue 684](https://github.com/allegroai/clearml/issues/684))
* Fix artifacts (and models) use task's original name and not new name
* Fix very long filenames from S3 can't be downloaded (with `get_local_copy()`)
* Fix overwrite of existing output models on pipeline task with `monitor_models` ([ClearML GitHub issue #757](https://github.com/allegroai/clearml/issues/757))

70
docs/release_notes/sdk/open_source/ver_1_8.md Normal file
View File

@@ -0,0 +1,70 @@
---
title: Version 1.8
---
### ClearML 1.8.3
**Bug Fixes**
* Set GCS credentials to `None` if invalid service account credentials are provided ([ClearML GitHub issue #840](https://github.com/allegroai/clearml/issues/840))
* Fix a sync issue when loading deferred configuration
### ClearML 1.8.2
**New Features and Improvements**
* Added `VCS_ENTRY_POINT` environment variable that overrides ClearML's entrypoint auto-detection
**Bug Fixes**
* Fix all parameters returned from a pipeline are considered strings
* Fix `Task.set_parameters()` does not add parameter type when parameter exists but does not have a type
### ClearML 1.8.1
**New Features and Improvements**
* Raise error on failed uploads ([ClearML GitHub issue #810](https://github.com/allegroai/clearml/issues/819))
* Add hyperdataset examples ([ClearML GitHub PR #823](https://github.com/allegroai/clearml/commit/f6b9efe54e1246adba4036c56bc6e8a0bdb99948))
* Change `report_event_flush_threshold` default to 100
* Add `ModelInfo.weights_object()` to store callback access to the actual model object being stored (valid for both
pre/post save calls, otherwise `None`)
* Support `num_workers` in dataset operation
* Support max connections setting for Azure storage using the `sdk.azure.storage.max_connection` configuration option
**Bug Fixes**
* Fix clearml logger default level cannot be changed ([ClearML GitHub issue #741](https://github.com/allegroai/clearml/issues/741))
* Fix Hydra doesn't get overridden information from ClearML ([ClearML GitHub issue #751](https://github.com/allegroai/clearml/issues/751))
* Fix `StorageManager.list(“s3://..”, with_metadata=True)` doesn't work
* Fix `ModelsList.keys()` is missing
* Fix `CLEARML_DEFERRED_TASK_INIT=1` doesn't work
* Fix default API method does not work when set in configuration
### ClearML 1.8.0
**New Features and Improvements**
* Add tarfile member sanitization to `extractall()`([ClearML GitHub PR #803](https://github.com/allegroai/clearml/pull/803))
* Add `Task.delete_artifacts()` with `raise_on_errors` argument ([ClearML GitHub issue #805](https://github.com/allegroai/clearml/issues/805))
* Add CI/CD example ([ClearML GitHub PR #815](https://github.com/allegroai/clearml/pull/815))
* Limit number of `_serialize` requests when adding list of links with `add_external_files()` ([ClearML GitHub issue #813](https://github.com/allegroai/clearml/issues/813))
* Add support for connecting Enum values as parameters
* Improve Colab integration (store entire colab, not history)
* Add `clearml.browser_login` to authenticate browser online sessions such as Colab, Jupyter Notebooks etc.
* Remove `import_bind` from stack trace of import errors
* Add `sdk.development.worker.report_event_flush_threshold` configuration option to control the number of events to trigger a report
* Return stub object from `Task.init()` if no `clearml.conf` file is found
* Improve manual model uploading example
* Remove deprecated demo server
**Bug Fixes**
* Fix passing `compression=ZIP_STORED` (or 0) to `Dataset.upload()` uses `ZIP_DEFLATED` and overrides the user-supplied
argument ([ClearML GitHub PR #812](https://github.com/allegroai/clearml/pull/812))
* Fix `unique_selector` is not applied properly on batches after the first batch. Remove default selector value since
it does not work for all event types (and we always specify it anyway)
* Fix `clearml-init` colab detection
* Fix cloning pipelines run with `start_locally()` doesn't work
* Fix if project has a default output uri there is no way to disable it in development mode (manual), allow passing
`output_uri=False` to disable it
* Fix git remote repository detection when remote is not "origin"
* Fix reported images might not all be reported when waiting to complete the task
* Fix `Dataset.get_local_copy()` deletes the source archive if it is stored locally
* Fix too many parts will cause preview to inflate Task object beyond its 16MB limit - set a total limit of 320kbs
* Fix media preview is created instead of a table preview
* Fix `task.update_output_model()` should always upload local models to a remote server
* Fix broken pip package might mess up requirements detection

80
docs/release_notes/sdk/open_source/ver_1_9.md Normal file
View File

@@ -0,0 +1,80 @@
---
title: Version 1.9
---
### ClearML 1.9.3
**Bug Fixes**
* Fix broken `Task._get_status()`, which was breaking `clearml-session` in the latest version
* Fix path substitution, making it possible to store unsubstituted URL for models ([ClearML GitHub PR #935](https://github.com/allegroai/clearml/pull/935))
### ClearML 1.9.2
**New Features and Improvements**
* Support parsing queue name when providing execution queue in pipelines code ([ClearML GitHub PR #857](https://github.com/allegroai/clearml/pull/857))
* Ignore `None` values for keys in the `click` argument parser ([ClearML GitHub issue #902](https://github.com/allegroai/clearml/issues/902))
* Improve docstrings for `Task.mark_completed()` and `Task.close()` (ClearML GitHub PRs [#920](https://github.com/allegroai/clearml/pull/920) and [#921](https://github.com/allegroai/clearml/pull/921))
* Add pre/post execution callbacks to pipeline steps through `@PipelineDecorator.component`
* Add status-change callback to pipeline steps through `PipelineController.add_step()`, `PipelineController.add_function_step()`,
and `@PipelineDecorator.component`
**Bug Fixes**
* Fix missing debug samples when reporting using TensorBoard ([ClearML GitHub issue #923](https://github.com/allegroai/clearml/issues/923))
* Fix wrong Jupyter API token during repository detection ([ClearML GitHub PR #904](https://github.com/allegroai/clearml/pull/904))
* Fix typo in the warning for very large git diffs ([ClearML GitHub PR #932](https://github.com/allegroai/clearml/pull/932))
* Fix pipelines from tasks don't propagate `parameter_override` values in `PipelineController.add_step()`
* Fix folders and files uploaded to S3 and Azure with `StorageManager.upload_file()` receive wrong MIME types
* Fix the CSV file preview in Datasets
* Fix `Task.connect_configuration()` doesn't work with non-string dictionary keys
* Fix `lightgbm_example` deprecation warning
* Fix potential race condition in `get_or_create_project()`
### ClearML 1.9.1
**New Features and Improvements**
* Add signature version to `boto3` configuration ([ClearML GitHub issue #883](https://github.com/allegroai/clearml/issues/883))
* Allow requesting custom token expiration using the `api.auth.req_token_expiration_sec` configuration setting
* Add Python 3.11 support
**Bug Fixes**
* Fix `UniformParameterRange.to_list` throws error when step size is not defined ([ClearML GitHub issue #858](https://github.com/allegroai/clearml/issues/858))
* Fix `StorageManager.list()` does not return size metadata ([ClearML GitHub issue #865](https://github.com/allegroai/clearml/issues/865))
* Fix storage with path substitutions ([ClearML GitHub issue #825](https://github.com/allegroai/clearml/issues/825))
* Fix extras in ClearML installation prevents clearml from being included in requirements ([ClearML GitHub issue #867](https://github.com/allegroai/clearml/issues/867))
* Fix metadata set on an uploaded model object is not accessible ([ClearML GitHub issue #890](https://github.com/allegroai/clearml/issues/890))
* Fix Azure storage upload not working ([ClearML GitHub issue #868](https://github.com/allegroai/clearml/issues/868))
* Fix `task.connect` list of dicts parsed incorrectly in remote
* Fix casting `None` to `int` fails uploads and permission checks
* Fix numpy 1.24 support
* Fix `clearml-data` previews are saved on file server even when `output_uri` is specified
* Fix connecting a dictionary to task sometimes raises an exception
* Fix authentication headers are not set on substituted fileserver URLs
* Fix `Task.get_project_id()` cannot find hidden projects
* Fix TriggerScheduler docstrings ([ClearML GitHub issue #881](https://github.com/allegroai/clearml/issues/881))
### ClearML 1.9.0
**New Features and Improvements**
* Add `r` prefix to `re.match()` strings ([ClearML GitHub issue #834](https://github.com/allegroai/clearml/issues/834))
* Add `path_substitution` to `clearml.conf` example file ([ClearML GitHub PR #842](https://github.com/allegroai/clearml/pull/842))
* Clarify `deferred_init` usage in `Task.init()` ([ClearML GitHub issue #855](https://github.com/allegroai/clearml/issues/855))
* Add pipeline decorator argument to control docker image ([ClearML GitHub issue #856](https://github.com/allegroai/clearml/issues/856))
* Add `StorageManager.set_report_upload_chunk_size()` and `StorageManager.set_report_download_chunk_size()` to set chunk
size for upload and download
* Add `allow_archived` argument in `Task.get_tasks()`
* Support querying model metadata in `Model.query_models()`
* Add `Dataset.set_metadata()` and `Dataset.get_metadata()`
* Add `delete_from_storage` (default `True`) to `Task.delete_artifacts()`
**Bug Fixes**
* Fix jsonargparse and pytorch lightning integration broken for remote execution ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403))
* Fix error when using `TaskScheduler` with `limit_execution_time` ([ClearML GitHub issue #648](https://github.com/allegroai/clearml/issues/648))
* Fix dataset not synced if the changes are only modified files ([ClearML GitHub issue #822](https://github.com/allegroai/clearml/issues/822))
* Fix `StorageHelper.delete()` does not respect path substitutions ([ClearML GitHub issue #825](https://github.com/allegroai/clearml/issues/825))
* Fix can't write more than 2 GB to a file
* Fix `StorageManager.get_file_size_bytes()` returns `ClientError` instead of `None` for invalid S3 links
* Fix Dataset lineage view is broken with multiple dataset dependencies
* Fix `tensorflow_macos` support
* Fix crash when calling `task.flush(wait_for_uploads=True)` while executing remotely
* Fix `None` values get casted to empty strings when connecting a dictionary

170
docs/release_notes/ver_0_14.md
View File

@@ -1,170 +0,0 @@
---
title: Version 0.14
---
:::important
**Trains** is now **ClearML**.
:::
## Version 0.14.3
### Trains
**Features**
* Add the `Logger.report_histogram` method `mode` parameter.
The values are `group` (the default), `stack`, and `relative`.
* Add [PEP610](https://www.python.org/dev/peps/pep-0610/) support for the new `pip` version 20.1 and newer Git reference
feature (see [pypa/pip#609](https://github.com/pypa/pip/issues/609)) ([GitHub Issue #62](https://github.com/allegroai/trains/issues/62)).
* Improve support for PyTorch Ignite integration.
* Support `#` and `?` in uploaded image file names.
**Bug Fixes**
* Support `ArgumentParser.parse_arg` use before initializing a Task when `auto_connect_arg_parser=False`.
* Fix `upload_object_via_stream` in the Azure storage driver.
* Fix the `Task.get_tasks` method return type.
## Version 0.14.2
### Trains
**Features**
* Add media (audio) support for both `Logger` and TensorBoard binding using `Logger.report_media` ([GitHub Issue #120](https://github.com/allegroai/trains/issues/120)).
* Add [autokeras example](https://github.com/allegroai/trains/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) ([GitHub Issue #125](https://github.com/allegroai/trains/issues/125)).
* Add caching of extracted zip artifacts.
* Add `Task.get_tasks` filtering support.
* Add `trains.storage.StorageManager`.
* Add nicer stdout log flush.
* Add text logging to TF V1 example.
* Improve SSL retries and error handling.
* Improve shutdown behavior in case of a user abort (Ctrl-C).
* Improve repository and module detection:
* Improve resource monitor.
* Report memory usage for process (and sub-processes).
* Report GPU memory usage for process (and sub-processes).
* Stability
* Improve Jupyter support:
* Make sure `trains` is included in Jupyter requirements.
* Ignore IPython directives in converted Python script (like `%` and `!` lines).
* Update PyTorch / TensorboardX examples.
**Bug Fixes**
* Fix renaming / deleting model file right after saving will break asynchronous upload ([GitHub Issue #123](https://github.com/allegroai/trains/issues/123)).
* Fix shutdown behavior:
* Logger flush delay and termination.
* Repository / package detection termination.
* Threads termination.
* Fix text encoding `utf-8` and `pr_curve` broken in Tensorboard support.
* Fix single log request exceeds max packet size.
* Fix upload debug assets with series / title including `,` or `/` did not add iteration index to the filename.
* Fix `scikit-learn` module detection.
* Use UID in message if username cannot be retrieved.
* Fix cast in Tensorflow v2 example.
### Trains Server
**User survey**
* Add survey invitation popup.
**Bug fixes**
* Fix misaligned titles in experiment results, scalars, and plots (**RESULTS** tab, **SCALARS** and **PLOTS** sub-tabs).
* Fix parallel coordinate comparison in comparing experiments. Adding dimension with no values removes all lines.
* Fix resizing the experiments tables window. The preferences wheel jumped over the titles.
* Fix the browser zoom (**CTRL +**, **CTRL -**, and mouse wheel) in experiment **RESULTS** **>** **SCALARS** tab. It broke labels ([GitHub Issue #127](https://github.com/allegroai/trains/issues/127)).
## Version 0.14.1
### Trains
**Features and Bug Fixes**
* Add OpenMPI / Slurm support, support `TRAINS_FORCE_MASTER_NODE` environment variable to allow forcefully specifying a node is master (not based on the MPI Rank).
* Add `Task.get_models` and `Task.models` for retrieving stored models for previously executed tasks
* Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) ([GitHub Issue #26](https://github.com/allegroai/trains/issues/26)).
* Support reusing models.
* Fix TensorFlow not a number (`NaN`) and infinity (`Inf`)
values support ([GitHub Issue #118](https://github.com/allegroai/trains/issues/118)).
* Fix joblib binding.
* Fix metrics / files / models upload on shutdown.
* Fix support for Task init / close multiple times.
* Update `manual_reporting.py` example with `Logger.report_table` example.
* Deprecate `Task.set_model_config`, `Task.get_model_config_text`, and `Task.get_model_config_dict`.
### Trains Server
**Features and Bug Fixes**
* Support default **Trains Server** Kubernetes Helm ports in the **Trains Web-App (UI)**.
* Fix issue switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/allegroai/trains/issues/75)).
* Do not reset view filtering when switching tabs in experiment view.
### Trains Agent
**Features and Bug Fixes**
* Add `daemon` detached mode (`--detached`, `-d`) that runs the agent as daemon in the background and returns immediately.
* Auto mount `~/.git-credentials` into a Docker container, if the file exists.
* Add `TRAINS_AGENT_EXTRA_PYTHON_PATH` environment variable to allow the adding of an additional Python path during experiment execution (helpful when using extra un-tracked modules).
* Fix "run as user" feature (using `TRAINS_AGENT_EXEC_USER` environment variable).
* Fix PyTorch support to ignore minor versions when looking for package to install / download.
* Fix experiment execution output handling.
## Version 0.14.0
### Trains
**Features and Bug Fixes**
* Add support for reporting tables (pandas.DataFrame, CSV files, URL to CSV file) using the `Logger.report_table` method.
* Add support for setting a Task's initial iteration to allow continuing previous runs using the `Task.set_initial_iteration` and `Task.get_initial_iteration` methods.
* Add multiprocess task support ([GitHub Issue #96](https://github.com/allegroai/trains/issues/96)).
* Add `trains-init` support for config file override using the `TRAINS_CONFIG_FILE` environment variable or `--file` command line argument ([GitHub Issue #105](https://github.com/allegroai/trains/issues/105)).
* Add initial slurm support.
* Allow disabling repository detection when calling `Task.init`.
* Add Jupyter Notebook server execution example.
* Update examples.
* Improve warning messages for storage errors and missing storage packages.
* Fix HTTP link quoting in stored links.
### Trains Server
**Features and Bug Fixes**
* Add debug image viewing for an unlimited number of iterations, filtering by metric, and comparing debug images for different iterations.
* Add debug image view zoom, fit to screen, resize, hover coordinates ([GitHub Issue #68](https://github.com/allegroai/trains/issues/68)).
* Add switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/allegroai/trains/issues/75)).
* Add scalar minimums and maximums to comparing experiments.
* Add opening HTML artifact files in the browser.
* Add Web-App notification for version updates.
* Add toggle button to switch between full screen and standard views in the **RESULTS** tab.
* Add a load more button to the global search results.
* Add automatic detection of debug video type from the file extension.
* Add `Task.get_parameters_as_dict` and `Task.set_parameters_as_dict`.
* Allow editing model description.
* Allow editing experiment uncommitted changes and installed packages separately.
* Allow deleting an input model from an experiment whose status is *Draft*.
* Fix dialog elements which are now accessible in non-100% screen scale. ([GitHub Issue #89](https://github.com/allegroai/trains/issues/89)).
* Fix custom columns in the experiments table not properly loaded from browser store ([GitHub Issue #106](https://github.com/allegroai/trains/issues/106)).
* Fix issue with showing 2D graphs containing sub-plots ([trains #slack](https://github.com/allegroai/trains/issues/106))
* Fix issues with vertical scrolling of experiment logs.
* Fix issue with image plots sometimes not visible in experiments comparison.
### Trains Agent
**Features and Bug Fixes**
* Add support for `trains-agent execute --id <experiment-id> --docker` that allows executing a specific experiment inside a docker container.
* Add support for `trains-agent execute --id <template-experiment-id> --clone` that clones the provided experiment and executes the cloned experiment.
* Add support for `APIClient.models.delete` to allow programmatically deleting a model ([GitHub trains-server Issue #32](https://github.com/allegroai/trains-server/issues/32)).
* Add daemon support for passing storage-related OS environment variables to experiments executed inside a Docker container (supported by trains>=0.13.3):
* AWS: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_DEFAULT_REGION`
* Azure: `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_KEY`
* Google: `GOOGLE_APPLICATION_CREDENTIALS`
* Fix Git checkout with submodules ([GitHub Issue #112](https://github.com/allegroai/trains/issues/112)).
* Docker image in command line takes precedence over the Docker image specified in an experiment.

199
docs/release_notes/ver_0_15.md
View File

@@ -1,199 +0,0 @@
---
title: Version 0.15
---
:::important
**Trains** is now **ClearML**.
:::
## Version 0.15.1
### Trains
**Features**
* Add direct plotly figure reporting ([GitHub Issue #136](https://github.com/allegroai/trains/issues/136)).
* Improve matplotlib integration ([GitHub Issue #140](https://github.com/allegroai/trains/issues/140)).
* Add `Task.set_parent` ([GitHub Issue #144](https://github.com/allegroai/trains/issues/144)).
* Add `Task.wait_for_status` ([GitHub Issue #145](https://github.com/allegroai/trains/issues/145)).
* Enhance framework `save` / `load` binding callbacks ([pytorch/ignite Issue #1056](https://github.com/pytorch/ignite/issues/1056)).
* Add `TrainsJob.is_failed`.
* Add APIClient support using `trains.backend_api.session.APIClient`.
* Add URL tracking when using `StorageManager.get_local_copy` (log remote URL instead of local file copy).
* Improve and expand examples.
**Bug Fixes**
* Fix `pytorch_tensorboard.py` example ([GitHub Issue #148](https://github.com/allegroai/trains/issues/148)).
* Fix `ssh://user@domain@server.com/` in Git repository ([GitHub Issue #149](https://github.com/allegroai/trains/issues/149)).
* Fix do not call `make_deterministic` on import.
* Fix initialization wizard (allow at most two verification retries, then print error) ([GitHub trains-agent Issue #16](https://github.com/allegroai/trains-agent/issues/16)).
* Fix Google CoLab code / package detection.
* Improve pytorch-ignite integration.
* Fix TensorFlow version 2 and later histogram binding.
* Fix `Logger.tensorboard_single_series_per_graph`.
* Fix anonymous named models.
* Fix incorrect entry point detection when called from Trains wrapper (e.g. `TrainsLogger` in PyTorch Ignite / Lightning).
### Trains Server
**Bug Fixes**
* Fix experiment clone dialog resets while typing ([GitHub trains-server Issue 44](https://github.com/allegroai/trains-server/issues/44)).
* Fix image-only plots are not displayed ([GitHub Issue #140](https://github.com/allegroai/trains/issues/140)).
* Fix last item in plots list is hidden.
* Fix input model edit view contains a redundant save button.
* Fix credentials popup shows incorrect URL for fileserver.
* Fix experiment description does not allow multiple lines.
* Fix tags filter ("No tags" selection, display only tags available for the project).
* Fix cannot save empty model configuration.
* Fix add tag dialog does not close after adding a tag.
* Show active column filters for values even when values are unavailable in current project.
* Fix project selection resets when cloning an experiment.
* Fix several broken icons.
* Fix cannot add tags in Results view.
* Add warning when deleting a model/experiment tag.
* New icons for experiment/task types.
### Trains Agent
**Features**
* Add Trains Agent Daemon and Services docker files.
**Bug Fixes**
* Fix initialization wizard (allow at most two verification retries, then print error).
* Add warning on `--gpus` with no detected CUDA version ([GitHub trains-agent Issue 24](https://github.com/allegroai/trains-agent/issues/24)).
* Add `agent.force_git_ssh_protocol` configuration option to force all Git links to `ssh://` ([GitHub trains-agent Issue 16](https://github.com/allegroai/trains-agent/issues/16)).
* Add Git user / pass permission into pip package installation from Git repository ([GitHub trains-agent Issue 22](https://github.com/allegroai/trains-agent/issues/22)).
## Version 0.15.0
### Trains
**Features**
* Add automation support including hyperparameter optimization (see example [here](https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py))
* `Task.init` `auto_connect_arg_parser` argument can accept a dictionary disabling specific keys from the argparser (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589355878206700))
* Allow `worker_id` override using `TRAINS_WORKER_NAME` environment variable (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589920581251600?thread_ts=1589920220.250100&cid=CTK20V944))
* Support layout configuration for plotly objects using `extra_layout` argument in all `Logger` reporting methods ([GitHub Issue #136](https://github.com/allegroai/trains/issues/136)).
* Add `Task.execute_remotely` to allow cloning and enqueuing a locally executed task (or stopping and re-enqueuing a remotely executed task) ([GitHub Issue #128](https://github.com/allegroai/trains/issues/128)).
* Add Parquet framework and model type.
* Support recursive model folder packaging.
* Add `Task.get_reported_console_output` and `Task.get_reported_scalars` to allow retrieval of reported output and scalar metrics.
* Add `Task.add_requirements` to force requirement package into "installed packages."
* Improve task reuse responsiveness.
* Add `raise_on_error` (default False) argument to `Model.get_local_copy` and `Artifact.get_local_copy` ([GitHub trains-agent Issue #17](https://github.com/allegroai/trains-agent/issues/17)).
* Support `Task.get_task` without project name (i.e. all projects)
* Support using the file calling `Task.init` as the task's script in case sys.argv doesn't point to a git repository
* Support detecting and remotely executing code running from a module (i.e. `-m module`)
* Add callback for framework `save` / `load` binding for better integration with pytorch / ignite ([GitHub pytorch/ignite Issue #1056](https://github.com/pytorch/ignite/issues/1056))
* Support new task types provided in Trains Server v0.15.0
* Add automation and distributed examples
* Upgrade default pip version to `<20.2`
**Bug Fixes**
* Fix `exact_match_regex` in case of empty pattern ([GitHub Issue #138](https://github.com/allegroai/trains/issues/138)).
* Address deprecation warning and newer `attrs` versions in MetricsEventAdapter ([GitHub Issue #134](https://github.com/allegroai/trains/issues/134)).
* Fix issues with plotly support (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589871253243600) and [thread](https://clearml.slack.com/archives/CTK20V944/p1589279340195000)).
* Fix default argument behavior to match argparse behavior.
* Fix `OutputModel` with `task=None` should use current task, if exists.
* Fix `Task.get_task()` to raise proper error on incorrect `task_id`.
* Fix `Task.enqueue()` to use an exact queue name match.
* Fix `NaN`, `Inf` and `-Inf` values display in reported table (not supported by JSON).
* Limit max requirement size to 0.5 MB.
* Fix issues with repository analysis.
* Fix `StorageManager` so it should only try to extract .zip files, `Model` should not auto extract package ([GitHub trains-agent Issue #17](https://github.com/allegroai/trains-agent/issues/17)).
### Trains Server
**Features**
* Add support for debug media (images, audio, video) ([GitHub Issue #120](https://github.com/allegroai/trains/issues/120)).
* Add support for previewing HTML debug samples.
* Add experiment and model tag management ([GitHub Issue #46](https://github.com/allegroai/trains/issues/46)).
* Add support for more task types (see Trains v0.15.0 and Trains Agent v0.15.0 for related support and services mode).
* Experiment comparison:
* Add Last / Max (maximal) / Min (minimal) scalar values to parallel coordinates in experiment comparison ([GitHub Issue #131](https://github.com/allegroai/trains/issues/131)).
* Add deep comparison support to experiment comparison, including uncommitted changes by file with line diff and search of all text.
* Enhance table columns:
* Add dynamic column ordering to the experiments and models tables.
* Add the User column to the experiments table.
* Add the Description to experiments and models tables.
* Add double click to table column resizers for column width reset.
* Add column customization to the models table.
* Add support for saving and sharing customized model table URLs by embedding table customizations in the URL.
* Add a popup text editor for all text areas.
* Move resource monitoring plots after the metrics plots in the experiment results SCALARS tab.
* Add support for the clear_all parameter in tasks.reset to improve Trains SDK Task reuse.
* Unify experiments table and comparison auto-refresh (switching between table and comparison auto refresh setting did not change).
* Add a cancel button to all text searches.
* Improve the experiment / info panel split to 75% / 25%.
* Improve infrastructure:
* Update pymongo and mongoengine to their latest versions.
* Improve mongo performance by eliminating non-indexed queries.
* Upgrade WebApp to Angular 9.
* Use native javascript instead of date-fns.
* Add survey invitation popup.
**Bug Fixes**
* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/allegroai/trains-agent/issues/19)).
* Improve docker host-mount support, use `TRAINS_AGENT_DOCKER_HOST_MOUNT` environment variable.
* Support pip v20.1 local / http package reference in pip freeze.
* Fix detached mode to correctly use cache folder slots.
* Fix `CUDA_VISIBLE_DEVICES` should never be set to "all".
* Do not monitor GPU when running with `--cpu-only`
* Fix experiment comparison issues:
* Scalar comparison type does not reflect displayed comparison ([GitHub Issue #40](https://github.com/allegroai/trains/issues/40)).
* Comparison animations overlapping other fields.
* Cannot choose a metric after clearing it in experiment comparison parallel coordinates.
* Adding a dimension without a value removes all lines in experiment comparison parallel coordinates.
* Fix experiment view issues:
* Custom URL gets erased when clicking on any tab on experiment.
* All experiments re-fetched when navigating through tabs of an experiment.
* Input and output models configuration box sizes differs in the experiments **ARTIFACTS** tab.
* Changing experiment table / info panel split redraws scalars and plots.
* Wrong download URL shown for S3 artifact links with special characters.
* Fix experiment results view issues:
* Browser zoom breaks labels (Ctrl + / Ctrl - / mouse wheel) in experiment results **SCALARS** tab ([GitHub Issue #127](https://github.com/allegroai/trains/issues/127)).
* Debug image name not shown if file cannot be loaded.
* minio images with spaces in the path not displaying correctly
* Plots should use the full panel space.
* Plot relative time displaying incorrectly in experiment results **SCALARS** tab.
* Wall time display in local time in experiment results **SCALARS** tab.
* Fix experiment table issues:
* Hyperparameters sorted as strings, not as numbers ([GitHub Issue #124](https://github.com/allegroai/trains/issues/124)).
* Prevent auto refresh reordering when the info panel is open.
* Table resizing made the preferences wheel to jump over titles.
* Fix experiment reset when both the input and output model are the same causes model deletion.
* Fix experiment clone not working if the input model was deleted.
* Fix missing trash icon (delete) if a custom column title is too long.
* Fix display of aborted tasks in the recent experiments on the Homepage.
### Trains Agent
**Features**
* Add daemon Services Mode (`daemon --services-mode`), where the daemon spins a task in its own Docker container, and verifies start-up and shut-down. This allows multiple tasks to be launched simultaneously on the same machine (currently in CPU mode only), where each task service will register itself as a worker for the lifetime of the task.
* Enhance `build --docker` mode:
* Add `--install-globally` option to install required packages in the Docker's system Python.
* Add `--entry-point option` to allow automatic task cloning when running the Docker container.
* Support PyTorch Nightly builds using the `agent.torch_nightly` configuration flag. If true, the agent looks for a nightly build when a stable torch wheel is not found.
* Add environment variables support for git user / password:
* Using `TRAINS_AGENT_GIT_USER` / `TRAINS_AGENT_GIT_PASS`.
* Pass git credentials to dockerized experiment execution.
* Support running code from module (i.e. -m in execution entry point).
* Add `daemon --create-queue` to automatically create a queue and use it, if the queue name doesn't exist in the server.
* Move `--gpus` and `--cpu-only` to worker args (used by `daemon`, `execute` and `build`).
**Bug Fixes**
* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/allegroai/trains-agent/issues/19)).
* Improve docker host-mount support, use `TRAINS_AGENT_DOCKER_HOST_MOUNT` environment variable.
* Support pip v20.1 local/http package reference in pip freeze.
* Fix detached mode to correctly use cache folder slots.
* Fix `CUDA_VISIBLE_DEVICES` should never be set to "all".
* Do not monitor GPU when running with `--cpu-only`

183
docs/release_notes/ver_1_2.md
View File

@@ -1,183 +0,0 @@
---
title: Version 1.2
---
### ClearML Serving 1.2.0
**New Features and Improvements**
* Improve GPU Performance, 50%-300% improvement over vanilla Triton
* Improve performance on CPU, optimize uvloop + multi-processing
* Add Hugging Face Transformer example
* Add binary input support ([ClearML Serving PR #37](https://github.com/allegroai/clearml-serving/pull/37) )
**Bug Fix**
* stdout/stderr in inference service not logging to dedicated Task
### ClearML Agent 1.2.3
**Bug Fixes**
- Fix PYTHONPATH is overwritten when executing a task (append to it instead)
- Fix pytorch package is reinstalled when the same version is already installed
- Fix copying configuration sets an empty worker name
- Protect dynamic GPUs from failing to parse worker GPU index
### ClearML Agent 1.2.2
**Bug Fixes**
- Fix `CLEARML_AGENT_SKIP_PIP_VENV_INSTALL` fails to find python executable
- Fix `apt-get update` failure causes `apt-get install` not to be executed
### ClearML Agent 1.2.1
**New Features and Improvements**
- Update S3 bucket verify option for minio ([ClearML Agent GitHub PR #83](https://github.com/allegroai/clearml-agent/pull/83))
- Add environment variable for request method ([ClearML Agent GitHub PR #91](https://github.com/allegroai/clearml-agent/pull/91))
- Add additional k8s-glue dockerfiles ([ClearML Agent GitHub PR #94](https://github.com/allegroai/clearml-agent/pull/94) )
- Update default docker image to `nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04`
- Add support for custom docker image resolving using the `agent.default_docker.match_rules` configuration setting (see [here](https://github.com/allegroai/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L169))
- Add `agent.force_git_root_python_path` configuration setting to force adding the git repository root folder to the `PYTHONPATH` (if set working directory is not added to the `PYHTONPATH`)
- Add `build --force-docker` command line argument to the to allow ignoring task container data
- Add `agent.poetry_version` configuration setting to specify poetry version (and force installation of poetry if missing, see [here](https://github.com/allegroai/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L65))
- Add custom build script support
- Add extra configurations when starting daemon
- Add `agent.package_manager.force_original_requirements` configuration option, allowing to only use original requirements produced by local execution (note that using this configuration option prevents editing installed packages using the UI)
- Add support for the `CLEARML_AGENT_PROPAGATE_EXITCODE` environment variable. Set this variable to `1` to allow ClearML Agent to return a nonzero exit code on failure
- Update `clearml-agent init` (use `app.clear.ml` as default server, add git token references)
**Bug Fixes**
- Fix virtualenv python interpreter used ([ClearML Agent GitHub PR #98](https://github.com/allegroai/clearml-agent/pull/98))
- Fix typing package incorrectly required for Python>3.5 ([ClearML Agent GitHub PR #103](https://github.com/allegroai/clearml-agent/pull/103))
- Fix symbolic links not copied from cached VCS into working copy (windows platform will result with default copy content instead of original symbolic link) ([ClearML Agent GitHub PR #89](https://github.com/allegroai/clearml-agent/pull/89))
- Fix agent fails to check out code from main branch when branch/commit is not explicitly specified ([ClearML GitHub issue #551](https://github.com/allegroai/clearml/issues/551))
- Fix `git+git://` requirements
- Fix `default_python` calculation (and verbosity)
- Fix using deprecated `abc` support (Python 3.10 compatibility)
- Fix no default value for `CLEARML_API_DEFAULT_REQ_METHOD` causes `ValueError` if not specified
- Fix `agent.hide_docker_command_env_vars` mode to include URL passwords and handle environment vars containing docker commands
- Fix conda package manager listed packages with local links (`@ file://`) should ignore the local package if it does not exist
- Fix cuda patch version support in conda
- Fix agent attempts to check out code when in standalone mode
- Fix `FORCE_LOCAL_CLEARML_AGENT_WHEEL` environment variable handling when running from a Windows host
- Fix user-provided `"` is unnecessarily replaced to `\\"`
- Fix token is not propagated to docker in case credentials are not available
- Fix PyTorch aarch64 and windows support
- Fix VCS packages are reinstalled when the same commit version is already installed
- Fix git packages are installed even if commit is given and is preinstalled when using cached virtual environment
### ClearML SDK 1.2.1
**Bug Fixes**
- Fix HTTP download fails constructing URL ([ClearML GitHub issue #593](https://github.com/allegroai/clearml/issues/593))
### ClearML SDK 1.2.0
**Features**
- Add fastai v2 support ([ClearML GitHub PR #571](https://github.com/allegroai/clearml/pull/571))
- Add catboost support ([ClearML GitHub PR #542](https://github.com/allegroai/clearml/pull/542))
- Add Python Fire support ([ClearML GitHub PR #550](https://github.com/allegroai/clearml/pull/550))
- Add new Azure Storage driver support ([ClearML GitHub PR #548](https://github.com/allegroai/clearml/pull/548))
- Add requirements file support in `Task.add_requirements` ([ClearML GitHub PR #575](https://github.com/allegroai/clearml/pull/575))
- Allow overriding `auto_delete_file` in `Task.update_output_model()` ([ClearML GitHub issue #554](https://github.com/allegroai/clearml/issues/554))
- Support `artifact_object` empty string
- Add `skip_zero_size_check` to `StorageManager.download_folder()`
- Add support for extra HTTP retry codes (see [here](https://github.com/allegroai/clearml/blob/2c916181b90c784fe0bd267cd67ea915e53e36e4/clearml/backend_api/config/default/api.conf#L29) or use `CLEARML_API_EXTRA_RETRY_CODES`)
- Add `Task.get_parameters()` cast back to original type
- Add callback support to `Task.delete()`
- Add autoscaler CPU-only support
- Add AWS autoscaler IAM instance profile support
- Update examples
- Edit HTML reporting examples ([ClearML GitHub PR #546](https://github.com/allegroai/clearml/pull/546))
- Add model reporting examples ([ClearML GitHub PR #553](https://github.com/allegroai/clearml/pull/553))
**Bug Fixes**
- Fix `nargs="?"` without type does not properly cast the default value ([ClearML GitHub issue #531](https://github.com/allegroai/clearml/issues/531))
- Fix using invalid configurations ([ClearML GitHub issue #544](https://github.com/allegroai/clearml/issues/544))
- Fix extra_layout not passed to report_matrix ([ClearML GitHub issue #559](https://github.com/allegroai/clearml/issues/559))
- Fix group arguments in click ([ClearML GitHub PR #561](https://github.com/allegroai/clearml/pull/561))
- Fix no warning when failing to patch argparse ([ClearML GitHub PR #576](https://github.com/allegroai/clearml/pull/576))
- Fix crash in `Dataset.upload()` when there is nothing to upload ([ClearML GitHub PR #579](https://github.com/allegroai/clearml/pull/579))
- Fix requirements, refactor and reformat examples ([ClearML GitHub PRs #567](https://github.com/allegroai/clearml/pull/567), [#573](https://github.com/allegroai/clearml/pull/573), [#582](https://github.com/allegroai/clearml/pull/582))
- Auto-scaler
- Change confusing log message
- Fix AWS tags support
- Fix instance startup script fails on any command (should only fail on the agent failing to launch)
- Fix spin down stuck machine, ignore unknown stale workers
- Fix pandas object passed as `Task.upload_artifact()` preview object
- Fix incorrect timeout used for stale workers
- Fix `clearml-task` calls `Task.init()` in the wrong place when a single local file is used
- Fix ArgumentParser `SUPPRESS` as default should be resolved at remote execution in the same way (i.e. empty string equals `SUPPRESS`)
- Upgrade six version (in case `pathlib2>2.3.7` is installed)
- Fix connected object base class members are not used
- Fix `clearml-init` changing web host after pasting full credentials
- Fix fileserver upload does not support path in URL
- Fix crash on semaphore acquire error
- Fix docs and docstrings ([ClearML GitHub PRs #558](https://github.com/allegroai/clearml/pull/558), [#560](https://github.com/allegroai/clearml/pull/560))
### ClearML Server 1.2.0
**New Features and Improvements**
* Add functionality to "Abort all children" UI action for controller and optimizer tasks
* Add parameter search to task configuration UI section ([ClearML GitHub issue #467](https://github.com/allegroai/clearml/issues/467))
* Add tag exclusion filters in UI experiment and model tables
* Add "Clear Filters" functionality button to UI table filters
* Improve full screen scalar graph to display all data points
* Add UI experiment table option for mass selection of out-of-screen items
* Improve experiment comparison UI:
* New experiment addition modal with table filtering and sorting
* Fix scalar selection being reset upon comparison contents change
* Add Task ID suffix to debug samples with same-name experiments
* Add task ID suffix to experiment name for same-experiment-name series in scalar experiment comparison
* New Settings pages
* User profile
* Webapp configuration
* Workspace settings
* Add "Updated" column to models table
* Add refresh button to full screen scalar display
* Improve display of float values in UI experiment and model tables. Tables will display rounded up float values
according to space allotted to a column. View precise value by hovering over value.
* Add capability to edit username in UI profile page
* Add build number to version information in UI Settings page
**Bug Fixes**
* Fix UI debug sample viewer ignoring metric filter on auto-refresh ([ClearML GitHub issue #512](https://github.com/allegroai/clearml/issues/512))
* Fix "Wall time" x-axis option in UI plots slowing down web app ([ClearML GitHub issue #441](https://github.com/allegroai/clearml/issues/441))
* Fix UI table plot titles don't display their iteration number ([ClearML GitHub issue #474](https://github.com/allegroai/clearml/issues/474))
* Fix auto-refresh resets vertical scroll in UI debug samples ([ClearML GitHub issue #474](https://github.com/allegroai/clearml/issues/474))
* Fix manually run experiments cannot be edited in UI after being reset ([ClearML GitHub issue #449](https://github.com/allegroai/clearml/issues/449))
* Fix UI Workers & Queues and Experiment Table pages display mismatching experiment runtime values ([ClearML Server GitHub issue #100](https://github.com/allegroai/clearml-server/issues/100))
* Fix UI plots sometimes disappear after task execution completion
* Fix UI experiment plots don't display some plots on a http server
* Fix failure to add Azure credentials through UI Settings page by new users
* Fix UI experiment table failing to clear selection of off-screen experiments
* Fix UI experiment table column resizing malfunctioning
* Fix UI table filters being reset after page is refreshed
* Fix failure to create a sub-project on-the-fly when cloning an experiment through the UI
* Fix UI project overview editing enabled when project is public
* Fix model deletion from UI "All Experiments" page navigates to model project's experiment table
* Fix missing "Project" custom column in "All Experiments" views of non-root projects
* Fix UI project cards' task enumeration summaries displaying negative number
* Fix experiment bar graph colors cannot be changed sometimes
* Fix fail to filter experiments on tags beginning with "-"
* Fix tag list order in UI experiment table filter after tag selection
* Fix long hyperparameter names are obscured in UI comparison screen
* Fix add tag context menu operation fails when experiment selection includes a system example
* Fix size of low value points in UI metric snapshot plot
* Fix long queue names obscures copy button in UI queue list and Worker & Queue page plot titles
* Fix UI breadcrumb menus missing values
* Fix UI scalar plot titles are unreadable
* Fix UI buttons not working after closing "Add Experiment" modal in UI experiment comparison page
* Fix underscores disappear while editing UI text blocks on Chromium
* Fix UI Workers & Queues page's queue panel does not display experiment information
* Fix long queue names obscure copy button in UI queue list
* Fix UI debug image viewer momentarily displays previously viewed image
* Fix "Load More" button in UI experiment table presents duplicates of already displayed experiments

138
docs/release_notes/ver_1_3.md
View File

@@ -1,138 +0,0 @@
---
title: Version 1.3
---
### ClearML Serving 1.3.1
**New Features and Bug Fixes**
* Add missing `await` ([ClearML Serving GitHub PR #55](https://github.com/allegroai/clearml-serving/pull/55))
* Add traceback for failing to load `preprocess` class ([ClearML Serving GitHub issue #57](https://github.com/allegroai/clearml-serving/issues/57))
* Fix Triton `config.pbtxt` for missing values or colliding specifications ([ClearML Serving GitHub issue #62](https://github.com/allegroai/clearml-serving/issues/62))
* Add safer code for pulling from Kafka
* Add `str` type to Triton type conversion
* Fix ignore auto-detected platform when passing `config.pbtxt` with platform entry
* Fix Triton engine model with multiple versions was not properly supported
* Fix serving session keep alive is also sent on idle
* Log preprocess exceptions with full stack trace to serving session console output
### ClearML Serving 1.3.0
**New Features and Improvements**
* 20% Overall performance increase
* gRPC channel configuration ([ClearML Serving GitHub issue #49](https://github.com/allegroai/clearml-serving/issues/49))
* Hugging Face Transformer example
**Bug fixes**
* Fix NumPy compatibility ([ClearML Serving GitHub issue #47](https://github.com/allegroai/clearml-serving/issues/47) and [#46](https://github.com/allegroai/clearml-serving/issues/46))
* Fix Triton examples ([ClearML Serving GitHub issue #48](https://github.com/allegroai/clearml-serving/issues/48))
* Add storage environment variables ([ClearML Serving GitHub PR #45](https://github.com/allegroai/clearml-serving/pull/45))
### ClearML Agent 1.3.0
**New Features and Improvements**
* Support private repos from `requirements.txt` file ([ClearML Agent GitHub PR #107](https://github.com/allegroai/clearml-agent/pull/107))
* Bump PyJWT version due to "*Key confusion through non-blocklisted public key formats*" vulnerability
* Add support for additional command line arguments in k8s glue example
* Add Python 3.10 support
**Bug Fixes**
* Fix git unsafe directory issue (disable check on cached vcs folder)
* Fix dynamic GPUs with "all" GPUs on the same worker
* Fix broken pytorch setuptools incompatibility (force setuptools < 59 if torch is below 1.11)
* Fix setuptools requirement issue by making sure that if we have "setuptools" in the original required packages, we preserve the line in the pip freeze list
* Fix optional priority packaged always compare lower case package name
* Fix potential requirement installation failure by making `pygobject` an optional package (i.e. if installation fails
continue the Task package environment setup)
* Fix repository URL contains credentials even when `agent.force_git_ssh_protocol: true`
### ClearML Server 1.3.1
**Bug Fixes**
* Fix scalar graph smoothing ([ClearML Server GitHub issue #127](https://github.com/allegroai/clearml-server/issues/127))
### ClearML SDK 1.3.2
**New Features and Improvements**
* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/allegroai/clearml/issues/604))
* Add reserved OS environments warning
* Add git credentials to colab example ([ClearML GitHub PR #621](https://github.com/allegroai/clearml/pull/621))
* Add jsonargparse support ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403))
* Update autokeras example
**Bug Fixes**
* Fix sub-project separators are incorrectly quoted in generated URLs ([ClearML GitHub PR #584](https://github.com/allegroai/clearml/pull/584))
* Revert Optuna deprecation fix ([ClearML GitHub PR #613](https://github.com/allegroai/clearml/pull/613))
* Fix HPO randomly aborts running tasks before the time limit
* Fix cloud driver overwrites `agent.extra_docker_arguments`
* Fix Pipeline Controller auto-magic framework connect
* Fix unused scroll is not cleared in `Task.get_reported_plots()`
### ClearML SDK 1.3.1
**New Features and Improvements**
* Add Python 3.10 support
**Bug Fixes**
* Update Slack SDK requirement ([ClearML GitHub issue #597](https://github.com/allegroai/clearml/issues/597))
* Fix fork after `task.close()` is called ([ClearML GitHub issue #605](https://github.com/allegroai/clearml/issues/605))
* Fix Azure storage upload ([ClearML GitHub issue #598](https://github.com/allegroai/clearml/issues/598))
* Fix offline mode crash
* Fix task delete response not checked
* Fix pipeline controller kwargs with list
* Fix `PipelineDecorator.debug_pipeline()`
* Fix PipelineDecorator example
* Fix Python 3.10 issues
* Fix handling of legacy fileserver (files.community.clear.ml)
* Fix cloud driver may use None credentials
* Fix APIClient worker raises exception when accessing .name attribute
* Fix minimum/default API version setting
### ClearML Server 1.3.0
**New Features and Improvements**
* New UI pipelines page:
* Pipelines statistics dashboard
* Pipeline run history: View pipeline structure, run configuration and outputs
* Pipeline control: Abort/Run pipelines from UI
* Add support for exporting UI table plots in CSV formats ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400))
* Improve UI breadcrumbs use of screen real-estate ([ClearML GitHub issue #529](https://github.com/allegroai/clearml/issues/529))
* Add UI adherence to user specification of plot colors and labels ([ClearML GitHub issue #518](https://github.com/allegroai/clearml/issues/518))
* Add Model metadata to UI model information (Model table and details page)
* Add multi-selection in UI experiment info-panel mode
* Add labels to workspace credentials
* Add UI Queues "Clear" action
* Add "Show/Hide" controls to UI experiments scalars tab.
* Add Previous/Next controls to UI text area search bars
**Bug Fixes**
* Fix UI plots do not display Plotly `imshow texttemplate` ([ClearML GitHub issue #538](https://github.com/allegroai/clearml/issues/538))
* Fix clicking F5 clears storage credentials from UI settings page
* Fix UI plots "Wall time" horizontal axis option is not working
* Fix disappearing UI full-screen scalar plot
* Fix disappearing object names in table info-panel mode
* Fix clicking "group by" menu in UI experiment scalars page opens the incorrect menu
* Fix disappearing scalar plots in UI experiment comparison page
* Fix hidden custom UI table column is not removed from main column list
* Fix UI plot toolbar missing tools
* Fix UI table plot tools only appear when hovering over the end of the plot
* Fix resizing UI table column causes subsequent columns to thin
* Fix "Tip of the day" window continues appearing after user-requested removal
### ClearML SDK 1.3.0
**New Features and Improvements**
* Add new pipeline visualization support (requires ClearML Server v1.3)
* Support IAM Instance Profile in AWS auto-scaler
* Remove old server API versions support (pre-ClearML Server)
* Restructure FastAI examples
**Bug Fixes**
* Fix failed catboost bind on GPU ([ClearML GitHub issue #592](https://github.com/allegroai/clearml/issues/592))
* Fix Optuna `n_jobs` deprecation warning
* Fix invalid method called on `delete()` error

133
docs/release_notes/ver_1_4.md
View File

@@ -1,133 +0,0 @@
---
title: Version 1.4
---
### ClearML Agent 1.4.1
**New Features and Improvements**
* Add warning if venv cache is disabled
* Add `agent.disable_ssh_mount` configuration option (same as the `CLEARML_AGENT_DISABLE_SSH_MOUNT` environment variable)
**Bug Fixes**
* Fix docker command for monitoring child agents
* Fix `--gpus all` not reporting GPU stats on worker machine
### ClearML Agent 1.4.0
**New Features and Improvements**
* Add support for MIG devices (use `0:1` for GPU 0 slice 1, or use 0.1)
* Add `agent.enable_git_ask_pass` to improve passing user/pass to git commands
* Add docker `ssh_ro_folder` (default `/.ssh`) and changed docker `ssh_folder `(default: `~/.ssh`)
* Allow overriding pytorch lookup page (See `torch_page`, `torch_nightly_page` and `torch_url_template_prefix` under the
`agent.package_manager` configuration settings)
* Add support for abort callback registration
* K8s glue
* Add `CLEARML_K8S_GLUE_START_AGENT_SCRIPT_PATH` environment variable to allow customizing the agent startup script location
* Add debug environment variable `CLEARML_AGENT_DEBUG_INFO`
* Add `CLEARML_AGENT_CHILD_AGENTS_COUNT_CMD` environment variable to allow overriding child agent count command in k8s
* Refactor template handling
**Bug Fixes**
* Fix Python 3.10+ support
* Fix `use_credentials_chain` is missing in config file example
* Fix Git PAT messages
* Fix home folder in `clearml.conf` to ~ (instead of `/root`)
* Fix docker mode uses `~/.clearml/venvs-builds` as default for easier user-mode containers
* Fix package `@ file://` with quoted (URL style) links should not be ignored
* Fix name not escaped as regex (all services "get_all" use regex for name)
* Fix second .ssh temp mount fails if container changes the files inside
* Fix GCP load balancer does not forward GET request body (allow changing default request action to `PUT/POST/GET`. See
`api.http.default_method` or `CLEARML_API_DEFAULT_REQ_METHOD`)
* K8s glue
* Fix resolving k8s pending queue may cause a queue with a UUID name to be created
* Fix template namespace should override default namespace
* Fix `extra_bash_init_cmd` location in initial bash script
* Fix debug mode
* Fixed documentation ([ClearML Agent PR #117](https://github.com/allegroai/clearml-agent/pull/117))
### ClearML SDK 1.4.1
**Bug Fixes**
* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674))
### ClearML SDK 1.4.0
**New Features**
* Add OpenMMLab example ([ClearML GitHub issue #654](https://github.com/allegroai/clearml/issues/654))
* Add support for saving artifacts with different formats ([ClearML GitHub issue #634](https://github.com/allegroai/clearml/issues/634))
* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/allegroai/clearml/issues/604))
* Support more than 500 results in `Task.get_tasks()` using the `fetch_only_first_page` argument ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612))
* Support links in `clearml-data` ([ClearML GitHub issue #585](https://github.com/allegroai/clearml/issues/585))
* Support deferred task initialization using `Task.init()` argument `deferred_init` (beta feature)
* Support resuming experiments when importing an Offline session
* Add `--import-offline-session` command line option to `clearml-task`
* Support automatically logging Tensorboard Hparams
* Add wildcard support for model auto-logging, see [`Task.init()`](../references/sdk/task.md#taskinit)
* Add support for Lightning CLI
* Support None values in `Task.connect()`
* Add `Model.project` getter/setter
* Add support for Task progress indication
* Datasets
* Improve Dataset version table
* Add warning to Dataset creation on current Task
* Examples and documentation
* Add manual seaborn logging example ([ClearML GitHub PR #628](https://github.com/allegroai/clearml/pull/628))
* Change package author
* Change pipeline example to run locally ([ClearML GitHub PR #642](https://github.com/allegroai/clearml/pull/642))
* Update PyTorch Lightning example for `pytorch-lightning>=v1.6.0` ([ClearML GitHub PR #650](https://github.com/allegroai/clearml/pull/650))
**Bug Fixes**
* Fix Keras model config serialization in `PatchKerasModelIO` ([ClearML GitHub issue #614](https://github.com/allegroai/clearml/issues/614))
* Fix `task.get_parameters_as_dict(cast=True)` casts `False` to `True` ([ClearML GitHub PR #622](https://github.com/allegroai/clearml/pull/622))
* Fix Fire integration is not compatible with typing library ([ClearML GitHub issue #610](https://github.com/allegroai/clearml/issues/610))
* Fix remote execution with `argparse` mutually exclusive groups raises "required" error even when no argument is required
* Fix Hydra tasks never fail and are only set to completed (fix handling return code)
* Fix `clearml-data` wildcard support
* Fix HPO randomly aborts running tasks before the time limit
* Fix matplotlib capture
* Fix issue with accessing images in projects containing `/`
* AutoScaler
* Fix resource name with a prefix matching a resource type may cause the auto-scaler to avoid spinning down idle instances
* Fix Idle workers should contain resource name and not instance type
* Fix backwards compatibility issue when using `abstractmethod`
* Matplotlib
* Fix uploading 3D plots with matplotlib plt shows 2D plot on task results page
* Fix wrong Histogram plotting using when matplotlib
* Fix PyTorch `ScriptModule` autobind
* Fix PyTorch auto-magic logging TorchScript models
* Fix forked process will not call `_at_exit` and flush all outstanding reports
* Fix matplotlib to plotly conversion fails on subplots (convert as image if figure has subplots)
* Fix Windows sub process might end up waiting forever for uploads to finish if subprocess is very shot lived
* Fix `StorageManager.get_local_copy()` returning None for a valid path in Windows
* Fix Jupyter notebook cannot be detected
* Fix PipelineController does not change node Task name, only pipeline step name
* Fix `Task.query_tasks()` specifying page size or page number
### ClearML Server 1.4.0
**New Features and Improvements**
- Allow serving with a URL path prefix ([ClearML Server GitHub issue #119](https://github.com/allegroai/clearml-server/issues/119))
- Allow overriding clearml web git url on build ([ClearML Server GitHub Pull Request #122](https://github.com/allegroai/clearml-server/pull/122))
- Ensure agent-services waits for API server to be ready ([ClearML Server GitHub issue #113](https://github.com/allegroai/clearml-server/issues/113))
- Add filter to UI experiment table "Project" column ([ClearML GitHub issue #529](https://github.com/allegroai/clearml/issues/529))
- Add filters to UI model table custom metadata columns
- Improve UI table object selection. Add buttons and context menu actions to switch between table and info panel mode
- Add task detail header to UI experiment comparison debug image section addressed in ([ClearML GitHub issue #81](https://github.com/allegroai/clearml/issues/81) ([comment](https://github.com/allegroai/clearml/issues/81#issuecomment-996841658)))
- Improve series name truncation behavior in UI scalar plots ([ClearML GitHub issue #562](https://github.com/allegroai/clearml/issues/562))
**Bug Fixes**
- Fix server limiting image and artifact upload size ([ClearML GitHub issue #606](https://github.com/allegroai/clearml/issues/606))
- Fix server unnecessarily displays deletion error message ([ClearML Server GitHub issue #112](https://github.com/allegroai/clearml-server/issues/112))
- Fix UI experiment comparison sections overly wide for many tag experiments ([ClearML GitHub issue #594](https://github.com/allegroai/clearml/issues/594))
- Fix model name display in UI pipeline run info panel
- Fix UI pipeline run info panel missing artifacts and models sections
- Fix UI pipeline run info panel displays unneeded resource utilization metrics
- Fix UI project cards missing task summary values
- Fix pipeline tag colors can't be changed in UI pipeline page
- Fix pipeline run UI page not displaying pipeline steps ([ClearML GitHub issue #618](https://github.com/allegroai/clearml/issues/618))
- Fix breadcrumb links don't navigate anywhere
- Fix horizontal scroll in UI experiment table causes column headers to jump
- Fix removing UI table column creates a blank column
- Fix internal server error (500) on `events.get_scalar_metrics_and_variants` ([ClearML Server GitHub issue #120](https://github.com/allegroai/clearml-server/issues/120))

127
docs/release_notes/ver_1_5.md
View File

@@ -1,127 +0,0 @@
---
title: Version 1.5
---
### ClearML Agent 1.5.2
**New Features and Improvements**
* Switch services agent entrypoint shell from sh to bash ([ClearML Agent GitHub issue #140](https://github.com/allegroai/clearml-agent/issues/140))
* Improve poetry support
* Add poetry cwd support ([ClearML Agent GitHub issue #138](https://github.com/allegroai/clearml-agent/issues/138))
* Add `agent.package_manager.poetry_install_extra_args` configuration option
* Do not allow request exceptions (keep retrying, throw error only on the initial login call)
**Bug Fixes**
* Fix agent update version ([ClearML Agent GitHub PR #132](https://github.com/allegroai/clearml-agent/pull/132))
* Fix login uses `GET` with payload, which breaks when trying to connect a server running in GCP
* Fix `clearml-agent build --docker` stuck on certain containers
* Fix build fails when target is relative path
* Fix pinging running task (change default to once a minute)
* Fix `_` is allowed in k8s label names
* Fix k8s glue does not delete pending pods if the tasks they represent were aborted
* Reintroduce `CLEARML_AGENT_SERVICES_DOCKER_RESTART` accidentally reverted by a previous merge
* Fix `git+ssh://` links inside installed packages not being converted properly to HTTPS-authenticated links
### ClearML Agent 1.5.1
**New Features and Improvements**
* Upgrade requirements for `attrs`, `jsonschema`, `pyparsing`, `six`, and `pyjwt` ([ClearML Agent GitHub issue #129](https://github.com/allegroai/clearml-agent/issues/129))
* Add default output URI selection to `clearml-agent init`
* Add `agent.disable_task_docker_override` configuration option to disable docker override specified in executing tasks
* Add `CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGES` env var (default `true`) to allow overriding default `system_site_packages: true`
behavior when running tasks in containers (docker mode and k8s-glue)
**Bug Fixes**
* Fix using deprecated types validator argument raises an error (deprecated even before `jsonschema` 3.0.0 and unsupported
since 4.0.0)
* Fix pip support allowing multiple pip version constraints (by default, one for < Python 3.10 and one for >= Python 3.10)
### ClearML Agent 1.5.0
**New Features and Improvements**
* Add option to crash agent on exception using `agent.crash_on_exception` configuration setting ([ClearML Agent GitHub issue #122](https://github.com/allegroai/clearml-agent/issues/122))
* Improve venv cache disabled message
* Upgrade packages for better Python 3.10 support
* Remove future package dependency (Python 2 is not supported for `clearml-agent`)
* Change default pip version used to `pip<21` for better Python 3.10 support
* Add support for operator `!=` in package version (mostly for better PyTorch resolving)
* Add support for PyTorch new `extra_index_url` repo (find the correct index url based on the cuda version, and let pip
do the rest)
* Make venv caching the default behavior
* Add support for `CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV` environment variable (see `agent.hide_docker_command_env_vars`
config option)
* Ping executing tasks to make sure the server does not consider them stale (set using the `agent.task_ping_interval_sec`
configuration option, defaults to every 120 seconds)
**Bug Fixes**
* Fix docker extra arguments showing up in configuration printout
* Fix an issue with running on Python 3.10 / 3.11
* Fix cached git token prevents cloning repository (using `agent.enable_git_ask_pass` forcing the agent to use `GIT_ASKPASS`
for user/password when cloning/fetching repositories)
* Fix setting `CLEARML_API_DEFAULT_REQ_METHOD` raises an error
* Fix `get_task_session()` may cause an old copy of the `APIClient` to be used containing a reference to the previous session
* K8s Glue
* Fix `agent.system_site_packages` is not turned on by default in k8s glue
* Make sure git_user/pass is passed to the task pod
* Remove support for `kubectl run`
### ClearML SDK 1.5.0
**New Features and Improvements**
* Add support for single value metric reporting ClearML GitHub issue ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400))
* Add support for specifying parameter sections in `PipelineDecorator` ([ClearML GitHub issue #629](https://github.com/allegroai/clearml/issues/629))
* Add support for parallel uploads and downloads (upload / download and zip / unzip of artifacts)
* Add support for specifying execution details (repository, branch, commit, packages, image) in `PipelineDecorator`
* Bump PyJWT version due to "*Key confusion through non-blocklisted public key formats*" vulnerability
* Add support for AWS Session Token (using boto3's `aws_session_token` argument)
**Bug Fixes**
* Fix `Task.get_projects()` retrieves only the first 500 results ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612))
* Fix failure to delete artifacts stored in Azure ([ClearML GitHub issue #660](https://github.com/allegroai/clearml/issues/660))
* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674))
* Fix number of unpacked values when syncing a dataset ([ClearML GitHub issue #682](https://github.com/allegroai/clearml/issues/682))
* Fix FastAI DeprecationWarning ([ClearML GitHub PR #683](https://github.com/allegroai/clearml/issues/683))
* Fix `StorageManager.download_folder()` crash
* Fix pipelines can't handle `None` return value
* Fix pre-existing pipeline raises an exception
* Fix deprecation warning in the `image_reporting` example
* Fix patches are kept binded after `Task.close()` is called
* Fix running pipeline code remotely without first running it locally (i.e. no configuration on the Task)
* Fix local task execution with empty working directory
* Fix permission check fails when using local storage folder that does not exist
* Fix pipeline `add_function_step` breaks in remote execution
* Fix wrong mimetype used for any file or folder uploaded to S3 using `StorageManager`
* Add missing default `default_cache_manager_size` in configuration files
### ClearML Server 1.5.0
**New Features and Improvements**
* Redesign UI tables info panel:
* Redesign experiment status banner
* Display all experiment tabs in single line
* Move experiment action buttons to the top right
* Add UI adherence to user specified plot dimensions ([ClearML GitHub issue #587](https://github.com/allegroai/clearml/issues/587))
* Add row highlight to experiment textual comparison UI ([ClearML GitHub issue #581](https://github.com/allegroai/clearml/issues/581))
* Add UI "Delete" action for Pipelines
* Add UI indicator for task and pipeline progress
* Add UI API credentials label "Edit" action
* Add option to create queues on-the-fly when enqueuing tasks in UI
* Add "Number of Workers" column to UI queues table
* Add pipeline results to UI dashboard search
**Bug Fixes**
* Fix UI experiment debug samples disappearing after refresh ([ClearML Server GitHub issue #136](https://github.com/allegroai/clearml-server/issues/136))
* Fix deleting tasks sometimes raises errors ([ClearML GitHub issue #632](https://github.com/allegroai/clearml/issues/632))
* Fix only partial task log shown when running on ES with multiple shards
* Fix move task to trash is not thread-safe
* Fix UI Project overview metric snapshot not showing
* Fix no progress indicator when performing off-screen selection in UI experiments table
* Fix removing long pipeline tags doesn't work
* Fix UI experiment scalar comparison graph titles are cropped with no available tooltips
* Fix UI model table status column filters incorrectly when multiple statuses are selected.
* Fix unreadable plot data causes UI to crash
* Fix UI project cards displaying incorrect task counts
* Fix hover over system tags doesn't display full tag name
* Fix `.txt` file debug sample previews are unreadable
* Fix UI table/info-panel toggle doesn't update when clicking project breadcrumbs
* Fix UI experiment debug samples leave artifacts when empty

196
docs/release_notes/ver_1_6.md
View File

@@ -1,196 +0,0 @@
---
title: Version 1.6
---
### ClearML Agent 1.6.1
**Bug Fix**
* Fix `requests` requirement lower constraint breaks backwards compatibility for Python 3.6
### ClearML Agent 1.6.0
**New Features and Improvements**
* Upgrade `requests` library ([ClearML Agent GitHub issue #161](https://github.com/allegroai/clearml-agent/issues/161))
* Add support for controlling PyTorch resolving mode using the `CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE` environment
variable and `agent.package_manager.pytorch_resolve configuration` setting with `none` (no resolving), `pip` (sets extra
index based on cuda and lets pip resolve) or `direct` (the previous parsing algorithm that does the matching and downloading). Default is `pip` ([ClearML Agent GitHub issue #152](https://github.com/allegroai/clearml-agent/issues/152))
* Add backwards compatibility in standalone mode using the `CLEARML_AGENT_STANDALONE_CONFIG_BC` environment variable
* Add `CLEARML_AGENT_DOCKER_AGENT_REPO` alias for the `FORCE_CLEARML_AGENT_REPO` environment variable
* Show a better message for `agent init` when an existing `clearml.conf` is found
* Add support for task field injection into container docker name using the `agent.docker_container_name_format_fields`
configuration setting
* Add support for adding additional labels to docker containers using the `CLEARML_AGENT_EXTRA_DOCKER_LABELS` environment variable
* Add support for setting file mode in files applied by the agent (using the `files` configuration option) using the
`mode` property
* Add support for skipping agent pip upgrade in the default k8s pod container bash script using the `CLEARML_AGENT_NO_UPDATE` environment variable
* Add support for additional pip install flags when installing dependencies using the `CLEARML_EXTRA_PIP_INSTALL_FLAGS`
environment variable and `agent.package_manager.extra_pip_install_flags` configuration option
* Add support for extra docker arguments referencing machines environment variables using the `agent.docker_allow_host_environ`
configuration option, allowing users to use `$ENV` in the task docker arguments (e.g. `-e HOST_NAME=$HOST_NAME`)
* Add support for k8s jobs execution (as opposed to only pods)
* Update default docker image versions
* Add Python 3.11 support
**Bug Fixes**
* Fix `git+ssh://` links inside installed packages not being properly converted to authenticated `https://` and vice versa
* Fix pip version required in the "Installed Packages" is now preserved and reinstalled
* Fix various agent paths not loaded correctly if an empty string or null is used (should be disabled, not converted to `.`)
* Fix docker container backwards compatibility for `API<2.13`
* Fix default docker match rules resolver (used incorrect field `"container" instead of "image")
* Fix task docker argument might be passed twice (might cause an error with flags such as `--network` and `--ipc`)
### ClearML SDK 1.6.4
**Bug Fix**
* Fix `APIClient` fails when calling `get_all` endpoints with API 2.20 (affects CLI tools such as `clearml-session`)
### ClearML SDK 1.6.3
**New Features and Improvements**
* Add option to specify an endpoint URL when creating S3 resource service ([ClearML GitHub issue #673](https://github.com/allegroai/clearml/issues/673))
* Add support for providing `ExtraArgs` to boto3 when uploading files using the `sdk.aws.s3.extra_args` configuration option
* Add support for Server API 2.20
* Add `Task.get_num_enqueued_tasks()` to get the number of tasks enqueued in a specific queue
* Add support for updating model metadata using `Model.set_metadata()`, `Model.get_metadata()`, `Model.get_all_metadata()`,
`Model.get_all_metadata_casted()`, and `Model.set_all_metadata()`
* Add `Task.get_reported_single_value()`
* Add a retry mechanism for models and artifacts upload
* Pipelines with empty configuration takes it from code
* Add support for running pipeline steps on preemptible instances
* Datasets
* Add description to Datasets
* Add wild-card support in `clearml-data`
**Bug Fixes**
* Fix dataset download ([ClearML GitHub issue #713](https://github.com/allegroai/clearml/issues/713))
* Fix lock is not released after dataset cache is downloaded ([ClearML GitHub issue #671](https://github.com/allegroai/clearml/issues/671))
* Fix deadlock might occur when using process pool large number processes ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674))
* Fix 'series' not appearing on UI when using `logger.report_table()` ([ClearML GitHub issue #684](https://github.com/allegroai/clearml/issues/684))
* Fix `Task.init()` docstring to include behavior when executing remotely ([ClearML GitHub PR #737](https://github.com/allegroai/clearml/pull/737))
* Fix `KeyError` when running remotely and no params were passed to click ([ClearML Agent GitHub issue #111](https://github.com/allegroai/clearml-agent/issues/111))
* Fix full path is stored when uploading a single artifact file
* Fix passing non-alphanumeric filename in `sdk.development.detect_with_pip_freeze`
* Fix Python 3.6 and 3.10 support
* Fix mimetype cannot be `None` when uploading to S3
* Pipelines
* Fix pipeline DAG
* Add support for pipelines with spot instances
* Fix pipeline proxy object is always resolved in main pipeline logic
* Fix pipeline steps with empty configuration should try and take it from code
* Fix wait for jobs based on local/remote pool frequency
* Fix `UniformIntegerParameterRange.to_list()` ignores min value
* Fix pipeline component returning a list of length 1
* Datasets
* Fix `Dataset.get()` does not respect `auto_create`
* Fix getting datasets fails with new ClearML Server v1.6
* Fix datasets can't be queried by project/name alone
* Fix adding child dataset to older parent dataset without stats
* Fix error when connecting an input model
* Fix deadlocks, including:
* Change thread Event/Lock to a process fork safe threading objects
* Use file lock instead of process lock to avoid future deadlocks since python process lock is not process safe
(killing a process holding a lock will Not release the lock)
* Fix `StorageManager.list()` on a local Windows path
* Fix model not created in the current project
* Fix `keras_tuner_cifar` example raises DeprecationWarning and ValueError
### ClearML Server 1.6.0
**New Features and Improvements**
* New ClearML Datasets UI pages for tracking dataset versions and exploring version lineage and contents
* Add history navigation to experiments plots UI page ([ClearML GitHub issues #81](https://github.com/allegroai/clearml/issues/81) and [#255](https://github.com/allegroai/clearml/issues/255)):
* Plots page shows last reported plot for each metric/variation combination
* Single plot view provides history navigation slider
* Add single value scalar reporting: Single value scalars are aggregated into a summary table in the experiment's scalars
UI page ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400))
* Add "show hidden projects" control ([ClearML GitHub issue #694](https://github.com/allegroai/clearml/issues/694))
* Improve UI projects page setting persistence - User's last chosen settings for recent/name sorting and team/personal
filter is saved ([ClearML GitHub issue #353](https://github.com/allegroai/clearml/issues/353))
* Improve UI presentation for object IDs:
* Display shortened ID next to task name
* Display full ID in tooltip
* Copy-to-clipboard on click
* Add "Clear all active filters" button to Projects and Pipelines details view
* Add notification for UI session expiration
* Add ID columns to UI object tables (experiments, models, etc.)
* Add "Info" section to "Details" tab of UI experiment comparison
* Add "loading" indicator for HTML debug samples
* Improve UI text editor behavior: when applicable, cursor returns to most recent edit, otherwise, cursor goes to start of text
* Maintain UI viewing mode when restoring archived items
**Bug Fixes**
* Fix experiment selection with 'Shift' key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/allegroai/clearml-server/issues/139))
* Fix UI search error message interferes with inputting search query ([ClearML Server GitHub issue #138](https://github.com/allegroai/clearml-server/issues/138))
* Fix refresh breaks UI breadcrumbs ClearML Server ([ClearML Server GitHub issue #142](https://github.com/allegroai/clearml-server/issues/142))
* Fix UI Workers and Queues page displays incorrect queue metrics
* Fix failure to publish models whose generating task was reset
* Fix listed models in UI pipeline run info panel doesn't link to model
* Fix "Load more" button disappears from UI experiment page
* Fix breadcrumb link to parent project does not navigate to the parent's project page
* Fix spaces deleted while typing query in UI search bars
* Fix UI plots not loading in experiments
* Fix UI experiment debug sample full screen failing to display multiple metrics
* Fix using search in UI tables removes custom columns
* Fix experiment debug samples remain in file server after experiment deletion
* Fix UI pipeline page sorting not working
* Fix UI scalar comparison does not display task tags
* Fix smoothed UI plots obscured by original graphs
* Fix broken task name links in UI scalar and hyperparameter comparison pages
* Fix "My work" filter not filtering out sub-projects
* Fix generic project link navigates to a blank page
* Fix UI experiment's model link inefficient use of screen real estate
* Fix network error causes UI models and experiments auto-refresh to stop working
* Fix Empty path menu appearing in UI breadcrumbs when there are no intermediary projects
* Fix long loading time in UI experiment comparison
* Fix slow performance of UI experiment plots and scalars in Chrome
* Fix Delete action not appearing in bottom bar of UI pipeline runs table archive
* Fix UI experiments not displaying dataset tasks created with `clearml` version lower than 1.6
* Fix UI projects with dataset tasks created with `clearml` version lower than 1.6 display incorrect statistics
* Fix maximizing image-plot chart in UI causes CORS error
* Fix UI experiment's "Select A Public Model" modal missing "Updated" and "Description" column data
* Fix UI experiments table's project filter displays duplicates
* Fix UI projects display incorrect statistics
* Fix clicking project with only hidden sub-projects doesn't navigate to the project's UI page
* Fix passing empty string parameter name to `Task.set_parameters` causes a 500 error
* Fix UI model page General tab's layout
* Fix using UI experiment plot navigation menu pushes WebApp's top navigation bar
* Fix example pipeline project created with "Generate Example" is empty
* Fix example pipeline created with "Generate Example" is not displayed in project dashboard when "Show hidden projects" is enabled
* Fix UI table/info-panel toggle doesn't update when switching viewing modes
### ClearML SDK 1.6.2
**Bug Fix**
* Fix format string construction sometimes causing delayed evaluation errors ([ClearML GitHub issue #706](https://github.com/allegroai/clearml/issues/706))
### ClearML SDK 1.6.1
**Bug Fixes**
* Fix `Task.get_tasks()` fails when sending `search_hidden=False`
* Fix LightGBM example shows UserWarning
### ClearML SDK 1.6.0
**New Features and Improvements**
* New Hyperparameter Optimization CLI `clearml-param-search`
* Improvements to ClearML Data
* Add support for a new ClearML Data UI in the ClearML WebApp
* Add `clearml-data` new options set-description and rename
* Add random seed control using `Task.set_random_seed()` allowing to set a new random seed for task initialization or
to disable it
* Improve error messages when failing to download an artifact
* Improve error messages when testing for permissions
**Bug Fixes**
* Fix axis range settings when logging plots
* Fix `Task.get_project()` to return more than 500 entries ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612))
* Fix pipeline progress calculation
* Fix `StorageManager.upload_folder()` returns `None` for both successful and unsuccessful uploads
* Fix script path capturing stores a relative path and not an absolute path
* Fix HTML debug samples are saved incorrectly on S3
* Fix Hydra deprecation warning in examples
* Fix missing requirement for TensorBoardX example
**Known Issues**
* When removing an image from a Dataset, its preview image won't be removed
* Moving Datasets between projects still shows the Dataset in the old project

133
docs/release_notes/ver_1_7.md
View File

@@ -1,133 +0,0 @@
---
title: Version 1.7
---
### ClearML Agent 1.7.0
**New Features**
* Add `agent.docker_args_extra_precedes_task` and `agent.protected_docker_extra_args` configuration settings to prevent
the same switch to be used by both `agent.extra_docker_args` and a Task's docker args
* Add `agent.resource_monitoring.disk_use_path` configuration option to allow monitoring a different volume than the one
containing the home folder
* Change default `agent.enable_git_ask_pass` to `true`
* Add example and support for pre-built containers including services-mode support with overrides `CLEARML_AGENT_FORCE_CODE_DIR`
and `CLEARML_AGENT_FORCE_EXEC_SCRIPT`
* Add `CLEARML_AGENT_SERVICE_TASK=1` environment variable in case of running a service task
* Add `CLEARML_AGENT_TEMP_STDOUT_FILE_DIR` to allow specifying temp dir used for storing agent log files and temporary
log files (daemon and execution)
* Update GPU stats and pynvml support
* Add git clone verbosity using `CLEARML_AGENT_GIT_CLONE_VERBOSE` environment variable
* k8s glue
* Add status reason when aborting before moving to `k8s_scheduler` queue
* When cleaning up pending pods, verify task is still aborted and pod is still pending before deleting the pod
* Set worker ID in k8s pod execution
**Bug Fixes**
* Fix `agent.package_manager.poetry_install_extra_args` are used in all Poetry commands and not just in `install` ([ClearML Agent GitHub issue #173](https://github.com/allegroai/clearml-agent/issues/173))
* Fix if process return code is SIGKILL (-9 or 137) and abort callback was called, do not mark as failed but as aborted
* Fix `agent.git_host` setting will cause git@domain URLs to not be replaced by SSH URLs since furl cannot parse them to
obtain host
* Fix an environment variable that should be set with a numerical value of 0 (i.e. end up as `"0"` or `"0.0"`) is set to
an empty string
* Fix `agent.package_manager.extra_index_url` URLs are not sanitized in configuration printout
* Fix recursion issue when deep-copying a session
* k8s glue
* Fix k8s glue configuration might be contaminated when changed during apply
* Fix `KeyError` if container does not contain the arguments field
### ClearML SDK 1.7.2
**New Features and Improvements**
* Support running Jupyter Notebook inside a git repository (repository will be referenced without uncommitted changes
and Jupyter Notebook will be stored on plain code as uncommitted changes)
* Add Jupyter Notebook fail warning
* Allow pipeline steps to return string paths without them being treated as a folder artifact and zipped ([ClearML GitHub issue #780](https://github.com/allegroai/clearml/issues/780))
* Remove `future` from Python 3 requirements
**Bug Fixes**
* Fix exception raised when using `ThreadPool` ([ClearML GitHub issue #790](https://github.com/allegroai/clearml/issues/790))
* Fix Pyplot/Matplotlib binding reports incorrect line labels and colors ([ClearML GitHub issue #791](https://github.com/allegroai/clearml/issues/791))
* Pipelines
* Fix crash when running cloned pipeline that invokes a step twice ([ClearML GitHub issue #769](https://github.com/allegroai/clearml/issues/769))
* Fix pipeline argument becomes `None` if default value is not set
* Fix `retry_on_failure` callback does nothing when specified on `PipelineController.add_step()`
* Fix pipeline clone logic
* Jupyter Notebook
* Fix support for multiple Jupyter servers running on the same machine
* Fix issue with old/new notebook packages installed
* Fix local cache with access rules disabling partial local access
* Fix `Task.upload_artifact()` fails uploading pandas `DataFrame`
* Fix relative paths in examples ([ClearML GitHub PR #787](https://github.com/allegroai/clearml/issues/787)
### ClearML Server 1.7.0
**New Features and Improvements**
* Add “Sync comparison” to UI experiment debug samples comparison: Control metric/iteration for all compared experiments ([ClearML GitHub issue #691](https://github.com/allegroai/clearml/issues/691))
* Support serving UI from a non-root path of the ClearML Server ([ClearML Helm Charts issue #101](https://github.com/allegroai/clearml-helm-charts/issues/101) and [ClearML Server issue #135](https://github.com/allegroai/clearml-server/issues/135))
* Add UI option for hiding “secret” experiment container arguments ([ClearML Server GitHub issue #146](https://github.com/allegroai/clearml-server/issues/146))
* Add UI tables switch to detail mode through double-click ([ClearML Server GitHub issue #134](https://github.com/allegroai/clearml-server/issues/134))
* Add customizable user activity timeout for UI logout
* Add UI navigation from experiment comparison back to originating experiment table
* Improve UI scalar comparison graph configuration - Persist user's choice for viewing graph data.
* Add model IDs display in UI experiment artifact tab
* Add dataset description to UI dataset information display
**Bug Fixes**
* Fix UI experiment Hyperparameter tab's inefficient use of screen real estate ([ClearML GitHub issue #705](https://github.com/allegroai/clearml/issues/705))
* Fix navigating to an archived experiment's configuration causes UI to return to non-archived view ([ClearML Server GitHub issue #148](https://github.com/allegroai/clearml-server/issues/148))
* Fix metric storage failure when large amount of metrics logged
* Fix UI plots downloaded as PNGs don't contain legends
* Fix UI plot colors can't be changed when colors are specified in code
* Fix experiment table hyperparameter column sorting fails when hyperparameter includes "."
* Fix artifacts aren't deleted when experiment is reset
* Fix UI project cards displaying incorrect task counts when "Show Hidden Projects" is enabled
* Fix changing UI filter to "My Work" within a project page causes main project to become undefined
* Fix clicking project with only hidden sub-projects doesn't navigate to the project's UI page
* Fix multiple UI projects can have same name
* Fix UI experiment plot legends disappear when plots take up whole row
* Fix cloning an experiment into a different project does not navigate to the new project
* Fix queue menu automatically opens when UI enqueue modal is opened
* Fix can't dequeue experiments from deleted queue.
* Fix UI Dataset content previews displays "No preview" message while previews are loading
### ClearML SDK 1.7.1
**New Features and Improvements**
* Add callback option for pipeline step retry
**Bug Fixes**
* Fix Python Fire binding
* Fix Dataset failing to load helper packages should not crash
* Fix `Dataset.get_local_copy()` is allowed for a non-finalized dataset
* Fix `Task.upload_artifact()` does not upload empty lists/tuples
* Fix pipeline retry mechanism interface
* Fix `Python <3.5` compatibility
* Fix local cache warning (should be a debug message)
### ClearML SDK 1.7.0
**New Features and Improvements**
* ClearML Data: Support providing list of links
* Upload artifacts with a custom serializer ([ClearML GitHub issue #689](https://github.com/allegroai/clearml/issues/689))
* Allow user to specify extension when using custom serializer functions (for artifacts)
* Skip server URL verification in `clearml-init` wizard process
* When calling `Dataset.get()` without "alias" field, tell user that he can use alias to log it in the UI
* Add support for mmcv for logging models
* Add support for Azure and GCP storage in `task.setup_upload()`
* Support pipeline retrying tasks which are failing on suspected non-stable failures
* Better storage (AWS, GCP) internal load balancing and configurations
* Add `Task.register_abort_callback`
**Bug Fixes**
* Allow getting datasets with non-semantic versioning ([ClearML GitHub issue #776](https://github.com/allegroai/clearml/issues/776))
* Fix interactive plots (instead of a generated png)
* Fix Python 2.7 support
* Fix clearml datasets `list` functionality
* Fix `Dataset.init` to not modify task (to `Dataset.create`)
* Fix failure with large files upload on HTTPS
* Fix 3d plots with plt shows 2d plot on task results page
* Fix uploading files with project's `default_upload_destination` ([ClearML GitHub issue #734](https://github.com/allegroai/clearml/issues/734))
* Fix broken Matplotlib reporting - using logarithmic scale breaks reporting
* Fix wildcard support in `clearml-data` CLI
* Fix `report_histogram` - does not show "horizontal" orientation ([ClearML GitHub issue 699](https://github.com/allegroai/clearml/issues/699))
* Fix table reporting - when using `logger.report_table(title, series, iteration, etc)`, the `series` arg does not appear in UI ([ClearML GitHub issue 684](https://github.com/allegroai/clearml/issues/684))
* Fix artifacts (and models) use task's original name and not new name
* Fix very long filenames from S3 can't be downloaded (with `get_local_copy()`)
* Fix overwrite of existing output models on pipeline task with `monitor_models` ([ClearML GitHub issue #757](https://github.com/allegroai/clearml/issues/757))

136
docs/release_notes/ver_1_8.md
View File

@@ -1,136 +0,0 @@
---
title: Version 1.8
---
### ClearML Agent 1.8.1
**New Features**
* Add option to set daemon polling interval ([ClearML Agent GitHub PR #197](https://github.com/allegroai/clearml-agent/pull/197))
* Add Python 3.12 support
**Bug Fixes**
* Fix git pulling on cached invalid git entry. On error, re-clone the entire repository again (enable using `agent.vcs_cache.clone_on_pull_fail: true`)
* Fix conda env should not be cached if installing into base conda or conda existing env
* Fix cached repositories not passing user/token when pulling
* Fix when disabling vcs cache do not add vcs mount point to container
### ClearML Agent 1.8.0
**New Features**
* Add `CLEARML_AGENT_FORCE_POETRY` environment variable to allow forcing poetry even when using pip requirements manager
* Add `CLEARML_AGENT_FORCE_TASK_INIT` environment variable to allow runtime patching of script even if no repository is
specified and the code is running a preinstalled docker
* Improve venv cache handling:
* Add `FileLock` readonly mode, default is write mode (i.e. exclusive lock, preserving behavior)
* Add venv cache now uses readonly lock when copying folders from venv cache into target folder. This enables multiple read, single write operation
* Do not lock the cache folder if we do not need to delete old entries
* Add `agent.venvs_cache.lock_timeout` to control the venv cache folder lock timeout (in seconds, default 30)
* Add protection for `truncate()` call
* Move configuration sanitization settings to the default config file
* Add queue ID report before pulling task
* Improve GPU monitoring for MIGs
**Bug Fixes**
* Use correct Python version in Poetry init ([ClearML Agent GitHub PR #179](https://github.com/allegroai/clearml-agent/pull/179))
* Fix queue handling in `K8sIntegration` and `k8s_glue_example.py` ([ClearML Agent GitHub PR #183](https://github.com/allegroai/clearml-agent/pull/183))
* Fix `FileNotFoundException` crash in `find_python_executable_for_version` ([ClearML Agent GitHub issue #164](https://github.com/allegroai/clearml-agent/issues/164))
* Fix delete temp console pipe log files after Task execution is completed (important for long-lasting services agents to avoid collecting temp files on host machine)
* Fix `agent.enable_git_ask_pass` does not show in configuration dump
* Fix `pip` is returned as a pip version if no value exists in `agent.package_manager.pip_version`
* Fix Python 3.12 support by removing `distutil` imports
* Fix `IOError` on file lock when using shared folder
* Fix torch resolver settings applied to `PytorchRequirement` instance are not used
* Fix comment lines (starting with `#`) are not ignored in docker startup bash script
* Fix dynamic GPU sometimes misses the initial print
### ClearML SDK 1.8.3
**Bug Fixes**
* Set GCS credentials to `None` if invalid service account credentials are provided ([ClearML GitHub issue #840](https://github.com/allegroai/clearml/issues/840))
* Fix a sync issue when loading deferred configuration
### ClearML SDK 1.8.2
**New Features and Improvements**
* Added `VCS_ENTRY_POINT` environment variable that overrides ClearML's entrypoint auto-detection
**Bug Fixes**
* Fix all parameters returned from a pipeline are considered strings
* Fix `Task.set_parameters()` does not add parameter type when parameter exists but does not have a type
### ClearML Server 1.8.0
**New Features and Improvements**
* Support `/` delimited project names for on-the-fly nested project creation in UI operations ([ClearML Server GitHub issue #151](https://github.com/allegroai/clearml-server/issues/151))
* Add keyboard shortcut iteration navigation in full screen view of debug samples and plots
* Add "Don't show again" option for UI notification when archiving tasks ([ClearML Server GitHub issue #149](https://github.com/allegroai/clearml-server/issues/149))
* Add task reset/delete modal automatically closes when no additional information is provided
* Add parent project name in UI dataset and pipeline cards
* Move UI plot legends to bottom of plot
**Bug Fixes**
* Fix task artifacts not deleted from file server when task deleted via UI ([ClearML GitHub issue #801](https://github.com/allegroai/clearml/issues/801))
* Fix deleting pipeline projects via UI fails ([ClearML GitHub issue #798](https://github.com/allegroai/clearml/issues/798))
* Fix multiple selection with `Shift` key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/allegroai/clearml-server/issues/139))
* Fix disappearing user filter in UI "All Experiments" table ([ClearML Web GitHub issue #33](https://github.com/allegroai/clearml-web/issues/33))
* Fix UI Dashboard missing My Work / Team Work filter
* Fix navigating to Scalars and Plots tabs in UI experiment comparison freezes browser
* Fix project stats do not exclude dataset and annotation tasks
* Fix subproject with no experiments displayed as "undefined" in parent project page
* Fix maximizing UI combined scalar plots displays split scalar plots
* Fix adding description to UI dataset does not work
* Fix empty "parent" field in task cloned in UI
* Fix running UI application instances can be deleted
* Fix queue option list does not open in UI enqueue modal
### ClearML SDK 1.8.1
**New Features and Improvements**
* Raise error on failed uploads ([ClearML GitHub issue #810](https://github.com/allegroai/clearml/issues/819))
* Add hyperdataset examples ([ClearML GitHub PR #823](https://github.com/allegroai/clearml/commit/f6b9efe54e1246adba4036c56bc6e8a0bdb99948))
* Change `report_event_flush_threshold` default to 100
* Add `ModelInfo.weights_object()` to store callback access to the actual model object being stored (valid for both
pre/post save calls, otherwise `None`)
* Support `num_workers` in dataset operation
* Support max connections setting for Azure storage using the `sdk.azure.storage.max_connection` configuration option
**Bug Fixes**
* Fix clearml logger default level cannot be changed ([ClearML GitHub issue #741](https://github.com/allegroai/clearml/issues/741))
* Fix Hydra doesn't get overridden information from ClearML ([ClearML GitHub issue #751](https://github.com/allegroai/clearml/issues/751))
* Fix `StorageManager.list(“s3://..”, with_metadata=True)` doesn't work
* Fix `ModelsList.keys()` is missing
* Fix `CLEARML_DEFERRED_TASK_INIT=1` doesn't work
* Fix default API method does not work when set in configuration
### ClearML SDK 1.8.0
**New Features and Improvements**
* Add tarfile member sanitization to `extractall()`([ClearML GitHub PR #803](https://github.com/allegroai/clearml/pull/803))
* Add `Task.delete_artifacts()` with `raise_on_errors` argument ([ClearML GitHub issue #805](https://github.com/allegroai/clearml/issues/805))
* Add CI/CD example ([ClearML GitHub PR #815](https://github.com/allegroai/clearml/pull/815))
* Limit number of `_serialize` requests when adding list of links with `add_external_files()` ([ClearML GitHub issue #813](https://github.com/allegroai/clearml/issues/813))
* Add support for connecting Enum values as parameters
* Improve Colab integration (store entire colab, not history)
* Add `clearml.browser_login` to authenticate browser online sessions such as Colab, Jupyter Notebooks etc.
* Remove `import_bind` from stack trace of import errors
* Add `sdk.development.worker.report_event_flush_threshold` configuration option to control the number of events to trigger a report
* Return stub object from `Task.init()` if no `clearml.conf` file is found
* Improve manual model uploading example
* Remove deprecated demo server
**Bug Fixes**
* Fix passing `compression=ZIP_STORED` (or 0) to `Dataset.upload()` uses `ZIP_DEFLATED` and overrides the user-supplied
argument ([ClearML GitHub PR #812](https://github.com/allegroai/clearml/pull/812))
* Fix `unique_selector` is not applied properly on batches after the first batch. Remove default selector value since
it does not work for all event types (and we always specify it anyway)
* Fix `clearml-init` colab detection
* Fix cloning pipelines run with `start_locally()` doesn't work
* Fix if project has a default output uri there is no way to disable it in development mode (manual), allow passing
`output_uri=False` to disable it
* Fix git remote repository detection when remote is not "origin"
* Fix reported images might not all be reported when waiting to complete the task
* Fix `Dataset.get_local_copy()` deletes the source archive if it is stored locally
* Fix too many parts will cause preview to inflate Task object beyond its 16MB limit - set a total limit of 320kbs
* Fix media preview is created instead of a table preview
* Fix `task.update_output_model()` should always upload local models to a remote server
* Fix broken pip package might mess up requirements detection

166
docs/release_notes/ver_1_9.md
View File

@@ -1,166 +0,0 @@
---
title: Version 1.9
---
### ClearML Agent 1.9.1
**New Features and Improvements**
* Add default pip version support for Python 3.12
### ClearML Agent 1.9.0
**New Features and Improvements**
* Add `NO_DOCKER` flag to `clearml-agent-services` entrypoint ([ClearML Agent GitHub PR #206](https://github.com/allegroai/clearml-agent/pull/206))
* Use `venv` module if `virtualenv` is not supported
* Find the correct python version when using a pre-installed python environment
* Add `/bin/bash` support in the task's `script.binary` property
* Add support for `.ipynb` script entry files (install nbconvert in runtime, convert file to python and execute the
python script). Includes `CLEARML_AGENT_FORCE_TASK_INIT` patching of `.ipynb` files (post-python conversion)
* Add `CLEARML_MULTI_NODE_SINGLE_TASK` (values -1, 0, 1, 2) for easier multi-node single Task workloads
* Add default docker `agent.default_docker.match_rules` configuration option (note: `matching_rules` are ignored if `--docker container` is passed in command line)
* Add `-m module args` in script entry now supports standalone script. Standalone script is placed in a file specified
by the `working_dir` setting in the `<dir>:<target_file>` format (e.g. `:standalone.py`), or in `untitled.py` if not specified
* Add `K8S_GLUE_POD_USE_IMAGE_ENTRYPOINT` env var to allow running k8s pods without overriding the image entrypoint
(useful for agents using prebuilt images in k8s)
* Add venv cache mount override for non-root containers (use: `agent.docker_internal_mounts.venvs_cache`)
* Add `/bin/bash -c "command"` support. Task binary should be set to `/bin/bash` and `entry_point` should be set to `-c command`
* Add support for tasks containing only bash script or python module command
* Add support for skipping container apt installs using `CLEARML_AGENT_SKIP_CONTAINER_APT` env var in k8s
**Bug Fixes**
* Fix git fetch did not update new tags ([ClearML Agent GitHub issue #209](https://github.com/allegroai/clearml-agent/issues/209))
* Fix file mode should be optional in configuration `files` section
* Fix `-m module $env` to support parsing `$env` before launching
* Fix setting tasks that were just marked as `aborted` to `started` - only force task to `started` after dequeueing it,
otherwise do nothing
* Fix slurm multi-node rank detection
* Fix passing only `--docker` (i.e. no default container image) when using `--dynamic-gpus` feature
* Fix logger object used even if `None`
* Fix a race condition where in rare conditions popping a task from a queue that was aborted did not set it to `started`
before the watchdog killed it (not applicable in k8s/slurm)
* Fix multi-node support to only send `pip freeze` update, only set task as `started`, and only update task status on exit for RANK 0
* Fix do not cache venv cache if venv/python skip env var was set
* Fix use same state transition in k8s if supported by the server (instead of stopping the task before re-enqueue)
* Fix failed Task in services mode logged as `User aborted` instead of `failed`. Add Task state reason string
* Fix remove task from pending queue and set to `failed` in k8s when applying the pod template fails
### ClearML SDK 1.9.3
**Bug Fixes**
* Fix broken `Task._get_status()`, which was breaking `clearml-session` in the latest version
* Fix path substitution, making it possible to store unsubstituted URL for models ([ClearML GitHub PR #935](https://github.com/allegroai/clearml/pull/935))
### ClearML SDK 1.9.2
**New Features and Improvements**
* Support parsing queue name when providing execution queue in pipelines code ([ClearML GitHub PR #857](https://github.com/allegroai/clearml/pull/857))
* Ignore `None` values for keys in the `click` argument parser ([ClearML GitHub issue #902](https://github.com/allegroai/clearml/issues/902))
* Improve docstrings for `Task.mark_completed()` and `Task.close()` (ClearML GitHub PRs [#920](https://github.com/allegroai/clearml/pull/920) and [#921](https://github.com/allegroai/clearml/pull/921))
* Add pre/post execution callbacks to pipeline steps through `@PipelineDecorator.component`
* Add status-change callback to pipeline steps through `PipelineController.add_step()`, `PipelineController.add_function_step()`,
and `@PipelineDecorator.component`
**Bug Fixes**
* Fix missing debug samples when reporting using TensorBoard ([ClearML GitHub issue #923](https://github.com/allegroai/clearml/issues/923))
* Fix wrong Jupyter API token during repository detection ([ClearML GitHub PR #904](https://github.com/allegroai/clearml/pull/904))
* Fix typo in the warning for very large git diffs ([ClearML GitHub PR #932](https://github.com/allegroai/clearml/pull/932))
* Fix pipelines from tasks don't propagate `parameter_override` values in `PipelineController.add_step()`
* Fix folders and files uploaded to S3 and Azure with `StorageManager.upload_file()` receive wrong MIME types
* Fix the CSV file preview in Datasets
* Fix `Task.connect_configuration()` doesn't work with non-string dictionary keys
* Fix `lightgbm_example` deprecation warning
* Fix potential race condition in `get_or_create_project()`
### ClearML SDK 1.9.1
**New Features and Improvements**
* Add signature version to `boto3` configuration ([ClearML GitHub issue #883](https://github.com/allegroai/clearml/issues/883))
* Allow requesting custom token expiration using the `api.auth.req_token_expiration_sec` configuration setting
* Add Python 3.11 support
**Bug Fixes**
* Fix `UniformParameterRange.to_list` throws error when step size is not defined ([ClearML GitHub issue #858](https://github.com/allegroai/clearml/issues/858))
* Fix `StorageManager.list()` does not return size metadata ([ClearML GitHub issue #865](https://github.com/allegroai/clearml/issues/865))
* Fix storage with path substitutions ([ClearML GitHub issue #825](https://github.com/allegroai/clearml/issues/825))
* Fix extras in ClearML installation prevents clearml from being included in requirements ([ClearML GitHub issue #867](https://github.com/allegroai/clearml/issues/867))
* Fix metadata set on an uploaded model object is not accessible ([ClearML GitHub issue #890](https://github.com/allegroai/clearml/issues/890))
* Fix Azure storage upload not working ([ClearML GitHub issue #868](https://github.com/allegroai/clearml/issues/868))
* Fix `task.connect` list of dicts parsed incorrectly in remote
* Fix casting `None` to `int` fails uploads and permission checks
* Fix numpy 1.24 support
* Fix `clearml-data` previews are saved on file server even when `output_uri` is specified
* Fix connecting a dictionary to task sometimes raises an exception
* Fix authentication headers are not set on substituted fileserver URLs
* Fix `Task.get_project_id()` cannot find hidden projects
* Fix TriggerScheduler docstrings ([ClearML GitHub issue #881](https://github.com/allegroai/clearml/issues/881))
### ClearML Server 1.9.2
**Bug Fixes**
* Fix "Unable to load image" error in UI experiment debug samples after entering storage credentials ([ClearML Web GitHub issue #46](https://github.com/allegroai/clearml-web/issues/46))
* Fix incorrect default project name displayed in UI "Clone Experiment" modal ([ClearML Server GitHub issue #171](https://github.com/allegroai/clearml-server/issues/171))
* Fix UI experiment debug sample viewer iteration slider not working
* Fix UI experiment console log jumps backwards unnecessarily on refresh
* Fix UI published report is not expandable
* Fix UI published report's description is not editable
* Fix UI experiment-reset popup displays incorrect experiment count
### ClearML Server 1.9.1
**New Features and Improvements**
* Add collapsible description panel for UI reports
* Add in-app markdown guide for UI reports
**Bug Fixes**
* Fix can't generate new pipeline runs from UI ([ClearML Server GitHub issue #169](https://github.com/allegroai/clearml-server/issues/169))
* Fix plot legend texts overlap in UI reports
* Fix UI embedded plot colors are sometimes obscure
* Fix `ctrl z` doesn't undo codeblocks in UI reports
* Fix "Getting Started" popup wrongfully showing when enqueuing tasks
### ClearML SDK 1.9.0
**New Features and Improvements**
* Add `r` prefix to `re.match()` strings ([ClearML GitHub issue #834](https://github.com/allegroai/clearml/issues/834))
* Add `path_substitution` to `clearml.conf` example file ([ClearML GitHub PR #842](https://github.com/allegroai/clearml/pull/842))
* Clarify `deferred_init` usage in `Task.init()` ([ClearML GitHub issue #855](https://github.com/allegroai/clearml/issues/855))
* Add pipeline decorator argument to control docker image ([ClearML GitHub issue #856](https://github.com/allegroai/clearml/issues/856))
* Add `StorageManager.set_report_upload_chunk_size()` and `StorageManager.set_report_download_chunk_size()` to set chunk
size for upload and download
* Add `allow_archived` argument in `Task.get_tasks()`
* Support querying model metadata in `Model.query_models()`
* Add `Dataset.set_metadata()` and `Dataset.get_metadata()`
* Add `delete_from_storage` (default `True`) to `Task.delete_artifacts()`
**Bug Fixes**
* Fix jsonargparse and pytorch lightning integration broken for remote execution ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403))
* Fix error when using `TaskScheduler` with `limit_execution_time` ([ClearML GitHub issue #648](https://github.com/allegroai/clearml/issues/648))
* Fix dataset not synced if the changes are only modified files ([ClearML GitHub issue #822](https://github.com/allegroai/clearml/issues/822))
* Fix `StorageHelper.delete()` does not respect path substitutions ([ClearML GitHub issue #825](https://github.com/allegroai/clearml/issues/825))
* Fix can't write more than 2 GB to a file
* Fix `StorageManager.get_file_size_bytes()` returns `ClientError` instead of `None` for invalid S3 links
* Fix Dataset lineage view is broken with multiple dataset dependencies
* Fix `tensorflow_macos` support
* Fix crash when calling `task.flush(wait_for_uploads=True)` while executing remotely
* Fix `None` values get casted to empty strings when connecting a dictionary
### ClearML Server 1.9.0
**New Features and Improvements**
* New ClearML Reports UI: Create and share rich MarkDown documents supporting embeddable online content ([ClearML GitHub issue #839](https://github.com/allegroai/clearml/issues/839))
* Add user option to not show example content in the UI ([ClearML GitHub issue #774](https://github.com/allegroai/clearml/issues/774))
* Add tag filter to UI Pipeline, Dataset, and Hyper-Dataset pages
* Extend UI experiment comparison up to 100 experiments
* Allow project default output destination S3 URLs to include dots (`.`) and hyphens (`-`)
**Bug Fixes**
* Fix long experiment names break UI experiment comparison scalar plot display ([ClearML GitHub issue #166](https://github.com/allegroai/clearml-server/issues/166))
* Fix plot legend definitions set by SDK don't override UI default settings
* Fix comparing experiments with read-only components causes error
* Fix UI model link doesn't preserve double spaces, breaking the URI
* Fix tasks/models/datasets can't be moved to UI root project
* Fix UI tag color list doesn't display tags
* Fix UI dashboard search returns results from team's work when "My Work" filter is enabled
* Fix UI experiment console does not display complete text when large text is reported

2
docusaurus.config.js
View File

@@ -101,7 +101,7 @@ module.exports = {
{
label: 'Release Notes',
to: '/docs/release_notes/ver_1_16',
to: '/docs/release_notes/clearml_server/open_source/ver_1_16',
},
{
label: 'Community Resources',

87
sidebars.js
View File

@@ -263,17 +263,82 @@ module.exports = {
{'Web UI': ['guides/ui/building_leader_board','guides/ui/tuning_exp']}
],
rnSidebar: {
'Release Notes': [
'release_notes/ver_1_16', 'release_notes/ver_1_15',
'release_notes/ver_1_14', 'release_notes/ver_1_13', 'release_notes/ver_1_12', 'release_notes/ver_1_11',
'release_notes/ver_1_10', 'release_notes/ver_1_9', 'release_notes/ver_1_8', 'release_notes/ver_1_7',
'release_notes/ver_1_6', 'release_notes/ver_1_5', 'release_notes/ver_1_4', 'release_notes/ver_1_3',
'release_notes/ver_1_2','release_notes/ver_1_1', 'release_notes/ver_1_0', 'release_notes/ver_0_17',
'release_notes/ver_0_16', 'release_notes/ver_0_15', 'release_notes/ver_0_14', 'release_notes/ver_0_13',
'release_notes/ver_0_12', 'release_notes/ver_0_11', 'release_notes/ver_0_10', 'release_notes/ver_0_9',
],
},
rnSidebar: [
{'Server': [
{
'Open Source':
[
'release_notes/clearml_server/open_source/ver_1_16',
'release_notes/clearml_server/open_source/ver_1_15', 'release_notes/clearml_server/open_source/ver_1_14',
'release_notes/clearml_server/open_source/ver_1_13', 'release_notes/clearml_server/open_source/ver_1_12',
'release_notes/clearml_server/open_source/ver_1_11', 'release_notes/clearml_server/open_source/ver_1_10',
'release_notes/clearml_server/open_source/ver_1_9', 'release_notes/clearml_server/open_source/ver_1_8',
'release_notes/clearml_server/open_source/ver_1_7', 'release_notes/clearml_server/open_source/ver_1_6',
'release_notes/clearml_server/open_source/ver_1_5', 'release_notes/clearml_server/open_source/ver_1_4',
'release_notes/clearml_server/open_source/ver_1_3', 'release_notes/clearml_server/open_source/ver_1_2',
'release_notes/clearml_server/open_source/ver_1_1', 'release_notes/clearml_server/open_source/ver_1_0',
'release_notes/clearml_server/open_source/ver_0_17', 'release_notes/clearml_server/open_source/ver_0_16',
'release_notes/clearml_server/open_source/ver_0_15', 'release_notes/clearml_server/open_source/ver_0_14',
'release_notes/clearml_server/open_source/ver_0_13', 'release_notes/clearml_server/open_source/ver_0_12',
'release_notes/clearml_server/open_source/ver_0_11', 'release_notes/clearml_server/open_source/ver_0_10',
]
},
{
'Enterprise':
[
'release_notes/clearml_server/enterprise/ver_3_22', 'release_notes/clearml_server/enterprise/ver_3_21',
'release_notes/clearml_server/enterprise/ver_3_20',
]
}
]},
{'SDK': [
{
'Open Source':
[
'release_notes/sdk/open_source/ver_1_16', 'release_notes/sdk/open_source/ver_1_15',
'release_notes/sdk/open_source/ver_1_14', 'release_notes/sdk/open_source/ver_1_13',
'release_notes/sdk/open_source/ver_1_12', 'release_notes/sdk/open_source/ver_1_11',
'release_notes/sdk/open_source/ver_1_10', 'release_notes/sdk/open_source/ver_1_9',
'release_notes/sdk/open_source/ver_1_8', 'release_notes/sdk/open_source/ver_1_7',
'release_notes/sdk/open_source/ver_1_6', 'release_notes/sdk/open_source/ver_1_5',
'release_notes/sdk/open_source/ver_1_4', 'release_notes/sdk/open_source/ver_1_3',
'release_notes/sdk/open_source/ver_1_2', 'release_notes/sdk/open_source/ver_1_1',
'release_notes/sdk/open_source/ver_1_0', 'release_notes/sdk/open_source/ver_0_17',
'release_notes/sdk/open_source/ver_0_16', 'release_notes/sdk/open_source/ver_0_15',
'release_notes/sdk/open_source/ver_0_14', 'release_notes/sdk/open_source/ver_0_13',
'release_notes/sdk/open_source/ver_0_12', 'release_notes/sdk/open_source/ver_0_11',
'release_notes/sdk/open_source/ver_0_10', 'release_notes/sdk/open_source/ver_0_9',
]
},
{
'Enterprise':
[
'release_notes/sdk/enterprise/ver_3_12', 'release_notes/sdk/enterprise/ver_3_11',
'release_notes/sdk/enterprise/ver_3_10',
]
}
]},
{'ClearML Agent':
[
'release_notes/clearml_agent/ver_1_9', 'release_notes/clearml_agent/ver_1_8',
'release_notes/clearml_agent/ver_1_7', 'release_notes/clearml_agent/ver_1_6',
'release_notes/clearml_agent/ver_1_5', 'release_notes/clearml_agent/ver_1_4',
'release_notes/clearml_agent/ver_1_3', 'release_notes/clearml_agent/ver_1_2',
'release_notes/clearml_agent/ver_1_1', 'release_notes/clearml_agent/ver_1_0',
'release_notes/clearml_agent/ver_0_17', 'release_notes/clearml_agent/ver_0_16',
'release_notes/clearml_agent/ver_0_15', 'release_notes/clearml_agent/ver_0_14',
'release_notes/clearml_agent/ver_0_13', 'release_notes/clearml_agent/ver_0_12',
]
},
{'ClearML Serving':
[
'release_notes/clearml_serving/ver_1_3', 'release_notes/clearml_serving/ver_1_2',
'release_notes/clearml_serving/ver_1_1', 'release_notes/clearml_serving/ver_1_0',
]
}
],
sdkSidebar: [
'references/sdk/task',
'references/sdk/logger',