--- title: Version 0.17 --- ## ClearML 0.17.5 **Features** - Add `force_download` argument to `Artifact.get()` and `Artifact.get_local_copy()` ([clearml GitHub Issue 319](https://github.com/allegroai/clearml/issues/319)). - Support all reporting using subprocesses instead of threads (default `sdk.development.report_use_subprocess=True`). - Improve Datasets support: * Add `clearml-data publish` to allow publishing a dataset task. * Add `clearml-data sync`, which can now create, sync and close a dataset with a single command. * Allow Dataset to be created on a running Task. * Add `dataset_tags` argument to `Dataset.get()`. * Add `Dataset.get_logger()`. - Add `Task.add_requirements()` support for specifying version `<>=~` etc. - Add `StorageManager.upload_folder()` and `StorageManager.download_folder()`. - Add progress report logging for `StorageHelper.upload_from_stream()`, `StorageHelper.upload()` and `StorageHelper.upload_from_stream()`. - Add jupyter auto-magic - store Jupyter Notebook as an artifact on the Task (default `sdk.development.store_jupyter_notebook_artifact=True`). - Add upload HTML preview of Jupyter Notebook as an artifact. - Add `PipelineController` disable clone base task option. - Add links to Tasks in optimization summary table (not just Task IDs). - Add support for datetime in scatter plots + matplotlib support. - Improve plotly value type conforming. - Improve PyTorch `DataLoader` speed ([clearml GitHub Issue 207](https://github.com/allegroai/clearml/issues/207)). - Update Auto Scaler default values and configuration. - Examples: * Add Hydra example. * Add artifacts retrieval example. * Update various examples. **Bug Fixes** - Fix warning or error message if requirements parsing failed ([clearml GitHub Issue 291](https://github.com/allegroai/clearml/issues/291)). - Fix pytorch-lighting multi node store ([clearml GitHub Issue 292](https://github.com/allegroai/clearml/issues/292)). - Fix strip remote diff ([clearml GitHub Issue 295](https://github.com/allegroai/clearml/issues/295)). - Fix Python package detection `sklearn` -> `scikit-learn` ([clearml GitHub Issue 296](https://github.com/allegroai/clearml/issues/296)). - Fix argparse issues: * Fix argparse with `[None]` in default parameter ([clearml GitHub Issue 297](https://github.com/allegroai/clearml/issues/297)). * Fix parsing of arguments in scientific notation ([clearml GitHub Issue 313](https://github.com/allegroai/clearml/issues/313)). * Fix argparser logging always captures defaults (Windows only, ignored cmd). * Fix argparse `nargs` passed in command line `--nargs 1 2` should be stored as `[1, 2]` not as `['1', '2']`. * Fix support for nonstandard argparse with default value that is not of defined type. * Fix server updated with the argparse in remote before Task.init() is called (respect skipped args). - Fix Dataset support: * Fix `Dataset.remove_files()` can't find files when files are in dataset root ([clearml GitHub Issue 303](https://github.com/allegroai/clearml/issues/303)). * Fix closing a dataset with only files removed. * Fix Dataset generate removed / modified / added of summary table. - Fix Hydra multi-run support ([clearml GitHub Issue 306](https://github.com/allegroai/clearml/issues/306)). - Fix TF/TensorBoard support: * Fix TensorBoard multiple `Task.init()` / `Task.close()` calls within the same process ([clearml GitHub Issue 312](https://github.com/allegroai/clearml/issues/312)). * Fix TensorBoard 2+ `pr_curve`. * Fix TF `pr_curve` should not be inverted. * Fix TF +2.3 mixed eager mode execution summary metrics not reported. * Fix TF bind keyboard interrupt. * Fix TF 2.4 keras load / save model. - Fix `clearml-task`: * Fix error when script cannot be found. * Fix `--docker` flag not passed. * Fix patching local git diff. - Fix `clearml-data`: * Fix `clearml-data sync` requires `--name`. * Fix missing required argument `--files` in `clearml-data remove`. - Fix `Task.execute_remotely()` from Jupyter Notebook. - Fix populate Task called from Jupyter Notebook (use `Task.create(packages=True)` to auto populate based on locally installed packages). - Fix plotly plot with numpy containing `NaN` / `datetime`. - Fix matplotlib with Agg backend (or in remote execution). - Fix trying to upload model file as a folder (automatically package the folder). - Fix broken packages on package `importlib` detection failed the entire requirements detection. - Fix `Task.connect(object)` should always return the same object instance. - Fix `Task.create()` with repo and script that exists locally. - Fix crash in case `Logger.get_logger()` cannot get the file name. - Fix exception at exit in Python 3.8+ on macOS / Windows. - Fix make pipeline summary table link to Task step logs. - Fix Hydra 1.1 support (argparse description). - Fix close task after logger is closed. - Fix `Task.set_base_docker()` in remote execution. - Fix artifact preview limit to 64Kb. - Fix JupyterLab Notebook detection. - Fix Python 2.7 support. ## ClearML Agent 0.17.2 **Features** - Add virtual environment caching: * Supports venv caching both in standard and docker mode. * Configurable using the `agent.venvs_cache` configuration section. * Disabled by default, enable [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L101). - Add support for `--services-mode` with venvs. - Add `agent.force_git_ssh_user` configuration value (default `git`, see [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L29)) ([clearml-agent GitHub Issue 42](https://github.com/allegroai/clearml-agent/issues/42)). - Add `agent.ignore_requested_python_version` configuration option for multi Python environments (default `false`). - Add `agent.enable_task_env` configuration option to set the OS environment based on the Environment section of the Task (default `false`, see [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L151)). - K8s glue: * Add support for detecting and deleting k8s pods that fail to start. * Allow providing namespace in k8s glue and k8s glue example. * Add base-pod-number parameter to k8s glue and example. - Change `agent.default_docker.image` to `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` (see [here](https://github.com/allegroai/clearml-agent/blob/8a46dc6b03860bc1cc022fd19e86bf4524986a0e/docs/clearml.conf#L144)). - Use shared git cache for multiple agents on the same machine. - Upgrade pynvml add detect CUDA version from driver level. - Update agent and services docker files. - Update documentation. **Bug Fixes** - Fix `docker --network` returns `None`. - Fix docker mode without venvs cache dir. - Fix applying git diff on a newly added file. - Fix environment variables `CLEARML_WEB_HOST`/`CLEARML_FILES_HOST` not passed to running tasks (or updated on the config object). - Fix `--detached` command line option not supported on Windows (ignore and issue warning). - Fix file not found error (errno 2) interpreted as aborted (i.e. `Ctrl-C`). - Fix `from clearml` runtime diff patching. - Fix cache to take cuda version into account. - Fix CPU mode. - Fix multi instances on Windows. - Fix conda support for `git+http` links. - Fix k8s glue does not pass docker environment variables, remove deprecated flags. ## Version 0.17.4 ### ClearML **Features** - Add `HyperParameterOptimizer` parallel coordinates ([clearml GitHub Issue 279](https://github.com/allegroai/trains/issues/279)). - Add `Task.init()` argument `tags`. - Change HPO parallel coordinates color scale, yellow is low. - Change `HyperParameterOptimizer` `spawn_task_project` to `spawn_project`. - Revert all Numpy requirement changes, use only lower limit. **Bug Fixes** - Fix argparse `nargs` support. - Fix argparse with `action="append"`. - Fix PyJWT v2.0 token parsing. - Fix Python package detection it should not list `file://` links. ## Version 0.17.3 ### ClearML **Features** * Add `Task.delete()` support. * Add `Task.debug_simulate_remote_task()` to simulate task execution by ClearML Agent. * Add warning on archived Task in pipeline ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)). * Add `Task.init(..., output_uri=True)`, which will use the default files_server as output uri. * Make `clearml-data` CLI stateful, so it remembers last dataset ID as default dataset. * Added `HyperParameterOptimizer.get_optimizer_top_experiments()` for querying post execution optimization pipeline. * Add `Task.set_archived()` and `Task.get_archived()`. * Add `Task.set_credentials()` option to store into credentials file. * Add `clearml-data` close now auto uploads. * Add `HyperParameterOptimizer` arguments `spawn_task_project` and `save_top_k_tasks_only`. **Bug Fixes** * Fix PipelineController running remotely without configuration does not execute the default code pipeline ([clearml GitHub Issue 273](https://github.com/allegroai/trains/issues/273)). * Fix reusing task after its project was deleted ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)). * Fix `Task.archived_tag read-only` property does not work ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)). * Fix argparse support to store consistent str representation of custom objects. Avoid changing default value if remote value matches. * Fix argparse type as function. * Fix Dataset add single and multiple file(s). * Fix get project name from parent dataset if not specified. * Fix mpl exporter. Added support for legend. * Fix model upload. * Fix optimizer callback best experiment. * Fix Optuna optimizer failing on tasks with None value in scalar query. * Fix auto Python package detection installed directly from URLs. * Fix dataset upload aborted on server watchdog. * Fix dataset genealogy, graph and restoring data. * Fix numpy dependency for Python versions. ## Version 0.17.2 ### ClearML **Bug Fixes** * Fix broken `clearml-task` CLI. ## Version 0.17.1 ### ClearML **Bug Fixes** * `clearml-task` CLI link fixed. ## Version 0.17.0 ### ClearML **Features** * Renamed `trains` as `clearml` (see [clearml](https://github.com/allegroai/clearml) in GitHub). * Add `clearml-data` CLI for Differential Dataset Management & Versioning (see details). * Add `clearml-task` CLI for creating (importing) code into **ClearML** (see details). * Add `Task.create`, which supports importing a code / repository into **ClearML**, including remote execution with `clearml-agent`. **Bug Fixes** * PyJWT v2.0 breaks the interface. ### Open Source ClearML Server **Features** * New profile page layout. * Move knowledge resource to UI top bar. * Add complete experiment full screen view. * Add scalar results split / grouped view. * Add experiment active duration custom column to experiment table. * Improve comparison experiment consistency in plots for experiment name and color. * Improve project last update time; updated whenever an experiment or model in the project is updated. * Improve table multi-select UX: shift+mouse-click ([clearml GitHub Issue 61](https://github.com/allegroai/trains/issues/61)). * Add experiment parent to experiment details INFO tab. * Improve UI experiment archiving; archiving queued experiments automatically dequeues them. * Improve experiment comparison; use available screen width. * Add debug sample history tracking ([clearml GitHub Issue 81](https://github.com/allegroai/trains/issues/81)). * Replace "Download" action with "Copy to Clipboard" for local files. * Add ClearML usage tips. * Add Task and Model, Clone and Move to allow creating new project on the fly. * Add support for all S3 / minio special character \(%\) encoding cases . * Add API for filter by parent. * Improve browser-search to find data not visible in uncommitted changes / installed packages window. * Improve Task artifacts optimization. * Upgrade plotly to 1.57.1. * Rename Log tab to Console. **Bug Fixes** * Fix HiDPI table issues. * Fix results / plots not scaling properly ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)). * Fix experiment comparison: * Cannot compare regular Tasks to example Tasks. * Cannot unhide scalars. * Scalars wall time view and relative view will not auto-refresh. * Fix log scrolling in Chrome. * Fix UI adds to plots legend labels unnecessary task ID string ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)). * Fix parallel coordinates: dimensions are not sorted by value. * Fix scalar auto color selection provides too similar colors. * Fix reordering a custom column does not restore column location. * Fix uncommitted changes - not possible to copy the entire text. * Fix queues wait time graph base should be 0, not in the middle. * Fix plots view - "No Data" shown for a moment when changing experiments. * Fix adding new tag during auto-refresh. * Fix hide empty input model on non draft experiment. * Fix experiment / model table - name column restores to default size after opening and closing info. * Fix double click resizer should auto fit column. ### ClearML Hosted Service only * Launched free [ClearML Hosted Service](https://app.community.clear.ml/dashboard). * Multiple workspaces. * Workspace invitation. * Add publicly sharing experiments. ### ClearML Agent * Renamed `trains-agent` as `clearml-agent` (see [clearml-agent](https://github.com/allegroai/clearml-agent) in GitHub).