ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-01-31 22:48:40 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
146b8f0f12
clearml-docs/docs/release_notes/ver_0_17.md
pollfly 1290c50835
Fix links (#167)
2022-01-19 14:37:16 +02:00

265 lines
13 KiB
Markdown
Raw Blame History

---
title: Version 0.17
---
## ClearML 0.17.5
**Features**
- Add `force_download` argument to `Artifact.get()` and `Artifact.get_local_copy()` ([clearml GitHub Issue 319](https://github.com/allegroai/clearml/issues/319)).
- Support all reporting using subprocesses instead of threads (default `sdk.development.report_use_subprocess=True`).
- Improve Datasets support:
* Add `clearml-data publish` to allow publishing a dataset task.
* Add `clearml-data sync`, which can now create, sync and close a dataset with a single command.
* Allow Dataset to be created on a running Task.
* Add `dataset_tags` argument to `Dataset.get()`.
* Add `Dataset.get_logger()`.
- Add `Task.add_requirements()` support for specifying version `<>=~` etc.
- Add `StorageManager.upload_folder()` and `StorageManager.download_folder()`.
- Add progress report logging for `StorageHelper.upload_from_stream()`, `StorageHelper.upload()` and `StorageHelper.upload_from_stream()`.
- Add jupyter auto-magic - store Jupyter Notebook as an artifact on the Task (default `sdk.development.store_jupyter_notebook_artifact=True`).
- Add upload HTML preview of Jupyter Notebook as an artifact.
- Add `PipelineController` disable clone base task option.
- Add links to Tasks in optimization summary table (not just Task IDs).
- Add support for datetime in scatter plots + matplotlib support.
- Improve plotly value type conforming.
- Improve PyTorch `DataLoader` speed ([clearml GitHub Issue 207](https://github.com/allegroai/clearml/issues/207)).
- Update Auto Scaler default values and configuration.
- Examples:
* Add Hydra example.
* Add artifacts retrieval example.
* Update various examples.
**Bug Fixes**
- Fix warning or error message if requirements parsing failed ([clearml GitHub Issue 291](https://github.com/allegroai/clearml/issues/291)).
- Fix pytorch-lighting multi node store ([clearml GitHub Issue 292](https://github.com/allegroai/clearml/issues/292)).
- Fix strip remote diff ([clearml GitHub Issue 295](https://github.com/allegroai/clearml/issues/295)).
- Fix Python package detection `sklearn` -> `scikit-learn` ([clearml GitHub Issue 296](https://github.com/allegroai/clearml/issues/296)).
- Fix argparse issues:
* Fix argparse with `[None]` in default parameter ([clearml GitHub Issue 297](https://github.com/allegroai/clearml/issues/297)).
* Fix parsing of arguments in scientific notation ([clearml GitHub Issue 313](https://github.com/allegroai/clearml/issues/313)).
* Fix argparser logging always captures defaults (Windows only, ignored cmd).
* Fix argparse `nargs` passed in command line `--nargs 1 2` should be stored as `[1, 2]` not as `['1', '2']`.
* Fix support for nonstandard argparse with default value that is not of defined type.
* Fix server updated with the argparse in remote before Task.init() is called (respect skipped args).
- Fix Dataset support:
* Fix `Dataset.remove_files()` can't find files when files are in dataset root ([clearml GitHub Issue 303](https://github.com/allegroai/clearml/issues/303)).
* Fix closing a dataset with only files removed.
* Fix Dataset generate removed / modified / added of summary table.
- Fix Hydra multi-run support ([clearml GitHub Issue 306](https://github.com/allegroai/clearml/issues/306)).
- Fix TF/TensorBoard support:
* Fix TensorBoard multiple `Task.init()` / `Task.close()` calls within the same process ([clearml GitHub Issue 312](https://github.com/allegroai/clearml/issues/312)).
* Fix TensorBoard 2+ `pr_curve`.
* Fix TF `pr_curve` should not be inverted.
* Fix TF +2.3 mixed eager mode execution summary metrics not reported.
* Fix TF bind keyboard interrupt.
* Fix TF 2.4 keras load / save model.
- Fix `clearml-task`:
* Fix error when script cannot be found.
* Fix `--docker` flag not passed.
* Fix patching local git diff.
- Fix `clearml-data`:
* Fix `clearml-data sync` requires `--name`.
* Fix missing required argument `--files` in `clearml-data remove`.
- Fix `Task.execute_remotely()` from Jupyter Notebook.
- Fix populate Task called from Jupyter Notebook (use `Task.create(packages=True)` to auto populate based on locally installed packages).
- Fix plotly plot with numpy containing `NaN` / `datetime`.
- Fix matplotlib with Agg backend (or in remote execution).
- Fix trying to upload model file as a folder (automatically package the folder).
- Fix broken packages on package `importlib` detection failed the entire requirements detection.
- Fix `Task.connect(object)` should always return the same object instance.
- Fix `Task.create()` with repo and script that exists locally.
- Fix crash in case `Logger.get_logger()` cannot get the file name.
- Fix exception at exit in Python 3.8+ on macOS / Windows.
- Fix make pipeline summary table link to Task step logs.
- Fix Hydra 1.1 support (argparse description).
- Fix close task after logger is closed.
- Fix `Task.set_base_docker()` in remote execution.
- Fix artifact preview limit to 64Kb.
- Fix JupyterLab Notebook detection.
- Fix Python 2.7 support.
## ClearML Agent 0.17.2
**Features**
- Add virtual environment caching:
* Supports venv caching both in standard and docker mode.
* Configurable using the `agent.venvs_cache` configuration section.
* Disabled by default, enable [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L101).
- Add support for `--services-mode` with venvs.
- Add `agent.force_git_ssh_user` configuration value (default `git`, see [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L29))
([clearml-agent GitHub Issue 42](https://github.com/allegroai/clearml-agent/issues/42)).
- Add `agent.ignore_requested_python_version` configuration option for multi Python environments (default `false`).
- Add `agent.enable_task_env` configuration option to set the OS environment based on the Environment section of the Task (default `false`, see
[here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L151)).
- K8s glue:
* Add support for detecting and deleting k8s pods that fail to start.
* Allow providing namespace in k8s glue and k8s glue example.
* Add base-pod-number parameter to k8s glue and example.
- Change `agent.default_docker.image` to `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` (see [here](https://github.com/allegroai/clearml-agent/blob/8a46dc6b03860bc1cc022fd19e86bf4524986a0e/docs/clearml.conf#L144)).
- Use shared git cache for multiple agents on the same machine.
- Upgrade pynvml add detect CUDA version from driver level.
- Update agent and services docker files.
- Update documentation.
**Bug Fixes**
- Fix `docker --network` returns `None`.
- Fix docker mode without venvs cache dir.
- Fix applying git diff on a newly added file.
- Fix environment variables `CLEARML_WEB_HOST`/`CLEARML_FILES_HOST` not passed to running tasks (or updated on the config object).
- Fix `--detached` command line option not supported on Windows (ignore and issue warning).
- Fix file not found error (errno 2) interpreted as aborted (i.e. `Ctrl-C`).
- Fix `from clearml` runtime diff patching.
- Fix cache to take cuda version into account.
- Fix CPU mode.
- Fix multi instances on Windows.
- Fix conda support for `git+http` links.
- Fix k8s glue does not pass docker environment variables, remove deprecated flags.
## Version 0.17.4
### ClearML
**Features**
- Add `HyperParameterOptimizer` parallel coordinates ([clearml GitHub Issue 279](https://github.com/allegroai/trains/issues/279)).
- Add `Task.init()` argument `tags`.
- Change HPO parallel coordinates color scale, yellow is low.
- Change `HyperParameterOptimizer` `spawn_task_project` to `spawn_project`.
- Revert all Numpy requirement changes, use only lower limit.
**Bug Fixes**
- Fix argparse `nargs` support.
- Fix argparse with `action="append"`.
- Fix PyJWT v2.0 token parsing.
- Fix Python package detection it should not list `file://` links.
## Version 0.17.3
### ClearML
**Features**
* Add `Task.delete()` support.
* Add `Task.debug_simulate_remote_task()` to simulate task execution by ClearML Agent.
* Add warning on archived Task in pipeline ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)).
* Add `Task.init(..., output_uri=True)`, which will use the default files_server as output uri.
* Make `clearml-data` CLI stateful, so it remembers last dataset ID as default dataset.
* Added `HyperParameterOptimizer.get_optimizer_top_experiments()` for querying post execution optimization pipeline.
* Add `Task.set_archived()` and `Task.get_archived()`.
* Add `Task.set_credentials()` option to store into credentials file.
* Add `clearml-data` close now auto uploads.
* Add `HyperParameterOptimizer` arguments `spawn_task_project` and `save_top_k_tasks_only`.
**Bug Fixes**
* Fix PipelineController running remotely without configuration does not execute the default code pipeline ([clearml GitHub Issue 273](https://github.com/allegroai/trains/issues/273)).
* Fix reusing task after its project was deleted ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)).
* Fix `Task.archived_tag read-only` property does not work ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)).
* Fix argparse support to store consistent str representation of custom objects. Avoid changing default value if remote value matches.
* Fix argparse type as function.
* Fix Dataset add single and multiple file(s).
* Fix get project name from parent dataset if not specified.
* Fix mpl exporter. Added support for legend.
* Fix model upload.
* Fix optimizer callback best experiment.
* Fix Optuna optimizer failing on tasks with None value in scalar query.
* Fix auto Python package detection installed directly from URLs.
* Fix dataset upload aborted on server watchdog.
* Fix dataset genealogy, graph and restoring data.
* Fix numpy dependency for Python versions.
## Version 0.17.2
### ClearML
**Bug Fixes**
* Fix broken `clearml-task` CLI.
## Version 0.17.1
### ClearML
**Bug Fixes**
* `clearml-task` CLI link fixed.
## Version 0.17.0
### ClearML
**Features**
* Renamed `trains` as `clearml` (see [clearml](https://github.com/allegroai/clearml) in GitHub).
* Add `clearml-data` CLI for Differential Dataset Management & Versioning (see <a href="https://github.com/allegroai/clearml/blob/master/docs/datasets.md">details</a>).
* Add `clearml-task` CLI for creating (importing) code into **ClearML** (see <a href="https://github.com/allegroai/clearml/blob/master/docs/clearml-task.md">details</a>).
* Add `Task.create`, which supports importing a code / repository into **ClearML**, including remote execution with `clearml-agent`.
**Bug Fixes**
* PyJWT v2.0 breaks the interface.
### Open Source ClearML Server
**Features**
* New profile page layout.
* Move knowledge resource to UI top bar.
* Add complete experiment full screen view.
* Add scalar results split / grouped view.
* Add experiment active duration custom column to experiment table.
* Improve comparison experiment consistency in plots for experiment name and color.
* Improve project last update time; updated whenever an experiment or model in the project is updated.
* Improve table multi-select UX: shift+mouse-click ([clearml GitHub Issue 61](https://github.com/allegroai/trains/issues/61)).
* Add experiment parent to experiment details INFO tab.
* Improve UI experiment archiving; archiving queued experiments automatically dequeues them.
* Improve experiment comparison; use available screen width.
* Add debug sample history tracking ([clearml GitHub Issue 81](https://github.com/allegroai/trains/issues/81)).
* Replace "Download" action with "Copy to Clipboard" for local files.
* Add ClearML usage tips.
* Add Task and Model, Clone and Move to allow creating new project on the fly.
* Add support for all S3 / minio special character \(%\) encoding cases .
* Add API for filter by parent.
* Improve browser-search to find data not visible in uncommitted changes / installed packages window.
* Improve Task artifacts optimization.
* Upgrade plotly to 1.57.1.
* Rename Log tab to Console.
**Bug Fixes**
* Fix HiDPI table issues.
* Fix results / plots not scaling properly ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)).
* Fix experiment comparison:
* Cannot compare regular Tasks to example Tasks.
* Cannot unhide scalars.
* Scalars wall time view and relative view will not auto-refresh.
* Fix log scrolling in Chrome.
* Fix UI adds to plots legend labels unnecessary task ID string ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)).
* Fix parallel coordinates: dimensions are not sorted by value.
* Fix scalar auto color selection provides too similar colors.
* Fix reordering a custom column does not restore column location.
* Fix uncommitted changes - not possible to copy the entire text.
* Fix queues wait time graph base should be 0, not in the middle.
* Fix plots view - "No Data" shown for a moment when changing experiments.
* Fix adding new tag during auto-refresh.
* Fix hide empty input model on non draft experiment.
* Fix experiment / model table - name column restores to default size after opening and closing info.
* Fix double click resizer should auto fit column.
### ClearML Hosted Service Only
* Launched free [ClearML Hosted Service](https://app.clear.ml/dashboard).
* Multiple workspaces.
* Workspace invitation.
* Add publicly sharing experiments.
### ClearML Agent
* Renamed `trains-agent` as `clearml-agent` (see [clearml-agent](https://github.com/allegroai/clearml-agent) in GitHub).
Reference in New Issue View Git Blame Copy Permalink