ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-01-31 14:37:18 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
b87ea3b9d7
clearml-docs/docs/release_notes/ver_0_17.md
allegroai 77c9a91a95 Initial commit
2021-05-14 02:48:51 +03:00

13 KiB
Raw Blame History

title
Version 0.17

ClearML 0.17.5

Features

  • Add force_download argument to Artifact.get() and Artifact.get_local_copy() (clearml GitHub Issue 319).
  • Support all reporting using subprocesses instead of threads (default sdk.development.report_use_subprocess=True).
  • Improve Datasets support:
    • Add clearml-data publish to allow publishing a dataset task.
    • Add clearml-data sync, which can now create, sync and close a dataset with a single command.
    • Allow Dataset to be created on a running Task.
    • Add dataset_tags argument to Dataset.get().
    • Add Dataset.get_logger().
  • Add Task.add_requirements() support for specifying version <>=~ etc.
  • Add StorageManager.upload_folder() and StorageManager.download_folder().
  • Add progress report logging for StorageHelper.upload_from_stream(), StorageHelper.upload() and StorageHelper.upload_from_stream().
  • Add jupyter auto-magic - store Jupyter Notebook as an artifact on the Task (default sdk.development.store_jupyter_notebook_artifact=True).
  • Add upload HTML preview of Jupyter Notebook as an artifact.
  • Add PipelineController disable clone base task option.
  • Add links to Tasks in optimization summary table (not just Task IDs).
  • Add support for datetime in scatter plots + matplotlib support.
  • Improve plotly value type conforming.
  • Improve PyTorch DataLoader speed (clearml GitHub Issue 207).
  • Update Auto Scaler default values and configuration.
  • Examples:
    • Add Hydra example.
    • Add artifacts retrieval example.
    • Update various examples.

Bug Fixes

  • Fix warning or error message if requirements parsing failed (clearml GitHub Issue 291).
  • Fix pytorch-lighting multi node store (clearml GitHub Issue 292).
  • Fix strip remote diff (clearml GitHub Issue 295).
  • Fix Python package detection sklearn -> scikit-learn (clearml GitHub Issue 296).
  • Fix argparse issues:
    • Fix argparse with [None] in default parameter (clearml GitHub Issue 297).
    • Fix parsing of arguments in scientific notation (clearml GitHub Issue 313).
    • Fix argparser logging always captures defaults (Windows only, ignored cmd).
    • Fix argparse nargs passed in command line --nargs 1 2 should be stored as [1, 2] not as ['1', '2'].
    • Fix support for nonstandard argparse with default value that is not of defined type.
    • Fix server updated with the argparse in remote before Task.init() is called (respect skipped args).
  • Fix Dataset support:
    • Fix Dataset.remove_files() can't find files when files are in dataset root (clearml GitHub Issue 303).
    • Fix closing a dataset with only files removed.
    • Fix Dataset generate removed / modified / added of summary table.
  • Fix Hydra multi-run support (clearml GitHub Issue 306).
  • Fix TF/TensorBoard support:
    • Fix TensorBoard multiple Task.init() / Task.close() calls within the same process (clearml GitHub Issue 312).
    • Fix TensorBoard 2+ pr_curve.
    • Fix TF pr_curve should not be inverted.
    • Fix TF +2.3 mixed eager mode execution summary metrics not reported.
    • Fix TF bind keyboard interrupt.
    • Fix TF 2.4 keras load / save model.
  • Fix clearml-task:
    • Fix error when script cannot be found.
    • Fix --docker flag not passed.
    • Fix patching local git diff.
  • Fix clearml-data:
    • Fix clearml-data sync requires --name.
    • Fix missing required argument --files in clearml-data remove.
  • Fix Task.execute_remotely() from Jupyter Notebook.
  • Fix populate Task called from Jupyter Notebook (use Task.create(packages=True) to auto populate based on locally installed packages).
  • Fix plotly plot with numpy containing NaN / datetime.
  • Fix matplotlib with Agg backend (or in remote execution).
  • Fix trying to upload model file as a folder (automatically package the folder).
  • Fix broken packages on package importlib detection failed the entire requirements detection.
  • Fix Task.connect(object) should always return the same object instance.
  • Fix Task.create() with repo and script that exists locally.
  • Fix crash in case Logger.get_logger() cannot get the file name.
  • Fix exception at exit in Python 3.8+ on macOS / Windows.
  • Fix make pipeline summary table link to Task step logs.
  • Fix Hydra 1.1 support (argparse description).
  • Fix close task after logger is closed.
  • Fix Task.set_base_docker() in remote execution.
  • Fix artifact preview limit to 64Kb.
  • Fix JupyterLab Notebook detection.
  • Fix Python 2.7 support.

ClearML Agent 0.17.2

Features

  • Add virtual environment caching:
    • Supports venv caching both in standard and docker mode.
    • Configurable using the agent.venvs_cache configuration section.
    • Disabled by default, enable here.
  • Add support for --services-mode with venvs.
  • Add agent.force_git_ssh_user configuration value (default git, see here) (clearml-agent GitHub Issue 42).
  • Add agent.ignore_requested_python_version configuration option for multi Python environments (default false).
  • Add agent.enable_task_env configuration option to set the OS environment based on the Environment section of the Task (default false, see here).
  • K8s glue:
    • Add support for detecting and deleting k8s pods that fail to start.
    • Allow providing namespace in k8s glue and k8s glue example.
    • Add base-pod-number parameter to k8s glue and example.
  • Change agent.default_docker.image to nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 (see here).
  • Use shared git cache for multiple agents on the same machine.
  • Upgrade pynvml add detect CUDA version from driver level.
  • Update agent and services docker files.
  • Update documentation.

Bug Fixes

  • Fix docker --network returns None.
  • Fix docker mode without venvs cache dir.
  • Fix applying git diff on a newly added file.
  • Fix environment variables CLEARML_WEB_HOST/CLEARML_FILES_HOST not passed to running tasks (or updated on the config object).
  • Fix --detached command line option not supported on Windows (ignore and issue warning).
  • Fix file not found error (errno 2) interpreted as aborted (i.e. Ctrl-C).
  • Fix from clearml runtime diff patching.
  • Fix cache to take cuda version into account.
  • Fix CPU mode.
  • Fix multi instances on Windows.
  • Fix conda support for git+http links.
  • Fix k8s glue does not pass docker environment variables, remove deprecated flags.

Version 0.17.4

ClearML

Features

  • Add HyperParameterOptimizer parallel coordinates (clearml GitHub Issue 279).
  • Add Task.init() argument tags.
  • Change HPO parallel coordinates color scale, yellow is low.
  • Change HyperParameterOptimizer spawn_task_project to spawn_project.
  • Revert all Numpy requirement changes, use only lower limit.

Bug Fixes

  • Fix argparse nargs support.
  • Fix argparse with action="append".
  • Fix PyJWT v2.0 token parsing.
  • Fix Python package detection it should not list file:// links.

Version 0.17.3

ClearML

Features

  • Add Task.delete() support.
  • Add Task.debug_simulate_remote_task() to simulate task execution by ClearML Agent.
  • Add warning on archived Task in pipeline (clearml GitHub Issue 274).
  • Add Task.init(..., output_uri=True), which will use the default files_server as output uri.
  • Make clearml-data CLI stateful, so it remembers last dataset ID as default dataset.
  • Added HyperParameterOptimizer.get_optimizer_top_experiments() for querying post execution optimization pipeline.
  • Add Task.set_archived() and Task.get_archived().
  • Add Task.set_credentials() option to store into credentials file.
  • Add clearml-data close now auto uploads.
  • Add HyperParameterOptimizer arguments spawn_task_project and save_top_k_tasks_only.

Bug Fixes

  • Fix PipelineController running remotely without configuration does not execute the default code pipeline (clearml GitHub Issue 273).
  • Fix reusing task after its project was deleted (clearml GitHub Issue 274).
  • Fix Task.archived_tag read-only property does not work (clearml GitHub Issue 274).
  • Fix argparse support to store consistent str representation of custom objects. Avoid changing default value if remote value matches.
  • Fix argparse type as function.
  • Fix Dataset add single and multiple file(s).
  • Fix get project name from parent dataset if not specified.
  • Fix mpl exporter. Added support for legend.
  • Fix model upload.
  • Fix optimizer callback best experiment.
  • Fix Optuna optimizer failing on tasks with None value in scalar query.
  • Fix auto Python package detection installed directly from URLs.
  • Fix dataset upload aborted on server watchdog.
  • Fix dataset genealogy, graph and restoring data.
  • Fix numpy dependency for Python versions.

Version 0.17.2

ClearML

Bug Fixes

  • Fix broken clearml-task CLI.

Version 0.17.1

ClearML

Bug Fixes

  • clearml-task CLI link fixed.

Version 0.17.0

ClearML

Features

  • Renamed trains as clearml (see clearml in GitHub).
  • Add clearml-data CLI for Differential Dataset Management & Versioning (see details).
  • Add clearml-task CLI for creating (importing) code into ClearML (see details).
  • Add Task.create, which supports importing a code / repository into ClearML, including remote execution with clearml-agent.

Bug Fixes

  • PyJWT v2.0 breaks the interface.

Open Source ClearML Server

Features

  • New profile page layout.
  • Move knowledge resource to UI top bar.
  • Add complete experiment full screen view.
  • Add scalar results split / grouped view.
  • Add experiment active duration custom column to experiment table.
  • Improve comparison experiment consistency in plots for experiment name and color.
  • Improve project last update time; updated whenever an experiment or model in the project is updated.
  • Improve table multi-select UX: shift+mouse-click (clearml GitHub Issue 61).
  • Add experiment parent to experiment details INFO tab.
  • Improve UI experiment archiving; archiving queued experiments automatically dequeues them.
  • Improve experiment comparison; use available screen width.
  • Add debug sample history tracking (clearml GitHub Issue 81).
  • Replace "Download" action with "Copy to Clipboard" for local files.
  • Add ClearML usage tips.
  • Add Task and Model, Clone and Move to allow creating new project on the fly.
  • Add support for all S3 / minio special character % encoding cases .
  • Add API for filter by parent.
  • Improve browser-search to find data not visible in uncommitted changes / installed packages window.
  • Improve Task artifacts optimization.
  • Upgrade plotly to 1.57.1.
  • Rename Log tab to Console.

Bug Fixes

  • Fix HiDPI table issues.
  • Fix results / plots not scaling properly (clearml GitHub Issue 247).
  • Fix experiment comparison:
    • Cannot compare regular Tasks to example Tasks.
    • Cannot unhide scalars.
    • Scalars wall time view and relative view will not auto-refresh.
  • Fix log scrolling in Chrome.
  • Fix UI adds to plots legend labels unnecessary task ID string (clearml GitHub Issue 247).
  • Fix parallel coordinates: dimensions are not sorted by value.
  • Fix scalar auto color selection provides too similar colors.
  • Fix reordering a custom column does not restore column location.
  • Fix uncommitted changes - not possible to copy the entire text.
  • Fix queues wait time graph base should be 0, not in the middle.
  • Fix plots view - "No Data" shown for a moment when changing experiments.
  • Fix adding new tag during auto-refresh.
  • Fix hide empty input model on non draft experiment.
  • Fix experiment / model table - name column restores to default size after opening and closing info.
  • Fix double click resizer should auto fit column.

ClearML Hosted Service only

  • Launched free ClearML Hosted Service.
  • Multiple workspaces.
  • Workspace invitation.
  • Add publicly sharing experiments.

ClearML Agent

  • Renamed trains-agent as clearml-agent (see clearml-agent in GitHub).