ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-01-31 14:37:18 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
43751dc64b
clearml-docs/docs/release_notes/ver_1_1.md
pollfly 43751dc64b
add ClearML SDK release notes (#109)
2021-11-08 11:59:24 +02:00

18 KiB
Raw Blame History

title
Version 1.1

ClearML SDK 1.1.4

Bug Fixes

  • Fix duplicate keyword argument (affects clearml-data, Dataset.get()) ClearML GitHub issue #490
  • Fix session raises missing host error when in offline mode ClearML GitHub issue #489
  • Fix Task.get_task() does not load output_uri from stored Task
  • Fix Task.get_models()['input'] returns string instead of clearml.Model
  • Fix tf.saved_model.load() binding for TensorFlow>=2.0
  • Fix hyperparams with None value converted to empty string causes inferred type to change to str in consecutive Task.connect() calls

ClearML Agent 1.1.1

Features and Bug Fixes

  • Add support for truncating task log file after reporting to server using agent.truncate_task_output_files configuration setting
  • Fix PyJWT resiliency support
  • Fix --stop checking default queue tag ClearML Agent GitHub issue #80
  • Fix queue tag default does not exist and --queue not specified (try queue named "default")
  • Fix Python 3.5 compatibility
  • Fix PY2.7 support for PyTorch

ClearML SDK 1.1.3

Features

  • Add support for MegEngine with examples ClearML GitHub issue #455
  • Add TaskTypes to main namespace ClearML GitHub issue #453
  • Add LogUnifomParameterRange for hyperparameter optimization with Optuna ClearML GitHub issue #462
  • Add joblib (equivalent to scikit) to Task.init(auto_connect_frameworks) argument
  • Log environment variables starting with * in environ_bind.py ClearML GitHub issue #459
  • Pipeline
    • Add eager decorated pipeline execution
    • Support pipeline monitoring for scalars/models/artifacts
    • Add PipelineController.upload_model()
    • Add PipelineController.add_step(configuration_overrides) argument allowing to override Task configuration objects
    • Change PipelineController.start_locally() default run_pipeline_steps_locally=False
    • Add PipelineController.stop(mark_failed, mark_aborted) arguments
    • Add PipelineController.run_locally decorator
    • Add PipelineController.continue_on_fail property
    • Add PipelineController.__init__(abort_on_failure) argument
    • Add nested pipeline components missing pipeline tags
    • Add ClearmlJob state cache (refresh every second)
  • Datasets
    • Add clearml-data multi-chunk support
    • Change clearml-data default chunk size to 512 MB
    • Change Dataset.create() now automatically reverts to using current Task if no project/name provided
  • Add Optimizer.get_top_experiments_id_metrics_pair() for top performing experiments
  • Add support for setting default value to auto connected argparse arguments
  • Add Task.get_script() and Task.set_script() for getting and setting task's script properties for execution
  • Add Task.mark_completed() force and status_message arguments
  • Add Task.stopped() reason argument
  • Add Task.query_tasks(), Task.get_task() and Task.get_tasks() tags argument

Bug Fixes

  • Fix PyJWT resiliency support
  • Fix xgb train overload ClearML GitHub issue #456
  • Fix http:// throws OSError in Windows by using pathlib2 instead of os ClearML GitHub issue #463
  • Fix local diff should include staged commits, otherwise applying git diff fails ClearML GitHub issue #457
  • Fix task.upload_artifact non-standard dictionary will now revert to pickle ClearML GitHub issue #452
  • Fix S3BucketConfig.is_valid() for EC2 environments with use_credentials_chain ClearML GitHub issue #478
  • Fix audio classifier example when training with a custom dataset ClearML GitHub issue #484
  • Fix clearml-task diff was corrupted by Windows drive letter and separator ClearML GitHub issue #483
  • Fix TQDM "line cleanup" not using CR but rather arrow-up escape sequence
  • Fix task.connect(dict) value casting - if None is the default value, use backend stored type
  • Fix Jupyter notebook should always set Task as completed/stopped, never failed (exceptions are caught in interactive session)
  • Fix Pipeline support
    • Fix LocalClearmlJob setting failed status
    • Fix pipeline stopping all running steps
    • Fix nested pipeline component parent point to pipeline Task
    • Fix PipelineController.start() should not kill the process when done
    • Fix pipeline failing to create Step Task should cause the pipeline to be marked failed
    • Fix nested pipeline components missing pipeline tags
  • Fix images reported over history size were not sent if frequency was too high
  • Fix git detectors missing git repository without origin
  • Fix support for upload LazyEvalWrapper artifacts
  • Fix duplicate task dataset tags
  • Fix FileLock create target folder
  • Fix crash inside forked subprocess might leave SafeQueue in a locked state, causing task.close() to hang
  • Fix PyTorch distributed example TimeoutSocket issue in Windows
  • Fix broken Dataset.finalize()
  • Fix Python 3.5 compatibility

ClearML SDK 1.1.2

Bug Fix

Fix PyJWT issue (limit dependency to <2.2.0)

ClearML SDK 1.1.1

Bug Fixes

  • Fix Logger.report_image() throws warning
  • Fix TensorBoard add_image() not being reported

ClearML SDK 1.1.0

:::info Breaking Changes

  • New PipelineController v2 was introduced. New constructor is not backwards compatible!
  • ClearML will no longer try to use the demo server by default (change this by setting the CLEARML_NO_DEFAULT_SERVER=0 environment variable)
  • Task.completed() was deprecated, use Task.mark_completed() instead :::

Features

  • Add Task Trigger Scheduler
  • Add Task Cron Scheduler
  • Add PipelineController from function
  • Add PipelineDecorator (PipelineDecorator.pipeline and PipelineDecorator.component decorators for full custom pipeline logic)
  • Add xgboost auto metric logging ClearML GitHub issue #381
  • Add sdk.storage.log.report_upload_chunk_size_mb and sdk.storage.log.report_download_chunk_size_mb configuration options to control upload/download log reporting ClearML GitHub issue #424
  • Add new optional auto_connect_frameworks argument value to Task.init() (e.g. auto_connect_frameworks={'tfdefines':False}) to allow disabling TF defines ClearML GitHub issue #408
  • Add support for CLEARNL_CONFIG_VERBOSE environment variable to allow external control over verbosity of the configuration loading process
  • Add support for uploading artifacts with a list of files using Task.upload_artifcats(name, [Path(), Path()])
  • Add missing clearml-task parameters --docker_args, --docker_bash_setup_script and --output-uri
  • Change CreateAndPopulate will auto list packages imported but not installed locally
  • Add clearml.task.populate.create_task_from_function() to create a Task from a function, wrapping function input arguments into hyper-parameter section as kwargs and storing function results as named artifacts
  • Add support for Task serialization (e.g. for pickle)
  • Add Task.get_configuration_object_as_dict()
  • Add docker_image argument to Task.set_base_docker() (deprecate docker_cmd)
  • Add auto_version_bump argument to PipelineController
  • Add sdk.development.detailed_import_report configuration option to provide a detailed report of all python package imports
  • Set current Task as Dataset parent when creating dataset
  • Add support for deferred configuration
  • Examples
    • Add Pipeline v2 examples
    • Add TaskScheduler and TriggerScheduler examples
    • Add pipeline controller callback example
    • Improve existing examples and docstrings

Bug Fixes

  • Fix poltly plots converting NaN to nan instead of null ClearML GitHub issue #373
  • Fix deprecation warning ClearML GitHub issue #376
  • Fix plotly multi-index without index names ClearML GitHub issue #399
  • Fix click support ClearML GitHub issue #437
  • Fix docstring ClearML GitHub issue #438
  • Fix passing task-type to clearml-task ClearML GitHub issue #422
  • Fix clearml-task --version throws an error ClearML GitHub issue #422
  • Fix clearml-task ssh repository links are not detected as remote repositories ClearML GitHub issue #423
  • Fix getattr throws an exception ClearML GitHub issue #426
  • Fix encoding while saving notebook preview ClearML GitHub issue #443
  • Fix poetry toml file without requirements.txt ClearML GitHub issue #444
  • Fix PY3.x fails calling SemLock._after_fork with forkserver context, forking while lock is acquired ClearML Agent GitHub issue #73
  • Fix wrong download path in StorageManager.download_folder()
  • Fix jupyter notebook display(...) convert to print(...)
  • Fix Tensorflow add_image() with description='text'
  • Fix Task.close() should remove current_task() reference
  • Fix TaskScheduler weekdays, change default execute_immediately to False
  • Fix Python2 compatibility
  • Fix clearml-task exit with error when failing to verify output_uri (output warning instead)
  • Fix unsafe Google Storage delete object
  • Fix multi-process spawning wait-for-uploads can create a deadlock in very rare cases
  • Fix task.set_parent() fails when passing Task object
  • Fix PipelineController skipping queued Tasks
  • Remove humanfriendly dependency (unused)

ClearML Agent 1.1.0

:::info Breaking Changes ClearML Agent will no longer try to use the demo server by default (change this by setting the CLEARML_NO_DEFAULT_SERVER=0 environment variable)

ClearML k8s glue default pod label was changed to CLEARML=agent (instead of TRAINS=agent) :::

Features

  • Add poetry cache into docker mapping ClearML Agent GitHub issue #74
  • Allow rewriting SSH URLs (see here), refers to ClearML Agent GitHub PR #72, ClearML Agent GitHub issue #42
  • Add docker environment arguments log masking support, customizable using the agent.hide_docker_command_env_vars configuration value (see here) ClearML Agent GitHub issue #67
  • Add support for naming docker containers using the agent.docker_container_name_format configuration option to set a name format (disabled by default) ClearML issue #412
  • k8s glue
    • Remove queue name from pod name, add queue name and ID to pod labels ClearML Agent GitHub issue #64
    • Update task status_message for non-responsive or hanging pods
    • Support the agent.docker_force_pull configuration option for scheduled pods
    • Add docker example for running the k8s glue as a pod in a k8s cluster
  • Add agent.ignore_requested_python_version configuration option to ignore any requested python version (default false, see here)
  • Add agent.docker_internal_mounts configuration option to control containers internal mounts (non-root containers, see here)
  • Add support for -r requirements.txt in the Installed Packages section
  • Add support for CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE environment variable to override initial server connection behavior (defaults to true, allows boolean value or an explicit number specifying the number of connect retries)
  • Add support for CLEARML_AGENT_DISABLE_SSH_MOUNT environment variable allowing to disable the auto .ssh mount into the docker
  • Add support for CLEARML_AGENT_SKIP_PIP_VENV_INSTALL environment variable to skip Python virtual env installation on execute and allow providing a custom venv binary
  • Add support for CLEARML_AGENT_VENV_CACHE_PATH environment variable to allow overriding venv cache folder configuration
  • Add support for CLEARML_AGENT_EXTRA_DOCKER_ARGS environment variable to allow overriding extra docker args configuration
  • Add support for environment variables containing bash-style string lists using shlex
  • Add printout when using ClearML key/secret from environment variables
  • Increase worker keep-alive timeout to 10 minutes instead of 1 minute
  • Update documentation

Bug Fixes

  • Fix auto mount SSH_AUTH_SOCK into docker ClearML Agent GitHub issue #45
  • Fix package manager configuration documentation ClearML Agent GitHub issue #78
  • Fix support for spaces in docker arguments ClearML GitHub issue #358
  • Fix standalone script with pre-exiting conda venv
  • Fix PyYAML v5.4, v5.4.1 versions not supported
  • Fix parsing VCS links starting with git+git@ (notice git+git:// was already supported)
  • Fix Python package with git+git:// links or git+ssh:// conversion
  • Fix --services-mode if the execute agent fails when starting to run with error code 0
  • Fix --stop with dynamic gpus
  • Fix support for unicode standalone scripts, changing default ascii encoding to UTF-8
  • Fix venv cache cannot reinstall package from git with http credentials
  • Fix PYTHONIOENCODING environment variable is overwritten when already defined
  • k8s glue
    • Fix support for multiple k8s glue instances with pod limits
    • Fix task container handling fails parsing docker image
    • Fix task container is not set when using default image/arguments
    • Fix task container image arguments are used when no image is specified
    • Fix task container arguments not supported in when template is not provided
    • Fix agent.extra_docker_bash_script not applied correctly
    • Fix task runtime properties are removed when re-enqueuing task
    • Fix error is not thrown when failing to push task to queue

ClearML Server 1.1.1

Bug Fixes

ClearML Server 1.1.0

New Features and Improvements

  • Add metric snapshot plot to project overview UI - Show an aggregated view of all project experiments value for a leading metric
  • Add logical AND option to UI experiment table tag filter
  • Add Task runtime properties to experiment INFO UI tab
  • Add full screen view for any experiment result plot
  • Add extended version information to UI profile page
  • Stop using special characters in secrets
  • Allow setting status_message in tasks.update
  • Improve UI table view configuration persistence - User table-view configuration is saved per project:
    • Visible columns
    • Column order
    • Column width
    • Active sort
    • Active filters

Bug Fixes

  • Fix experiment details UI failure opening hyperparameter sections beginning with # ClearML Server GitHub issue #79
  • Fix performance issues with UI comparison of large experiments Slack Channel
  • Fix filtering on hyperparameters ClearML GitHub issue #385 Slack Channel
  • Fix profile page user options toggle control area of effect
  • Fix browser resizing affecting plot zoom
  • Fix UI dataview filter resetting version on filter addition
  • Fix UI project overview:
    • Fix links in project overview not working
    • Allow editing empty overview for legacy projects
  • Fix Table plots using fraction of available space
  • Fix scalars color assignment broken by . in scalar name
  • Fix tasks cannot be moved between queues
  • Fix UI Docker argument input - Force arguments in designated field
  • Fix extraneous "tag" string prefixing Commit ID in Task execution information UI
  • Fix missing 'no value' option in hyperparameters table filters
  • Fix queued task is not dequeued on tasks.stop
  • General aesthetic fixes:
    • Fix input title alignment in UI clone experiment window
    • Fix UI empty experiment table message alignment
    • Fix UI table filter menu proportions
    • Fix debug sample dropdown menu coloring
    • Fix dashboard card alignment
    • Fix redundant separator in plot control when maximized