ClearML/clearml-docs
1
0
Fork 0
mirror of https://github.com/clearml/clearml-docs synced 2025-02-07 21:24:49 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
df94ca111b
clearml-docs/docs/release_notes/sdk/open_source/ver_1_1.md
pollfly 8249768619
Refactor Release Notes (#938)
2024-09-26 16:09:25 +03:00

17 KiB
Raw Blame History

title
Version 1.1

ClearML 1.1.6

Features

  • Add Task.force_store_standalone_script() to force storing standalone script instead of a Git repository reference (ClearML GitHub issue #340)
  • Add Logger.set_default_debug_sample_history() and Logger.get_default_debug_sample_history() to allow controlling maximum debug samples programmatically
  • Add populate now stores function arg types as part of the hyperparameters
  • Add status_message argument to Task.mark_stopped()
  • Change HTTP driver timeout and retry codes (connection timeout will now trigger a retry)

Bug Fixes

  • Fix and upgrade the SlackMonitor (ClearML GitHub issue #533)
  • Fix network issues causing Task to stop on status change when no status change has occurred (ClearML GitHub issue #535)
  • Fix Pipeline controller function support for dict as input argument
  • Fix uploading the same metric/variant from multiple processes in threading mode should create a unique file per process (since global counter is not passed between the subprocesses)
  • Fix resource monitoring should only run in the main process when using threaded logging mode
  • Fix fork patching so that the signal handler (at_exit) will be called on time
  • Fix fork (process pool) hangs or drops reports when reports are at the end of the forked function in both threaded and subprocess mode reporting
  • Fix multi-pipeline support
  • Fix delete artifacts after upload
  • Fix artifact preview has no truth value
  • Fix storage cache cleanup does not remove all entries on a silent fail
  • Fix always store session cache in ~/.clearml (regardless of the cache folder)
  • Fix StorageManager.download_folder() fails on Windows path

ClearML 1.1.5

Features

  • Add support for jsonargpraser (ClearML GitHub issue #403)
  • Add HyperParameterOptimizer.get_top_experiments_details() returns the hparams and metrics of the top performing experiments of an HPO (ClearML GitHub issue #473)
  • Allow overriding initial iteration offset using environment variable (CLEARML_SET_ITERATION_OFFSET) or Task.init(continue_last_task==<offset>) ( ClearML GitHub issue #496)
  • Add better input handling for clearml-init in colab (ClearML GitHub issue #515)
  • Add environment variable for default request method (ClearML GitHub issue #521)
  • Add LocalClearmlJob as possible option for HPO (ClearML GitHub issue #525)
  • Add convenience functionality to clearml-data (ClearML GitHub issue #526)
  • Add support for vscode-jupyter (microsoft/vscode-jupyter#8531)
  • Improve detection of running reporting subprocess (including zombie state)
  • Support controlling S3/Google Cloud Storage _stream_download_pool_connections using the stream_connections configuration setting in clearml.conf (default 128)
  • Add warning when losing reporting subprocess
  • Add Model.remove() to allow removing a model from the model repository
  • Add HTTP download timeout control (change default connection timeout to 30 seconds)
  • Add initial setup callback to monitoring class
  • Add Task.get_reported_plots()
  • Allow Monitor.get_query_parameters to override defaults
  • Add support for Google Cloud Storage pool_connections and pool_maxsize overrides
  • Add last worker time to AutoScaler
  • Add warning when opening an aborted Dataset
  • Store multi-pipeline execution plots on the master pipeline Task
  • Support pipeline return value stored on pipeline Task
  • Add PipelineDecorator.multi_instance_support
  • Add PipelineDecorator to clearml and clearml.automation namespaces
  • Documentation and examples

Bug Fixes

  • Fix TriggerScheduler on Dataset change (ClearML GitHub issue #491)
  • Fix links in Jupyter Notebooks (ClearML GitHub issue #505)
  • Fix pandas delta datetime conversion (ClearML GitHub issue #510)
  • Fix matplotlib auto-magic detect bar graph series name (ClearML GitHub issue #518)
  • Fix path limitation on storage services (posix, object storage) when storing target artifacts by limiting length of project name (full path) and task name used for object path (ClearML GitHub issue #516)
  • Fix multi-processing context block catching exception
  • Fix Google Cloud Storage with no default project causes a crash
  • Fix main process's reporting subprocess lost, switch back to thread mode
  • Fix forked StorageHelper should use its own ThreadExecuter
  • Fix local StorageHelper.delete() raising exception on non-existent file instead of returning false
  • Fix StorageHelper rename partial file throwing errors on multiple access
  • Fix resource monitor fails on permission issues (skip over parts)
  • Fix reusing Task does not reset it
  • Fix support clearml PyCharm Plugin 1.0.2 (support partial pycharm git repo sync)
  • Fix Task.reset() force argument ineffective
  • Fix PY3.5 compatibility
  • Fix validation error causes infinite loop
  • Fix tasks schema prevents sending null container parts
  • Fix missing CLEARML_SET_ITERATION_OFFSET definition
  • Fix Model.get_weights_package() returns None on error
  • Fix download progress bar based on sdk.storage.log.report_download_chunk_size_mb configuration
  • Fix Conda lists the CudaToolkit version installed (for the agent to reproduce)
  • Fix Jupyter kernel shutdown causing nested atexit callbacks leaving Task in running state
  • Fix multi-subprocess can cause Task to hand at close
  • Fix TF 2.7 support (get logdir on with multiple TB writers)

ClearML 1.1.4

Bug Fixes

  • Fix duplicate keyword argument (affects clearml-data, Dataset.get()) (ClearML GitHub issue #490)
  • Fix session raises missing host error when in offline mode (ClearML GitHub issue #489)
  • Fix Task.get_task() does not load output_uri from stored Task
  • Fix Task.get_models()['input'] returns string instead of clearml.Model
  • Fix tf.saved_model.load() binding for TensorFlow>=2.0
  • Fix hyperparams with None value converted to empty string causes inferred type to change to str in consecutive Task.connect() calls

ClearML 1.1.3

Features

  • Add support for MegEngine with examples (ClearML GitHub issue #455)
  • Add TaskTypes to main namespace (ClearML GitHub issue #453)
  • Add LogUnifomParameterRange for hyperparameter optimization with Optuna (ClearML GitHub issue #462)
  • Add joblib (equivalent to scikit) to Task.init(auto_connect_frameworks) argument
  • Log environment variables starting with * in environ_bind.py (ClearML GitHub issue #459)
  • Pipeline
    • Add eager decorated pipeline execution
    • Support pipeline monitoring for scalars/models/artifacts
    • Add PipelineController.upload_model()
    • Add PipelineController.add_step(configuration_overrides) argument allowing to override Task configuration objects
    • Change PipelineController.start_locally() default run_pipeline_steps_locally=False
    • Add PipelineController.stop(mark_failed, mark_aborted) arguments
    • Add PipelineController.run_locally decorator
    • Add PipelineController.continue_on_fail property
    • Add PipelineController.__init__(abort_on_failure) argument
    • Add nested pipeline components missing pipeline tags
    • Add ClearmlJob state cache (refresh every second)
  • Datasets
    • Add clearml-data multi-chunk support
    • Change clearml-data default chunk size to 512 MB
    • Change Dataset.create() now automatically reverts to using current Task if no project/name provided
  • Add Optimizer.get_top_experiments_id_metrics_pair() for top performing experiments
  • Add support for setting default value to auto connected argparse arguments
  • Add Task.get_script() and Task.set_script() for getting and setting task's script properties for execution
  • Add Task.mark_completed() force and status_message arguments
  • Add Task.stopped() reason argument
  • Add Task.query_tasks(), Task.get_task() and Task.get_tasks() tags argument

Bug Fixes

  • Fix PyJWT resiliency support
  • Fix xgb train overload (ClearML GitHub issue #456)
  • Fix http:// throws OSError in Windows by using pathlib2 instead of os (ClearML GitHub issue #463)
  • Fix local diff should include staged commits, otherwise applying git diff fails (ClearML GitHub issue #457)
  • Fix task.upload_artifact non-standard dictionary will now revert to pickle (ClearML GitHub issue #452)
  • Fix S3BucketConfig.is_valid() for EC2 environments with use_credentials_chain (ClearML GitHub issue #478)
  • Fix audio classifier example when training with a custom dataset (ClearML GitHub issue #484)
  • Fix clearml-task diff was corrupted by Windows drive letter and separator (ClearML GitHub issue #483)
  • Fix TQDM "line cleanup" not using CR but rather arrow-up escape sequence
  • Fix task.connect(dict) value casting - if None is the default value, use backend stored type
  • Fix Jupyter notebook should always set Task as completed/stopped, never failed (exceptions are caught in interactive session)
  • Fix Pipeline support
    • Fix LocalClearmlJob setting failed status
    • Fix pipeline stopping all running steps
    • Fix nested pipeline component parent point to pipeline Task
    • Fix PipelineController.start() should not kill the process when done
    • Fix pipeline failing to create Step Task should cause the pipeline to be marked failed
    • Fix nested pipeline components missing pipeline tags
  • Fix images reported over history size were not sent if frequency was too high
  • Fix git detectors missing git repository without origin
  • Fix support for upload LazyEvalWrapper artifacts
  • Fix duplicate task dataset tags
  • Fix FileLock create target folder
  • Fix crash inside forked subprocess might leave SafeQueue in a locked state, causing task.close() to hang
  • Fix PyTorch distributed example TimeoutSocket issue in Windows
  • Fix broken Dataset.finalize()
  • Fix Python 3.5 compatibility

ClearML 1.1.2

Bug Fix

  • Fix PyJWT issue (limit dependency to <2.2.0)

ClearML 1.1.1

Bug Fixes

  • Fix Logger.report_image() throws warning
  • Fix TensorBoard add_image() not being reported

ClearML 1.1.0

:::info Breaking Changes

  • New PipelineController v2 was introduced. New constructor is not backwards compatible!
  • ClearML will no longer try to use the demo server by default (change this by setting the CLEARML_NO_DEFAULT_SERVER=0 environment variable)
  • Task.completed() was deprecated, use Task.mark_completed() instead :::

Features

  • Add Task Trigger Scheduler
  • Add Task Cron Scheduler
  • Add PipelineController from function
  • Add PipelineDecorator (PipelineDecorator.pipeline and PipelineDecorator.component decorators for full custom pipeline logic)
  • Add xgboost auto metric logging (ClearML GitHub issue #381)
  • Add sdk.storage.log.report_upload_chunk_size_mb and sdk.storage.log.report_download_chunk_size_mb configuration options to control upload/download log reporting (ClearML GitHub issue #424)
  • Add new optional auto_connect_frameworks argument value to Task.init() (e.g. auto_connect_frameworks={'tfdefines':False}) to allow disabling TF defines (ClearML GitHub issue #408)
  • Add support for CLEARNL_CONFIG_VERBOSE environment variable to allow external control over verbosity of the configuration loading process
  • Add support for uploading artifacts with a list of files using Task.upload_artifcats(name, [Path(), Path()])
  • Add missing clearml-task parameters --docker_args, --docker_bash_setup_script and --output-uri
  • Change CreateAndPopulate will auto list packages imported but not installed locally
  • Add clearml.task.populate.create_task_from_function() to create a Task from a function, wrapping function input arguments into hyper-parameter section as kwargs and storing function results as named artifacts
  • Add support for Task serialization (e.g. for pickle)
  • Add Task.get_configuration_object_as_dict()
  • Add docker_image argument to Task.set_base_docker() (deprecate docker_cmd)
  • Add auto_version_bump argument to PipelineController
  • Add sdk.development.detailed_import_report configuration option to provide a detailed report of all python package imports
  • Set current Task as Dataset parent when creating dataset
  • Add support for deferred configuration
  • Examples
    • Add Pipeline v2 examples
    • Add TaskScheduler and TriggerScheduler examples
    • Add pipeline controller callback example
    • Improve existing examples and docstrings

Bug Fixes

  • Fix plotly plots converting NaN to nan instead of null (ClearML GitHub issue #373)
  • Fix deprecation warning (ClearML GitHub issue #376)
  • Fix plotly multi-index without index names (ClearML GitHub issue #399)
  • Fix click support (ClearML GitHub issue #437)
  • Fix docstring (ClearML GitHub issue #438)
  • Fix passing task-type to clearml-task (ClearML GitHub issue #422)
  • Fix clearml-task --version throws an error (ClearML GitHub issue #422)
  • Fix clearml-task ssh repository links are not detected as remote repositories (ClearML GitHub issue #423)
  • Fix getattr throws an exception (ClearML GitHub issue #426)
  • Fix encoding while saving notebook preview (ClearML GitHub issue #443)
  • Fix poetry toml file without requirements.txt (ClearML GitHub issue #444)
  • Fix PY3.x fails calling SemLock._after_fork with forkserver context, forking while lock is acquired (ClearML Agent GitHub issue #73)
  • Fix wrong download path in StorageManager.download_folder()
  • Fix jupyter notebook display(...) convert to print(...)
  • Fix TensorFlow add_image() with description='text'
  • Fix Task.close() should remove current_task() reference
  • Fix TaskScheduler weekdays, change default execute_immediately to False
  • Fix Python2 compatibility
  • Fix clearml-task exit with error when failing to verify output_uri (output warning instead)
  • Fix unsafe Google Storage delete object
  • Fix multi-process spawning wait-for-uploads can create a deadlock in very rare cases
  • Fix task.set_parent() fails when passing Task object
  • Fix PipelineController skipping queued Tasks
  • Remove humanfriendly dependency (unused)