From e1b179ed544b1017168aca226928e4a9bc95556f Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Thu, 13 Feb 2025 13:21:35 +0200 Subject: [PATCH] Update links (#1041) --- README.md | 2 +- docs/apps/clearml_session.md | 2 +- docs/apps/clearml_task.md | 6 +- .../clearml_agent/clearml_agent_deployment.md | 4 +- .../clearml_agent_fractional_gpus.md | 6 +- docs/clearml_agent/clearml_agent_setup.md | 2 +- .../data_man_cifar_classification.md | 2 +- .../data_man_folder_sync.md | 2 +- .../data_man_python.md | 8 +- .../data_man_simple.md | 2 +- docs/clearml_sdk/clearml_sdk.md | 18 ++-- docs/clearml_sdk/task_sdk.md | 8 +- docs/clearml_serving/clearml_serving.md | 2 +- docs/clearml_serving/clearml_serving_setup.md | 2 +- .../clearml_serving_tutorial.md | 26 ++--- .../cloud_autoscaling/autoscaling_overview.md | 2 +- docs/community.md | 10 +- docs/configs/clearml_conf.md | 2 +- .../clearml_server_config.md | 2 +- .../clearml_server_es7_migration.md | 8 +- .../clearml_server_kubernetes_helm.md | 2 +- .../clearml_server_mongo44_migration.md | 4 +- .../clearml_server_security.md | 4 +- docs/deploying_clearml/upgrade_server_gcp.md | 2 +- .../upgrade_server_kubernetes_helm.md | 2 +- .../upgrade_server_linux_mac.md | 2 +- docs/deploying_clearml/upgrade_server_win.md | 2 +- docs/faq.md | 4 +- docs/fundamentals/agents_and_queues.md | 2 +- docs/fundamentals/hpo.md | 2 +- docs/fundamentals/task.md | 2 +- docs/getting_started/ds/ds_second_steps.md | 10 +- docs/getting_started/main.md | 6 +- .../mlops/mlops_first_steps.md | 2 +- .../video_tutorials/quick_introduction.md | 2 +- docs/guides/advanced/execute_remotely.md | 2 +- .../advanced/multiple_tasks_single_process.md | 2 +- .../manual_random_param_search_example.md | 4 +- docs/guides/automation/task_piping.md | 4 +- .../clearml-task/clearml_task_tutorial.md | 10 +- .../executable_exp_containers.md | 4 +- .../exp_environment_containers.md | 2 +- .../distributed_pytorch_example.md | 2 +- docs/guides/distributed/subprocess_example.md | 2 +- .../autokeras/autokeras_imdb_example.md | 2 +- docs/guides/frameworks/catboost/catboost.md | 2 +- .../fastai/fastai_with_tensorboard.md | 4 +- .../frameworks/huggingface/transformers.md | 2 +- docs/guides/frameworks/keras/jupyter.md | 2 +- .../frameworks/keras/keras_tensorboard.md | 6 +- .../frameworks/lightgbm/lightgbm_example.md | 2 +- .../matplotlib/matplotlib_example.md | 6 +- .../frameworks/megengine/megengine_mnist.md | 2 +- .../frameworks/pytorch/model_updating.md | 2 +- .../audio_classification_UrbanSound8K.md | 2 +- .../audio/audio_preprocessing_example.md | 2 +- .../notebooks/image/hyperparameter_search.md | 4 +- .../image/image_classification_CIFAR10.md | 2 +- .../table/download_and_preprocessing.md | 4 +- .../table/tabular_training_pipeline.md | 20 ++-- .../text/text_classification_AG_NEWS.md | 2 +- .../frameworks/pytorch/pytorch_abseil.md | 2 +- .../pytorch/pytorch_distributed_example.md | 2 +- .../frameworks/pytorch/pytorch_matplotlib.md | 2 +- .../frameworks/pytorch/pytorch_mnist.md | 2 +- .../frameworks/pytorch/pytorch_tensorboard.md | 2 +- .../pytorch/pytorch_tensorboardx.md | 2 +- .../pytorch/tensorboard_toy_pytorch.md | 2 +- .../integration_pytorch_ignite.md | 2 +- .../pytorch_lightning_example.md | 2 +- .../scikit-learn/sklearn_joblib_example.md | 2 +- .../sklearn_matplotlib_example.md | 2 +- .../frameworks/tensorboardx/tensorboardx.md | 2 +- .../tensorboardx/video_tensorboardx.md | 2 +- .../tensorflow/integration_keras_tuner.md | 4 +- .../tensorflow/tensorboard_pr_curve.md | 2 +- .../frameworks/tensorflow/tensorboard_toy.md | 2 +- .../frameworks/tensorflow/tensorflow_mnist.md | 2 +- .../frameworks/xgboost/xgboost_metrics.md | 2 +- .../frameworks/xgboost/xgboost_sample.md | 2 +- docs/guides/ide/google_colab.md | 2 +- docs/guides/ide/integration_pycharm.md | 2 +- docs/guides/ide/remote_jupyter_tutorial.md | 6 +- docs/guides/main.md | 2 +- .../examples_hyperparam_opt.md | 4 +- docs/guides/pipeline/pipeline_controller.md | 18 ++-- docs/guides/pipeline/pipeline_decorator.md | 4 +- docs/guides/pipeline/pipeline_functions.md | 4 +- docs/guides/reporting/3d_plots_reporting.md | 2 +- docs/guides/reporting/artifacts.md | 2 +- .../reporting/clearml_logging_example.md | 2 +- docs/guides/reporting/explicit_reporting.md | 6 +- docs/guides/reporting/html_reporting.md | 12 +-- docs/guides/reporting/hyper_parameters.md | 2 +- docs/guides/reporting/image_reporting.md | 2 +- .../reporting/manual_matplotlib_reporting.md | 2 +- docs/guides/reporting/media_reporting.md | 2 +- docs/guides/reporting/model_config.md | 2 +- docs/guides/reporting/pandas_reporting.md | 2 +- docs/guides/reporting/plotly_reporting.md | 2 +- docs/guides/reporting/scalar_reporting.md | 2 +- .../scatter_hist_confusion_mat_reporting.md | 2 +- docs/guides/reporting/text_reporting.md | 2 +- docs/guides/reporting/using_artifacts.md | 2 +- docs/guides/services/aws_autoscaler.md | 6 +- docs/guides/services/cleanup_service.md | 4 +- docs/guides/services/slack_alerts.md | 2 +- docs/guides/ui/building_leader_board.md | 2 +- docs/guides/ui/tuning_exp.md | 6 +- docs/hyperdatasets/code_examples.md | 12 +-- docs/integrations/click.md | 2 +- docs/integrations/hydra.md | 2 +- docs/integrations/ignite.md | 2 +- docs/integrations/jsonargparse.md | 2 +- docs/integrations/mmcv.md | 2 +- docs/integrations/python_fire.md | 2 +- docs/integrations/seaborn.md | 2 +- docs/integrations/tensorflow.md | 2 +- docs/pipelines/pipelines_sdk_tasks.md | 2 +- docs/release_notes/clearml_agent/ver_0_12.md | 6 +- docs/release_notes/clearml_agent/ver_0_14.md | 4 +- docs/release_notes/clearml_agent/ver_0_15.md | 8 +- docs/release_notes/clearml_agent/ver_0_16.md | 6 +- docs/release_notes/clearml_agent/ver_0_17.md | 12 +-- docs/release_notes/clearml_agent/ver_1_0.md | 6 +- docs/release_notes/clearml_agent/ver_1_1.md | 22 ++-- docs/release_notes/clearml_agent/ver_1_2.md | 18 ++-- docs/release_notes/clearml_agent/ver_1_3.md | 2 +- docs/release_notes/clearml_agent/ver_1_4.md | 2 +- docs/release_notes/clearml_agent/ver_1_5.md | 10 +- docs/release_notes/clearml_agent/ver_1_6.md | 4 +- docs/release_notes/clearml_agent/ver_1_7.md | 2 +- docs/release_notes/clearml_agent/ver_1_8.md | 8 +- docs/release_notes/clearml_agent/ver_1_9.md | 6 +- .../clearml_server/enterprise/ver_3_23.md | 16 +-- .../clearml_server/enterprise/ver_3_24.md | 2 +- .../clearml_server/open_source/ver_0_10.md | 4 +- .../clearml_server/open_source/ver_0_12.md | 2 +- .../clearml_server/open_source/ver_0_13.md | 10 +- .../clearml_server/open_source/ver_0_14.md | 14 +-- .../clearml_server/open_source/ver_0_15.md | 18 ++-- .../clearml_server/open_source/ver_0_16.md | 6 +- .../clearml_server/open_source/ver_0_17.md | 8 +- .../clearml_server/open_source/ver_1_0.md | 18 ++-- .../clearml_server/open_source/ver_1_1.md | 4 +- .../clearml_server/open_source/ver_1_10.md | 12 +-- .../clearml_server/open_source/ver_1_11.md | 12 +-- .../clearml_server/open_source/ver_1_12.md | 16 +-- .../clearml_server/open_source/ver_1_13.md | 18 ++-- .../clearml_server/open_source/ver_1_14.md | 4 +- .../clearml_server/open_source/ver_1_15.md | 10 +- .../clearml_server/open_source/ver_1_16.md | 6 +- .../clearml_server/open_source/ver_1_17.md | 24 ++--- .../clearml_server/open_source/ver_1_2.md | 14 +-- .../clearml_server/open_source/ver_1_3.md | 10 +- .../clearml_server/open_source/ver_1_4.md | 22 ++-- .../clearml_server/open_source/ver_1_5.md | 8 +- .../clearml_server/open_source/ver_1_6.md | 14 +-- .../clearml_server/open_source/ver_1_7.md | 12 +-- .../clearml_server/open_source/ver_1_8.md | 12 +-- .../clearml_server/open_source/ver_1_9.md | 12 +-- .../clearml_server/open_source/ver_2_0.md | 6 +- docs/release_notes/clearml_serving/ver_1_2.md | 2 +- docs/release_notes/clearml_serving/ver_1_3.md | 14 +-- .../release_notes/sdk/open_source/ver_0_10.md | 12 +-- .../release_notes/sdk/open_source/ver_0_11.md | 4 +- .../release_notes/sdk/open_source/ver_0_12.md | 14 +-- .../release_notes/sdk/open_source/ver_0_13.md | 14 +-- .../release_notes/sdk/open_source/ver_0_14.md | 16 +-- .../release_notes/sdk/open_source/ver_0_15.md | 28 ++--- .../release_notes/sdk/open_source/ver_0_16.md | 74 ++++++------- .../release_notes/sdk/open_source/ver_0_17.md | 38 +++---- docs/release_notes/sdk/open_source/ver_1_0.md | 40 +++---- docs/release_notes/sdk/open_source/ver_1_1.md | 100 +++++++++--------- .../release_notes/sdk/open_source/ver_1_10.md | 42 ++++---- .../release_notes/sdk/open_source/ver_1_11.md | 16 +-- .../release_notes/sdk/open_source/ver_1_12.md | 18 ++-- .../release_notes/sdk/open_source/ver_1_13.md | 28 ++--- .../release_notes/sdk/open_source/ver_1_14.md | 18 ++-- .../release_notes/sdk/open_source/ver_1_15.md | 18 ++-- .../release_notes/sdk/open_source/ver_1_16.md | 24 ++--- .../release_notes/sdk/open_source/ver_1_17.md | 4 +- docs/release_notes/sdk/open_source/ver_1_2.md | 36 +++---- docs/release_notes/sdk/open_source/ver_1_3.md | 18 ++-- docs/release_notes/sdk/open_source/ver_1_4.md | 24 ++--- docs/release_notes/sdk/open_source/ver_1_5.md | 14 +-- docs/release_notes/sdk/open_source/ver_1_6.md | 18 ++-- docs/release_notes/sdk/open_source/ver_1_7.md | 22 ++-- docs/release_notes/sdk/open_source/ver_1_8.md | 20 ++-- docs/release_notes/sdk/open_source/ver_1_9.md | 46 ++++---- 190 files changed, 796 insertions(+), 796 deletions(-) diff --git a/README.md b/README.md index c94a1083..eeb012b5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@
- + **ClearML - The End-to-End Platform for AI Builders diff --git a/docs/apps/clearml_session.md b/docs/apps/clearml_session.md index 549a0b32..91f7d276 100644 --- a/docs/apps/clearml_session.md +++ b/docs/apps/clearml_session.md @@ -112,7 +112,7 @@ session. ### Kubernetes Support With ClearML k8s-glue you can enable launching ClearML sessions directly within Kubernetes pods. Set up the network and -ingress settings for `clearml-session` in the `sessions` section of the [`values.yaml`](https://github.com/allegroai/clearml-helm-charts/blob/main/charts/clearml-agent/values.yaml) +ingress settings for `clearml-session` in the `sessions` section of the [`values.yaml`](https://github.com/clearml/clearml-helm-charts/blob/main/charts/clearml-agent/values.yaml) file. Make sure to set the following values: diff --git a/docs/apps/clearml_task.md b/docs/apps/clearml_task.md index dd7fb019..f40bf3cd 100644 --- a/docs/apps/clearml_task.md +++ b/docs/apps/clearml_task.md @@ -71,7 +71,7 @@ errors in identifying the correct default branch. | `--packages` | Manually specify a list of required packages. Example: `--packages "tqdm>=2.1" "scikit-learn"` | No | | `--project`| Set the project name for the task (required, unless using `--base-task-id`). If the named project does not exist, it is created on-the-fly | Yes | | `--queue` | Select a task's execution queue. If not provided, a task is created but not launched | No | -| `--repo` | URL of remote repository. Example: `--repo https://github.com/allegroai/clearml.git` | No | +| `--repo` | URL of remote repository. Example: `--repo https://github.com/clearml/clearml.git` | No | | `--requirements` | Specify `requirements.txt` file to install when setting the session. By default, the` requirements.txt` from the repository will be used | No | | `--script` | Entry point script for the remote execution. When used with `--repo`, input the script's relative path inside the repository. For example: `--script source/train.py`. When used with `--folder`, it supports a direct path to a file inside the local repository itself, for example: `--script ~/project/source/train.py` | Yes | | `--skip-task-init` | If set, `Task.init()` call is not added to the entry point, and is assumed to be called within the script | No | @@ -87,10 +87,10 @@ These commands demonstrate a few useful use cases for `clearml-task`. ### Executing Code from a Remote Repository ```bash -clearml-task --project examples --name remote_test --repo https://github.com/allegroai/events.git --branch master --script /webinar-0620/keras_mnist.py --args batch_size=64 epochs=1 --queue default +clearml-task --project examples --name remote_test --repo https://github.com/clearml/events.git --branch master --script /webinar-0620/keras_mnist.py --args batch_size=64 epochs=1 --queue default ``` -The `keras_mnist.py` script from the [events](https://github.com/allegroai/events) GitHub repository is imported as a +The `keras_mnist.py` script from the [events](https://github.com/clearml/events) GitHub repository is imported as a ClearML task named `remote_test` in the `examples` project. Its command line arguments `batch_size` and `epochs` values are set, and the task is enqueued for execution on the `default` queue. diff --git a/docs/clearml_agent/clearml_agent_deployment.md b/docs/clearml_agent/clearml_agent_deployment.md index c95ee43e..c23ca2e1 100644 --- a/docs/clearml_agent/clearml_agent_deployment.md +++ b/docs/clearml_agent/clearml_agent_deployment.md @@ -107,10 +107,10 @@ ClearML Agent is deployed onto a Kubernetes cluster using **Kubernetes-Glue**, w You can deploy ClearML Agent onto Kubernetes using one of the following methods: 1. **ClearML Agent Helm Chart (Recommended)**: - Use the [ClearML Agent Helm Chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-agent) to spin up an agent pod acting as a controller. This is the recommended and scalable approach. + Use the [ClearML Agent Helm Chart](https://github.com/clearml/clearml-helm-charts/tree/main/charts/clearml-agent) to spin up an agent pod acting as a controller. This is the recommended and scalable approach. 2. **K8s Glue Script**: - Run a [K8s Glue script](https://github.com/allegroai/clearml-agent/blob/master/examples/k8s_glue_example.py) on a Kubernetes CPU node. This approach is less scalable and typically suited for simpler use cases. + Run a [K8s Glue script](https://github.com/clearml/clearml-agent/blob/master/examples/k8s_glue_example.py) on a Kubernetes CPU node. This approach is less scalable and typically suited for simpler use cases. ### How It Works The ClearML Kubernetes-Glue performs the following: diff --git a/docs/clearml_agent/clearml_agent_fractional_gpus.md b/docs/clearml_agent/clearml_agent_fractional_gpus.md index f5d3e2e5..79c4c2b1 100644 --- a/docs/clearml_agent/clearml_agent_fractional_gpus.md +++ b/docs/clearml_agent/clearml_agent_fractional_gpus.md @@ -214,7 +214,7 @@ Where `` must be set to one of the following values: * "0.875" ## Container-based Memory Limits -Use [`clearml-fractional-gpu`](https://github.com/allegroai/clearml-fractional-gpu)'s pre-packaged containers with +Use [`clearml-fractional-gpu`](https://github.com/clearml/clearml-fractional-gpu)'s pre-packaged containers with built-in hard memory limitations. Workloads running in these containers will only be able to use up to the container's memory limit. Multiple isolated workloads can run on the same GPU without impacting each other. @@ -223,7 +223,7 @@ memory limit. Multiple isolated workloads can run on the same GPU without impact #### Manual Execution 1. Choose the container with the appropriate memory limit. ClearML supports CUDA 11.x and CUDA 12.x with memory limits -ranging from 2 GB to 12 GB (see [clearml-fractional-gpu repository](https://github.com/allegroai/clearml-fractional-gpu/blob/main/README.md#-containers) for full list). +ranging from 2 GB to 12 GB (see [clearml-fractional-gpu repository](https://github.com/clearml/clearml-fractional-gpu/blob/main/README.md#-containers) for full list). 1. Launch the container: ```bash @@ -312,7 +312,7 @@ when limiting memory usage. Build your own custom fractional GPU container by inheriting from one of ClearML's containers: In your Dockerfile, make sure to include `From ` so the container will inherit from the relevant container. -See example custom Dockerfiles in the [clearml-fractional-gpu repository](https://github.com/allegroai/clearml-fractional-gpu/tree/main/examples). +See example custom Dockerfiles in the [clearml-fractional-gpu repository](https://github.com/clearml/clearml-fractional-gpu/tree/main/examples). ## Kubernetes Static MIG Fractions Set up NVIDIA MIG (Multi-Instance GPU) support for Kubernetes to define GPU fraction profiles for specific workloads diff --git a/docs/clearml_agent/clearml_agent_setup.md b/docs/clearml_agent/clearml_agent_setup.md index 2f460f19..a64eb93a 100644 --- a/docs/clearml_agent/clearml_agent_setup.md +++ b/docs/clearml_agent/clearml_agent_setup.md @@ -146,7 +146,7 @@ In case a `clearml.conf` file already exists, add a few ClearML Agent specific c worker_id: "" } ``` - View a complete ClearML Agent configuration file sample including an `agent` section [here](https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf). + View a complete ClearML Agent configuration file sample including an `agent` section [here](https://github.com/clearml/clearml-agent/blob/master/docs/clearml.conf). 1. Save the configuration. diff --git a/docs/clearml_data/data_management_examples/data_man_cifar_classification.md b/docs/clearml_data/data_management_examples/data_man_cifar_classification.md index a0071000..1e4cc2a3 100644 --- a/docs/clearml_data/data_management_examples/data_man_cifar_classification.md +++ b/docs/clearml_data/data_management_examples/data_man_cifar_classification.md @@ -74,7 +74,7 @@ In the panel's **CONTENT** tab, you can see a table summarizing version contents Now that you have a new dataset registered, you can consume it. -The [data_ingestion.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/data_ingestion.py) example +The [data_ingestion.py](https://github.com/clearml/clearml/blob/master/examples/datasets/data_ingestion.py) example script demonstrates using the dataset within Python code. ```python diff --git a/docs/clearml_data/data_management_examples/data_man_folder_sync.md b/docs/clearml_data/data_management_examples/data_man_folder_sync.md index 85563b9c..a222459d 100644 --- a/docs/clearml_data/data_management_examples/data_man_folder_sync.md +++ b/docs/clearml_data/data_management_examples/data_man_folder_sync.md @@ -9,7 +9,7 @@ from time to time. When the point of truth is updated, users can call `clearml-d changes (file addition, modification, or removal) will be reflected in ClearML. ## Prerequisites -1. First, make sure that you have cloned the [clearml](https://github.com/allegroai/clearml) repository. It contains all +1. First, make sure that you have cloned the [clearml](https://github.com/clearml/clearml) repository. It contains all the needed files. 1. Open terminal and change directory to the cloned repository's examples folder diff --git a/docs/clearml_data/data_management_examples/data_man_python.md b/docs/clearml_data/data_management_examples/data_man_python.md index ddebe234..c8e5ea14 100644 --- a/docs/clearml_data/data_management_examples/data_man_python.md +++ b/docs/clearml_data/data_management_examples/data_man_python.md @@ -2,14 +2,14 @@ title: Data Management with Python --- -The [dataset_creation.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/dataset_creation.py) and -[data_ingestion.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/data_ingestion.py) scripts +The [dataset_creation.py](https://github.com/clearml/clearml/blob/master/examples/datasets/dataset_creation.py) and +[data_ingestion.py](https://github.com/clearml/clearml/blob/master/examples/datasets/data_ingestion.py) scripts together demonstrate how to use ClearML's [`Dataset`](../../references/sdk/dataset.md) class to create a dataset and subsequently ingest the data. ## Dataset Creation -The [dataset_creation.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/dataset_creation.py) script +The [dataset_creation.py](https://github.com/clearml/clearml/blob/master/examples/datasets/dataset_creation.py) script demonstrates how to do the following: * Create a dataset and add files to it * Upload the dataset to the ClearML Server @@ -85,7 +85,7 @@ In the panel's **CONTENT** tab, you can see a table summarizing version contents Now that a new dataset is registered, you can consume it! -The [data_ingestion.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/data_ingestion.py) script +The [data_ingestion.py](https://github.com/clearml/clearml/blob/master/examples/datasets/data_ingestion.py) script demonstrates data ingestion using the dataset created in the first script. The following script gets the dataset and uses [`Dataset.get_local_copy()`](../../references/sdk/dataset.md#get_local_copy) diff --git a/docs/clearml_data/data_management_examples/data_man_simple.md b/docs/clearml_data/data_management_examples/data_man_simple.md index b9733c09..03f4fa1c 100644 --- a/docs/clearml_data/data_management_examples/data_man_simple.md +++ b/docs/clearml_data/data_management_examples/data_man_simple.md @@ -5,7 +5,7 @@ title: Data Management from CLI In this example we'll create a simple dataset and demonstrate basic actions on it, using the `clearml-data` CLI. ## Prerequisites -1. First, make sure that you have cloned the [clearml](https://github.com/allegroai/clearml) repository. It contains all +1. First, make sure that you have cloned the [clearml](https://github.com/clearml/clearml) repository. It contains all the needed files. 1. Open terminal and change directory to the cloned repository's examples folder: diff --git a/docs/clearml_sdk/clearml_sdk.md b/docs/clearml_sdk/clearml_sdk.md index 2b2e911c..d5802c82 100644 --- a/docs/clearml_sdk/clearml_sdk.md +++ b/docs/clearml_sdk/clearml_sdk.md @@ -57,7 +57,7 @@ ClearML's `optimization` module includes classes that support hyperparameter opt controller class * Optimization search strategy classes including [Optuna](../references/sdk/hpo_optuna_optuna_optimizeroptuna.md), [HpBandSter](../references/sdk/hpo_hpbandster_bandster_optimizerbohb.md), [GridSearch](../references/sdk/hpo_optimization_gridsearch.md), [RandomSearch](../references/sdk/hpo_optimization_randomsearch.md), - and a base [SearchStrategy](https://github.com/allegroai/clearml/blob/master/clearml/automation/optimization.py#L310) + and a base [SearchStrategy](https://github.com/clearml/clearml/blob/master/clearml/automation/optimization.py#L310) that can be customized See the [HyperParameterOptimizer SDK reference page](../references/sdk/hpo_optimization_hyperparameteroptimizer.md). @@ -96,22 +96,22 @@ configurations, and other execution details. See [reference page](../references/sdk/automation_job_clearmljob.md). ### AutoScaler -The `AutoScaler` class facilitates implementing resource budgeting. See class methods [here](https://github.com/allegroai/clearml/blob/master/clearml/automation/auto_scaler.py). -ClearML also provides a class specifically for AWS autoscaling. See [code](https://github.com/allegroai/clearml/blob/master/clearml/automation/aws_auto_scaler.py#L22) -and [example script](https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py). +The `AutoScaler` class facilitates implementing resource budgeting. See class methods [here](https://github.com/clearml/clearml/blob/master/clearml/automation/auto_scaler.py). +ClearML also provides a class specifically for AWS autoscaling. See [code](https://github.com/clearml/clearml/blob/master/clearml/automation/aws_auto_scaler.py#L22) +and [example script](https://github.com/clearml/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py). ### TaskScheduler -The `TaskScheduler` class supports methods for scheduling periodic execution (like cron jobs). See the [code](https://github.com/allegroai/clearml/blob/master/clearml/automation/scheduler.py#L481) -and [example](https://github.com/allegroai/clearml/blob/master/examples/scheduler/cron_example.py). +The `TaskScheduler` class supports methods for scheduling periodic execution (like cron jobs). See the [code](https://github.com/clearml/clearml/blob/master/clearml/automation/scheduler.py#L481) +and [example](https://github.com/clearml/clearml/blob/master/examples/scheduler/cron_example.py). ### TriggerScheduler The `TriggerScheduler` class facilitates triggering task execution in the case that specific events occur in the system -(such as model publication, dataset creation, task failure). See [code](https://github.com/allegroai/clearml/blob/master/clearml/automation/trigger.py#L148) -and [usage example](https://github.com/allegroai/clearml/blob/master/examples/scheduler/trigger_example.py). +(such as model publication, dataset creation, task failure). See [code](https://github.com/clearml/clearml/blob/master/clearml/automation/trigger.py#L148) +and [usage example](https://github.com/clearml/clearml/blob/master/examples/scheduler/trigger_example.py). ## Examples -The `clearml` GitHub repository includes an [examples folder](https://github.com/allegroai/clearml/tree/master/examples) +The `clearml` GitHub repository includes an [examples folder](https://github.com/clearml/clearml/tree/master/examples) with example scripts demonstrating how to use the various functionalities of the ClearML SDK. These examples are preloaded in the [ClearML Hosted Service](https://app.clear.ml), and can be viewed, cloned, diff --git a/docs/clearml_sdk/task_sdk.md b/docs/clearml_sdk/task_sdk.md index b8eaec25..611e4a29 100644 --- a/docs/clearml_sdk/task_sdk.md +++ b/docs/clearml_sdk/task_sdk.md @@ -209,7 +209,7 @@ For example: task = Task.create( project_name='example', task_name='task template', - repo='https://github.com/allegroai/clearml.git', + repo='https://github.com/clearml/clearml.git', branch='master', script='examples/reporting/html_reporting.py', working_directory='.', @@ -423,7 +423,7 @@ different number of epochs and using a new base model: cloned_task.set_parameters({'epochs':7, 'lr': 0.5}) # Override git repo information -cloned_task.set_repo(repo="https://github.com/allegroai/clearml.git", branch="my_branch_name") +cloned_task.set_repo(repo="https://github.com/clearml/clearml.git", branch="my_branch_name") # Remove input model and set a new one cloned_task.remove_input_models(models_to_remove=[""]) cloned_task.set_input_model(model_id="") @@ -441,7 +441,7 @@ Task.enqueue( ) ``` -See enqueue [example](https://github.com/allegroai/clearml/blob/master/examples/automation/programmatic_orchestration.py). +See enqueue [example](https://github.com/clearml/clearml/blob/master/examples/automation/programmatic_orchestration.py). ## Advanced Flows @@ -720,7 +720,7 @@ preprocess_task = Task.get_task(task_id='the_preprocessing_task_id') local_csv = preprocess_task.artifacts['data'].get_local_copy() ``` -See more details in the [Using Artifacts example](https://github.com/allegroai/clearml/blob/master/examples/reporting/using_artifacts_example.py). +See more details in the [Using Artifacts example](https://github.com/clearml/clearml/blob/master/examples/reporting/using_artifacts_example.py). ## Models The following is an overview of working with models through a `Task` object. You can also work directly with model diff --git a/docs/clearml_serving/clearml_serving.md b/docs/clearml_serving/clearml_serving.md index 4d507bb5..43c4b84e 100644 --- a/docs/clearml_serving/clearml_serving.md +++ b/docs/clearml_serving/clearml_serving.md @@ -39,7 +39,7 @@ solution. ## Components -![ClearML Serving](https://github.com/allegroai/clearml-serving/raw/main/docs/design_diagram.png?raw=true) +![ClearML Serving](https://github.com/clearml/clearml-serving/raw/main/docs/design_diagram.png?raw=true) * **CLI** - Secure configuration interface for on-line model upgrade/deployment on running Serving Services diff --git a/docs/clearml_serving/clearml_serving_setup.md b/docs/clearml_serving/clearml_serving_setup.md index 4da8e030..1d7da9d4 100644 --- a/docs/clearml_serving/clearml_serving_setup.md +++ b/docs/clearml_serving/clearml_serving_setup.md @@ -37,7 +37,7 @@ The following page goes over how to set up and upgrade `clearml-serving`. 1. Clone the `clearml-serving` repository: ```bash - git clone https://github.com/allegroai/clearml-serving.git + git clone https://github.com/clearml/clearml-serving.git ``` 1. Edit the environment variables file (docker/example.env) with your clearml-server credentials and Serving Service UID. diff --git a/docs/clearml_serving/clearml_serving_tutorial.md b/docs/clearml_serving/clearml_serving_tutorial.md index 2fad3a66..64667af5 100644 --- a/docs/clearml_serving/clearml_serving_tutorial.md +++ b/docs/clearml_serving/clearml_serving_tutorial.md @@ -3,7 +3,7 @@ title: Tutorial --- In this tutorial, you will go over the model lifecycle -- from training to serving -- in the following steps: -* Training a model using the [sklearn example script](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/train_model.py) +* Training a model using the [sklearn example script](https://github.com/clearml/clearml-serving/blob/main/examples/sklearn/train_model.py) * Serving the model using **ClearML Serving** * Spinning the inference container @@ -22,7 +22,7 @@ Before executing the steps below, make sure you have completed `clearml-serving` Train a model using the example script. Start from the root directory of your local `clearml-serving` repository. 1. Create a Python virtual environment 1. Install the script requirements: `pip3 install -r examples/sklearn/requirements.txt` -1. Execute the [training script](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/train_model.py): `python3 examples/sklearn/train_model.py`. +1. Execute the [training script](https://github.com/clearml/clearml-serving/blob/main/examples/sklearn/train_model.py): `python3 examples/sklearn/train_model.py`. During execution, ClearML automatically registers the sklearn model and uploads it to the model repository. For information about explicit model registration, see [Registering and Deploying New Models Manually](#registering-and-deploying-new-models-manually). @@ -50,7 +50,7 @@ and downloaded in real time when updated. ### Step 3: Spin Inference Container Spin the Inference Container: -1. Customize container [Dockerfile](https://github.com/allegroai/clearml-serving/blob/main/clearml_serving/serving/Dockerfile) if needed +1. Customize container [Dockerfile](https://github.com/clearml/clearml-serving/blob/main/clearml_serving/serving/Dockerfile) if needed 1. Build container: ```bash @@ -76,7 +76,7 @@ everything is cached, responses will return almost immediately. :::note Review the model repository in the ClearML web UI, under the "serving examples" Project on your ClearML -account/server ([free hosted](https://app.clear.ml) or [self-deployed](https://github.com/allegroai/clearml-server)). +account/server ([free hosted](https://app.clear.ml) or [self-deployed](https://github.com/clearml/clearml-server)). Inference services status, console outputs and machine metrics are available in the ClearML UI in the Serving Service project (default: "DevOps" project). @@ -207,7 +207,7 @@ Example: ### Model Monitoring and Performance Metrics -![Grafana Screenshot](https://github.com/allegroai/clearml-serving/raw/main/docs/grafana_screenshot.png) +![Grafana Screenshot](https://github.com/clearml/clearml-serving/raw/main/docs/grafana_screenshot.png) ClearML serving instances send serving statistics (count/latency) automatically to Prometheus and Grafana can be used to visualize and create live dashboards. @@ -232,7 +232,7 @@ that you will be able to visualize on Grafana. :::info time-series values You can also log time-series values with `--variable-value x2` or discrete results (e.g. classifications strings) with `--variable-enum animal=cat,dog,sheep`. Additional custom variables can be added in the preprocess and postprocess with -a call to `collect_custom_statistics_fn({'new_var': 1.337})`. See [preprocess_template.py](https://github.com/allegroai/clearml-serving/blob/main/clearml_serving/preprocess/preprocess_template.py). +a call to `collect_custom_statistics_fn({'new_var': 1.337})`. See [preprocess_template.py](https://github.com/clearml/clearml-serving/blob/main/clearml_serving/preprocess/preprocess_template.py). ::: With the new metrics logged, you can create a visualization dashboard over the latency of the calls, and the output distribution. @@ -258,10 +258,10 @@ You can also specify per-endpoint log frequency with the `clearml-serving` CLI. See examples of ClearML Serving with other supported frameworks: -* [scikit-learn](https://github.com/allegroai/clearml-serving/blob/main/examples/sklearn/readme.md) - Random data -* [scikit-learn Model Ensemble](https://github.com/allegroai/clearml-serving/blob/main/examples/ensemble/readme.md) - Random data -* [XGBoost](https://github.com/allegroai/clearml-serving/blob/main/examples/xgboost/readme.md) - Iris dataset -* [LightGBM](https://github.com/allegroai/clearml-serving/blob/main/examples/lightgbm/readme.md) - Iris dataset -* [PyTorch](https://github.com/allegroai/clearml-serving/blob/main/examples/pytorch/readme.md) - MNIST dataset -* [TensorFlow/Keras](https://github.com/allegroai/clearml-serving/blob/main/examples/keras/readme.md) - MNIST dataset -* [Model Pipeline](https://github.com/allegroai/clearml-serving/blob/main/examples/pipeline/readme.md) - Random data +* [scikit-learn](https://github.com/clearml/clearml-serving/blob/main/examples/sklearn/readme.md) - Random data +* [scikit-learn Model Ensemble](https://github.com/clearml/clearml-serving/blob/main/examples/ensemble/readme.md) - Random data +* [XGBoost](https://github.com/clearml/clearml-serving/blob/main/examples/xgboost/readme.md) - Iris dataset +* [LightGBM](https://github.com/clearml/clearml-serving/blob/main/examples/lightgbm/readme.md) - Iris dataset +* [PyTorch](https://github.com/clearml/clearml-serving/blob/main/examples/pytorch/readme.md) - MNIST dataset +* [TensorFlow/Keras](https://github.com/clearml/clearml-serving/blob/main/examples/keras/readme.md) - MNIST dataset +* [Model Pipeline](https://github.com/clearml/clearml-serving/blob/main/examples/pipeline/readme.md) - Random data diff --git a/docs/cloud_autoscaling/autoscaling_overview.md b/docs/cloud_autoscaling/autoscaling_overview.md index 0366cbf0..381b63be 100644 --- a/docs/cloud_autoscaling/autoscaling_overview.md +++ b/docs/cloud_autoscaling/autoscaling_overview.md @@ -100,5 +100,5 @@ underutilized nodes. See [charts](https://github.com/kubernetes/autoscaler/tree/ :::important Enterprise features The ClearML Enterprise plan supports K8S servicing multiple ClearML queues, as well as providing a pod template for each -queue for describing the resources for each pod to use. See [ClearML Helm Charts](https://github.com/allegroai/clearml-helm-charts/tree/main). +queue for describing the resources for each pod to use. See [ClearML Helm Charts](https://github.com/clearml/clearml-helm-charts/tree/main). ::: diff --git a/docs/community.md b/docs/community.md index 4d690a7e..31dbd5dd 100644 --- a/docs/community.md +++ b/docs/community.md @@ -4,7 +4,7 @@ title: Community Resources ## Join the ClearML Conversation -For feature requests or bug reports, see **ClearML** [GitHub issues](https://github.com/allegroai/clearml/issues). +For feature requests or bug reports, see **ClearML** [GitHub issues](https://github.com/clearml/clearml/issues). If you have any questions, post on the **ClearML** [Slack channel](https://joinslack.clear.ml). @@ -28,8 +28,8 @@ Firstly, thank you for taking the time to contribute! Contributions come in many forms: -* Reporting [issues](https://github.com/allegroai/clearml/issues) you've come upon -* Participating in issue discussions in the [issue tracker](https://github.com/allegroai/clearml/issues) and the +* Reporting [issues](https://github.com/clearml/clearml/issues) you've come upon +* Participating in issue discussions in the [issue tracker](https://github.com/clearml/clearml/issues) and the [ClearML Community Slack space](https://joinslack.clear.ml) * Suggesting new features or enhancements * Implementing new features or fixing outstanding issues @@ -40,7 +40,7 @@ The list above is primarily guidelines, not rules. Use your best judgment and fe By following these guidelines, you help maintainers and the community understand your report, reproduce the behavior, and find related reports. -Before reporting an issue, please check whether it already appears [here](https://github.com/allegroai/clearml/issues). If +Before reporting an issue, please check whether it already appears [here](https://github.com/clearml/clearml/issues). If it does, join the ongoing discussion instead. :::note @@ -85,7 +85,7 @@ Enhancement suggestions are tracked as GitHub issues. After you determine which Before you submit a new PR: -* Verify that the work you plan to merge addresses an existing [issue](https://github.com/allegroai/clearml/issues) (if not, open a new one) +* Verify that the work you plan to merge addresses an existing [issue](https://github.com/clearml/clearml/issues) (if not, open a new one) * Check related discussions in the [ClearML Slack community](https://joinslack.clear.ml) (or start your own discussion on the ``#clearml-dev`` channel) * Make sure your code conforms to the ClearML coding standards by running: diff --git a/docs/configs/clearml_conf.md b/docs/configs/clearml_conf.md index cc172aa9..525445b2 100644 --- a/docs/configs/clearml_conf.md +++ b/docs/configs/clearml_conf.md @@ -14,7 +14,7 @@ This reference page is organized by configuration file section: * [files](#files-section) - Define auto-generated files to apply into local file system -See an [example configuration file](https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf) +See an [example configuration file](https://github.com/clearml/clearml-agent/blob/master/docs/clearml.conf) in the ClearML Agent GitHub repository. :::info diff --git a/docs/deploying_clearml/clearml_server_config.md b/docs/deploying_clearml/clearml_server_config.md index 4b3f8036..0c69cb66 100644 --- a/docs/deploying_clearml/clearml_server_config.md +++ b/docs/deploying_clearml/clearml_server_config.md @@ -101,7 +101,7 @@ The ClearML Server uses the following configuration files: * `services.conf` When starting up, the ClearML Server will look for these configuration files, in the `/opt/clearml/config` directory -(this path can be modified using the `CLEARML_CONFIG_DIR` environment variable). The default configuration files are in the [clearml-server](https://github.com/allegroai/clearml-server/tree/master/apiserver/config/default) repository. +(this path can be modified using the `CLEARML_CONFIG_DIR` environment variable). The default configuration files are in the [clearml-server](https://github.com/clearml/clearml-server/tree/master/apiserver/config/default) repository. If you want to modify server configuration, and the relevant configuration file doesn't exist, you can create the file, and input the relevant modified configuration. diff --git a/docs/deploying_clearml/clearml_server_es7_migration.md b/docs/deploying_clearml/clearml_server_es7_migration.md index 0360dbf1..abc6b671 100644 --- a/docs/deploying_clearml/clearml_server_es7_migration.md +++ b/docs/deploying_clearml/clearml_server_es7_migration.md @@ -72,10 +72,10 @@ and ClearML Server needs to be installed. 1. Download the migration package archive: ``` - curl -L -O https://github.com/allegroai/clearml-server/releases/download/0.16.0/trains-server-0.16.0-migration.zip + curl -L -O https://github.com/clearml/clearml-server/releases/download/0.16.0/trains-server-0.16.0-migration.zip ``` - If the file needs to be downloaded manually, use this direct link: [trains-server-0.16.0-migration.zip](https://github.com/allegroai/clearml-server/releases/download/0.16.0/trains-server-0.16.0-migration.zip). + If the file needs to be downloaded manually, use this direct link: [trains-server-0.16.0-migration.zip](https://github.com/clearml/clearml-server/releases/download/0.16.0/trains-server-0.16.0-migration.zip). 1. Extract the archive: @@ -109,7 +109,7 @@ and ClearML Server needs to be installed. 1. Clone the `trains-server-k8s` repository and change to the new `trains-server-k8s/upgrade-elastic` directory: ``` - git clone https://github.com/allegroai/clearml-server-k8s.git && cd clearml-server-k8s/upgrade-elastic + git clone https://github.com/clearml/clearml-server-k8s.git && cd clearml-server-k8s/upgrade-elastic ``` 1. Create the `upgrade-elastic` namespace and deployments: @@ -170,7 +170,7 @@ If the migration script does not complete successfully, the migration script pri :::important For help in resolving migration issues, check the **ClearML** [Slack channel](https://joinslack.clear.ml), -[GitHub issues](https://github.com/allegroai/clearml-server/issues), and the **ClearML Server** sections of the [FAQ](../faq.md). +[GitHub issues](https://github.com/clearml/clearml-server/issues), and the **ClearML Server** sections of the [FAQ](../faq.md). ::: ### Upgrading to ClearML Server v.1.2 or Newer diff --git a/docs/deploying_clearml/clearml_server_kubernetes_helm.md b/docs/deploying_clearml/clearml_server_kubernetes_helm.md index 363de29d..5325b336 100644 --- a/docs/deploying_clearml/clearml_server_kubernetes_helm.md +++ b/docs/deploying_clearml/clearml_server_kubernetes_helm.md @@ -21,7 +21,7 @@ Ensure that the `helm` binary is in the PATH of your shell. ## Deployment You will create a multi-node Kubernetes cluster using Helm, and then install ClearML in your cluster. For deployment -instructions with up-to-date Helms charts, see the [clearml-helm-charts repository](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml#local-environment). +instructions with up-to-date Helms charts, see the [clearml-helm-charts repository](https://github.com/clearml/clearml-helm-charts/tree/main/charts/clearml#local-environment). :::warning Server Access By default, ClearML Server launches with unrestricted access. To restrict ClearML Server access, follow the diff --git a/docs/deploying_clearml/clearml_server_mongo44_migration.md b/docs/deploying_clearml/clearml_server_mongo44_migration.md index ca529b80..0e69466f 100644 --- a/docs/deploying_clearml/clearml_server_mongo44_migration.md +++ b/docs/deploying_clearml/clearml_server_mongo44_migration.md @@ -11,7 +11,7 @@ In v1.2, the MongoDB subsystem of ClearML Server has been upgraded from version necessitates the migration of the database contents to be compatible with the new version. :::note Kubernetes installations -[ClearML’s helm chart](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml) is already running +[ClearML’s helm chart](https://github.com/clearml/clearml-helm-charts/tree/main/charts/clearml) is already running mongodb version 4.4. If your ClearML server had been deployed with this chart (with the default mongodb bitnami chart) - You can stop reading here, as no migration is required. ::: @@ -49,7 +49,7 @@ To avoid data corruption, shut down your ClearML server before applying the migr ### Migrating by Script A migration script is available to automatically run this process for all supported OSs. -[Download the script](https://github.com/allegroai/clearml-server/releases/download/1.2.0/clearml-server-1.2.0-migration.py) and run it on your ClearML server. +[Download the script](https://github.com/clearml/clearml-server/releases/download/1.2.0/clearml-server-1.2.0-migration.py) and run it on your ClearML server. Run `clearml-server-1.2.0-migration.py -–help` to see execution options. Note the script will create a backup archive of your data in the original directory. diff --git a/docs/deploying_clearml/clearml_server_security.md b/docs/deploying_clearml/clearml_server_security.md index 02baf210..73e00a13 100644 --- a/docs/deploying_clearml/clearml_server_security.md +++ b/docs/deploying_clearml/clearml_server_security.md @@ -39,7 +39,7 @@ with the ClearML SDK. However, this also means that the **server must be secured** by either preventing any external access, or by changing defaults so that the server's credentials are not publicly known. -The ClearML Server default secrets can be found [here](https://github.com/allegroai/clearml-server/blob/master/apiserver/config/default/secure.conf), and can be changed using the `secure.conf` configuration file or using environment variables +The ClearML Server default secrets can be found [here](https://github.com/clearml/clearml-server/blob/master/apiserver/config/default/secure.conf), and can be changed using the `secure.conf` configuration file or using environment variables (see [ClearML Server Feature Configurations](clearml_server_config.md#clearml-server-feature-configurations)). Specifically, the relevant settings are: @@ -101,6 +101,6 @@ services: :::important When generating new user keys and secrets, make sure to use sufficiently long strings (we use 30 chars for keys and 50-60 -chars for secrets). See [here](https://github.com/allegroai/clearml-server/blob/master/apiserver/service_repo/auth/utils.py) +chars for secrets). See [here](https://github.com/clearml/clearml-server/blob/master/apiserver/service_repo/auth/utils.py) for Python example code to generate these strings. ::: diff --git a/docs/deploying_clearml/upgrade_server_gcp.md b/docs/deploying_clearml/upgrade_server_gcp.md index 93e53998..b352ef68 100644 --- a/docs/deploying_clearml/upgrade_server_gcp.md +++ b/docs/deploying_clearml/upgrade_server_gcp.md @@ -7,7 +7,7 @@ title: Google Cloud Platform MongoDB major version was upgraded from `v5.x` to `6.x`. Please note that if your current ClearML Server version is older than `v1.17` (where MongoDB `v5.x` was first used), you'll need to first upgrade to ClearML Server v1.17. -First upgrade to ClearML Server v1.17 following the procedure below and using [this `docker-compose` file](https://github.com/allegroai/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose.yml). Once successfully upgraded, +First upgrade to ClearML Server v1.17 following the procedure below and using [this `docker-compose` file](https://github.com/clearml/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose.yml). Once successfully upgraded, you can proceed to upgrade to v2.x. diff --git a/docs/deploying_clearml/upgrade_server_kubernetes_helm.md b/docs/deploying_clearml/upgrade_server_kubernetes_helm.md index cdb140e4..a2d2fb0a 100644 --- a/docs/deploying_clearml/upgrade_server_kubernetes_helm.md +++ b/docs/deploying_clearml/upgrade_server_kubernetes_helm.md @@ -16,7 +16,7 @@ helm upgrade clearml allegroai/clearml helm upgrade clearml allegroai/clearml --version -f custom_values.yaml ``` -See the [clearml-helm-charts repository](https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml#local-environment) +See the [clearml-helm-charts repository](https://github.com/clearml/clearml-helm-charts/tree/main/charts/clearml#local-environment) to view the up-to-date charts. :::tip diff --git a/docs/deploying_clearml/upgrade_server_linux_mac.md b/docs/deploying_clearml/upgrade_server_linux_mac.md index 818fe375..7be52d46 100644 --- a/docs/deploying_clearml/upgrade_server_linux_mac.md +++ b/docs/deploying_clearml/upgrade_server_linux_mac.md @@ -7,7 +7,7 @@ title: Linux or macOS MongoDB major version was upgraded from `v5.x` to `6.x`. Please note that if your current ClearML Server version is older than `v1.17` (where MongoDB `v5.x` was first used), you'll need to first upgrade to ClearML Server v1.17. -First upgrade to ClearML Server v1.17 following the procedure below and using [this `docker-compose` file](https://github.com/allegroai/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose.yml). Once successfully upgraded, +First upgrade to ClearML Server v1.17 following the procedure below and using [this `docker-compose` file](https://github.com/clearml/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose.yml). Once successfully upgraded, you can proceed to upgrade to v2.x. diff --git a/docs/deploying_clearml/upgrade_server_win.md b/docs/deploying_clearml/upgrade_server_win.md index 36ec3007..ab22f542 100644 --- a/docs/deploying_clearml/upgrade_server_win.md +++ b/docs/deploying_clearml/upgrade_server_win.md @@ -7,7 +7,7 @@ title: Windows MongoDB major version was upgraded from `v5.x` to `6.x`. Please note that if your current ClearML Server version is older than `v1.17` (where MongoDB `v5.x` was first used), you'll need to first upgrade to ClearML Server v1.17. -First upgrade to ClearML Server v1.17 following the procedure below and using [this `docker-compose` file](https://github.com/allegroai/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose-win10.yml). Once successfully upgraded, +First upgrade to ClearML Server v1.17 following the procedure below and using [this `docker-compose` file](https://github.com/clearml/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose-win10.yml). Once successfully upgraded, you can proceed to upgrade to v2.x. diff --git a/docs/faq.md b/docs/faq.md index 9324813f..3b5f642c 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -368,7 +368,7 @@ Your firewall may be preventing the connection. Try one of the following solutio ``` pip install -U clearml ``` - 1. Create a new `clearml.conf` configuration file (see a [sample configuration file](https://github.com/allegroai/clearml/blob/master/docs/clearml.conf)), containing: + 1. Create a new `clearml.conf` configuration file (see a [sample configuration file](https://github.com/clearml/clearml/blob/master/docs/clearml.conf)), containing: ``` api { verify_certificate = False } @@ -419,7 +419,7 @@ Conda and the [typing](https://pypi.org/project/typing/) package may have some c However, [since Python 3.5](https://docs.python.org/3.5/library/typing.html), the `typing` package is part of the standard library. -To resolve the error, uninstall `typing` and rerun your script. If this does not fix the issue, create a [new ClearML issue](https://github.com/allegroai/clearml/issues/new), including the full error, and your environment details. +To resolve the error, uninstall `typing` and rerun your script. If this does not fix the issue, create a [new ClearML issue](https://github.com/clearml/clearml/issues/new), including the full error, and your environment details. diff --git a/docs/fundamentals/agents_and_queues.md b/docs/fundamentals/agents_and_queues.md index 2c2e16e5..50509f53 100644 --- a/docs/fundamentals/agents_and_queues.md +++ b/docs/fundamentals/agents_and_queues.md @@ -6,7 +6,7 @@ Two major components of MLOps/LLMOps are experiment reproducibility, and the abi coupled with execution queues, address both these needs. A ClearML worker is instantiated by launching a ClearML Agent, which is the base for **Automation** in ClearML and can be leveraged to build automated pipelines, launch custom services -(e.g. a [monitor and alert service](https://github.com/allegroai/clearml/tree/master/examples/services/monitoring)) and more. +(e.g. a [monitor and alert service](https://github.com/clearml/clearml/tree/master/examples/services/monitoring)) and more. ## What Does a ClearML Agent Do? The ClearML agent allows users to execute code on any machine it's installed on, thus facilitating the diff --git a/docs/fundamentals/hpo.md b/docs/fundamentals/hpo.md index d384cc15..cd559d92 100644 --- a/docs/fundamentals/hpo.md +++ b/docs/fundamentals/hpo.md @@ -61,7 +61,7 @@ optimization. documentation and a [code example](../guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md). * **Random** uniform sampling of hyperparameters - [`automation.RandomSearch`](../references/sdk/hpo_optimization_randomsearch.md). * **Full grid** sampling strategy of every hyperparameter combination - [`automation.GridSearch`](../references/sdk/hpo_optimization_gridsearch.md). -* **Custom** - [`automation.optimization.SearchStrategy`](https://github.com/allegroai/clearml/blob/master/clearml/automation/optimization.py#L268) - Use a custom class and inherit from the ClearML automation base strategy class. +* **Custom** - [`automation.optimization.SearchStrategy`](https://github.com/clearml/clearml/blob/master/clearml/automation/optimization.py#L268) - Use a custom class and inherit from the ClearML automation base strategy class. ## Defining a Hyperparameter Optimization Search Example diff --git a/docs/fundamentals/task.md b/docs/fundamentals/task.md index 5c8cf6ab..afdb8633 100644 --- a/docs/fundamentals/task.md +++ b/docs/fundamentals/task.md @@ -113,7 +113,7 @@ they are attached to, and then retrieving the artifact with one of its following * `get_local_copy()` - caches the files for later use and returns a path to the cached file. * `get()` - use for Python objects. The method that returns the Python object. -See more details in the [Using Artifacts example](https://github.com/allegroai/clearml/blob/master/examples/reporting/using_artifacts_example.py). +See more details in the [Using Artifacts example](https://github.com/clearml/clearml/blob/master/examples/reporting/using_artifacts_example.py). ## Task Types Tasks have a *type* attribute, which denotes their purpose. This helps to further diff --git a/docs/getting_started/ds/ds_second_steps.md b/docs/getting_started/ds/ds_second_steps.md index 21b1640d..3aaf3f87 100644 --- a/docs/getting_started/ds/ds_second_steps.md +++ b/docs/getting_started/ds/ds_second_steps.md @@ -89,7 +89,7 @@ Calling `get_local_copy()` returns a local cached copy of the artifact. Therefor need to download the artifact again. Calling `get()` gets a deserialized pickled object. -Check out the [artifacts retrieval](https://github.com/allegroai/clearml/blob/master/examples/reporting/artifacts_retrieval.py) example code. +Check out the [artifacts retrieval](https://github.com/clearml/clearml/blob/master/examples/reporting/artifacts_retrieval.py) example code. ### Models @@ -112,10 +112,10 @@ Now, whenever the framework (TensorFlow/Keras/PyTorch etc.) stores a snapshot, t Loading models by a framework is also logged by the system; these models appear in an experiment's **Artifacts** tab, under the "Input Models" section. -Check out model snapshots examples for [TensorFlow](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py), -[PyTorch](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py), -[Keras](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py), -[scikit-learn](https://github.com/allegroai/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py). +Check out model snapshots examples for [TensorFlow](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py), +[PyTorch](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py), +[Keras](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py), +[scikit-learn](https://github.com/clearml/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py). #### Loading Models Loading a previously trained model is quite similar to loading artifacts. diff --git a/docs/getting_started/main.md b/docs/getting_started/main.md index 99bdcdfb..26fffcd0 100644 --- a/docs/getting_started/main.md +++ b/docs/getting_started/main.md @@ -35,19 +35,19 @@ training, and deploying models at every scale on any AI infrastructure. - + - + - + diff --git a/docs/getting_started/mlops/mlops_first_steps.md b/docs/getting_started/mlops/mlops_first_steps.md index cba8d3b7..34635cd3 100644 --- a/docs/getting_started/mlops/mlops_first_steps.md +++ b/docs/getting_started/mlops/mlops_first_steps.md @@ -39,7 +39,7 @@ required Python packages, and execute and monitor the process. ``` :::note - If you've already created credentials, you can copy-paste the default agent section from [here](https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf#L15) (this is optional. If the section is not provided the default values will be used) + If you've already created credentials, you can copy-paste the default agent section from [here](https://github.com/clearml/clearml-agent/blob/master/docs/clearml.conf#L15) (this is optional. If the section is not provided the default values will be used) ::: 1. Start the agent's daemon and assign it to a [queue](../../fundamentals/agents_and_queues.md#what-is-a-queue): diff --git a/docs/getting_started/video_tutorials/quick_introduction.md b/docs/getting_started/video_tutorials/quick_introduction.md index 7777f80f..b6028dc3 100644 --- a/docs/getting_started/video_tutorials/quick_introduction.md +++ b/docs/getting_started/video_tutorials/quick_introduction.md @@ -32,6 +32,6 @@ It's essentially a toolbox stuffed with everything you'll need to go from experi Doesn't matter if you're starting small or already in production, there's always a ClearML tool that can make your life easier. -Start for free at [app.clear.ml](https://app.clear.ml) or host your own server from our [GitHub page](https://github.com/allegroai/clearml-server). +Start for free at [app.clear.ml](https://app.clear.ml) or host your own server from our [GitHub page](https://github.com/clearml/clearml-server). diff --git a/docs/guides/advanced/execute_remotely.md b/docs/guides/advanced/execute_remotely.md index afe6628a..9502deb1 100644 --- a/docs/guides/advanced/execute_remotely.md +++ b/docs/guides/advanced/execute_remotely.md @@ -2,7 +2,7 @@ title: Remote Execution --- -The [execute_remotely_example](https://github.com/allegroai/clearml/blob/master/examples/advanced/execute_remotely_example.py) +The [execute_remotely_example](https://github.com/clearml/clearml/blob/master/examples/advanced/execute_remotely_example.py) script demonstrates the use of the [`Task.execute_remotely()`](../../references/sdk/task.md#execute_remotely) method. :::note diff --git a/docs/guides/advanced/multiple_tasks_single_process.md b/docs/guides/advanced/multiple_tasks_single_process.md index 74e9aeed..65314e56 100644 --- a/docs/guides/advanced/multiple_tasks_single_process.md +++ b/docs/guides/advanced/multiple_tasks_single_process.md @@ -2,7 +2,7 @@ title: Multiple Tasks in Single Process --- -The [multiple_tasks_single_process](https://github.com/allegroai/clearml/blob/master/examples/advanced/multiple_tasks_single_process.py) +The [multiple_tasks_single_process](https://github.com/clearml/clearml/blob/master/examples/advanced/multiple_tasks_single_process.py) script demonstrates the capability to log a single script in multiple ClearML tasks. In order to log a script in multiple tasks, each task needs to be initialized using [`Task.init()`](../../references/sdk/task.md#taskinit) diff --git a/docs/guides/automation/manual_random_param_search_example.md b/docs/guides/automation/manual_random_param_search_example.md index 1d791166..fd71f4ba 100644 --- a/docs/guides/automation/manual_random_param_search_example.md +++ b/docs/guides/automation/manual_random_param_search_example.md @@ -2,13 +2,13 @@ title: Manual Random Parameter Search --- -The [manual_random_param_search_example.py](https://github.com/allegroai/clearml/blob/master/examples/automation/manual_random_param_search_example.py) +The [manual_random_param_search_example.py](https://github.com/clearml/clearml/blob/master/examples/automation/manual_random_param_search_example.py) script demonstrates a random parameter search by automating the execution of a task multiple times, each time with a different set of random hyperparameters. This example accomplishes the automated random parameter search by doing the following: -1. Creating a template Task named `Keras HP optimization base`. To create it, run the [base_template_keras_simple.py](https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/base_template_keras_simple.py) +1. Creating a template Task named `Keras HP optimization base`. To create it, run the [base_template_keras_simple.py](https://github.com/clearml/clearml/blob/master/examples/optimization/hyper-parameter-optimization/base_template_keras_simple.py) script. This task must be executed first, so it will be stored in the server, and then it can be accessed, cloned, and modified by another Task. 1. Creating a parameter dictionary, which is connected to the Task by calling [`Task.connect()`](../../references/sdk/task.md#connect) diff --git a/docs/guides/automation/task_piping.md b/docs/guides/automation/task_piping.md index d0924632..9cbde00f 100644 --- a/docs/guides/automation/task_piping.md +++ b/docs/guides/automation/task_piping.md @@ -2,7 +2,7 @@ title: Programmatic Orchestration --- -The [programmatic_orchestration.py](https://github.com/allegroai/clearml/blob/master/examples/automation/programmatic_orchestration.py) +The [programmatic_orchestration.py](https://github.com/clearml/clearml/blob/master/examples/automation/programmatic_orchestration.py) example demonstrates: 1. Creating an instance of a Task from a template Task. 1. Customizing that instance by changing the value of a parameter @@ -11,7 +11,7 @@ example demonstrates: This example accomplishes a task pipe by doing the following: 1. Creating the template Task which is named `Toy Base Task`. It must be stored in ClearML Server before instances of - it can be created. To create it, run another ClearML example script, [toy_base_task.py](https://github.com/allegroai/clearml/blob/master/examples/automation/toy_base_task.py). + it can be created. To create it, run another ClearML example script, [toy_base_task.py](https://github.com/clearml/clearml/blob/master/examples/automation/toy_base_task.py). The template Task has a parameter dictionary, which is connected to the Task: `{'Example_Param': 1}`. 1. Back in `programmatic_orchestration.py`, creating a parameter dictionary, which is connected to the Task by calling [`Task.connect`](../../references/sdk/task.md#connect) so that the parameters are logged by ClearML. The dictionary contains the name of the parameter from the template diff --git a/docs/guides/clearml-task/clearml_task_tutorial.md b/docs/guides/clearml-task/clearml_task_tutorial.md index 718916b5..085f352c 100644 --- a/docs/guides/clearml-task/clearml_task_tutorial.md +++ b/docs/guides/clearml-task/clearml_task_tutorial.md @@ -2,7 +2,7 @@ title: ClearML Task Tutorial --- -In this tutorial, you will use `clearml-task` to execute [a script](https://github.com/allegroai/events/blob/master/webinar-0620/keras_mnist.py) +In this tutorial, you will use `clearml-task` to execute [a script](https://github.com/clearml/events/blob/master/webinar-0620/keras_mnist.py) on a remote or local machine, from a remote repository and your local machine. ### Prerequisites @@ -13,13 +13,13 @@ on a remote or local machine, from a remote repository and your local machine. ### Executing Code from a Remote Repository ``` bash -clearml-task --project keras_examples --name remote_test --repo https://github.com/allegroai/events.git --branch master --script /webinar-0620/keras_mnist.py --args batch_size=64 epochs=1 --queue default +clearml-task --project keras_examples --name remote_test --repo https://github.com/clearml/events.git --branch master --script /webinar-0620/keras_mnist.py --args batch_size=64 epochs=1 --queue default ``` This sets the following arguments: * `--project keras_examples --name remote_test` - The project and task names -* `--repo https://github.com/allegroai/events.git` - The repository's URL. By default, `clearml-task` uses the latest +* `--repo https://github.com/clearml/events.git` - The repository's URL. By default, `clearml-task` uses the latest commit from the master branch * `--branch master` - The repository branch * `--script /webinar-0620/keras_mnist.py` - The script to be executed @@ -48,8 +48,8 @@ Execution log at: https://app.clear.ml/projects/552d5399112d47029c146d5248570295 ### Executing a Local Script -For this example, use a local version of [this script](https://github.com/allegroai/events/blob/master/webinar-0620/keras_mnist.py). -1. Clone the [allegroai/events](https://github.com/allegroai/events) repository +For this example, use a local version of [this script](https://github.com/clearml/events/blob/master/webinar-0620/keras_mnist.py). +1. Clone the [allegroai/events](https://github.com/clearml/events) repository 1. Go to the root folder of the cloned repository 1. Run the following command: diff --git a/docs/guides/clearml_agent/executable_exp_containers.md b/docs/guides/clearml_agent/executable_exp_containers.md index e0a45063..35cd57da 100644 --- a/docs/guides/clearml_agent/executable_exp_containers.md +++ b/docs/guides/clearml_agent/executable_exp_containers.md @@ -4,13 +4,13 @@ title: Executable Task Containers This tutorial demonstrates using [`clearml-agent`](../../clearml_agent.md)'s [`build`](../../clearml_agent/clearml_agent_ref.md#build) command to package a task into an executable container. In this example, you will build a container image that, when -run, will automatically execute the [keras_tensorboard.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) +run, will automatically execute the [keras_tensorboard.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) script. ## Prerequisites * [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) installed and configured * [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured -* [clearml](https://github.com/allegroai/clearml) repo cloned (`git clone https://github.com/allegroai/clearml.git`) +* [clearml](https://github.com/clearml/clearml) repo cloned (`git clone https://github.com/clearml/clearml.git`) ## Creating the ClearML Task 1. Set up the task's execution environment: diff --git a/docs/guides/clearml_agent/exp_environment_containers.md b/docs/guides/clearml_agent/exp_environment_containers.md index 6bc2145d..0398e017 100644 --- a/docs/guides/clearml_agent/exp_environment_containers.md +++ b/docs/guides/clearml_agent/exp_environment_containers.md @@ -12,7 +12,7 @@ be used when running optimization tasks. ## Prerequisites * [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) installed and configured * [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured -* [clearml](https://github.com/allegroai/clearml) repo cloned (`git clone https://github.com/allegroai/clearml.git`) +* [clearml](https://github.com/clearml/clearml) repo cloned (`git clone https://github.com/clearml/clearml.git`) ## Creating the ClearML Task 1. Set up the task's execution environment: diff --git a/docs/guides/distributed/distributed_pytorch_example.md b/docs/guides/distributed/distributed_pytorch_example.md index 0d4c1843..c57e0341 100644 --- a/docs/guides/distributed/distributed_pytorch_example.md +++ b/docs/guides/distributed/distributed_pytorch_example.md @@ -2,7 +2,7 @@ title: PyTorch Distributed --- -The [pytorch_distributed_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_distributed_example.py) +The [pytorch_distributed_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_distributed_example.py) script demonstrates integrating ClearML into code that uses the [PyTorch Distributed Communications Package](https://pytorch.org/docs/stable/distributed.html) (`torch.distributed`). diff --git a/docs/guides/distributed/subprocess_example.md b/docs/guides/distributed/subprocess_example.md index 735e315c..42eab687 100644 --- a/docs/guides/distributed/subprocess_example.md +++ b/docs/guides/distributed/subprocess_example.md @@ -2,7 +2,7 @@ title: Subprocess --- -The [subprocess_example.py](https://github.com/allegroai/clearml/blob/master/examples/distributed/subprocess_example.py) +The [subprocess_example.py](https://github.com/clearml/clearml/blob/master/examples/distributed/subprocess_example.py) script demonstrates multiple subprocesses interacting and reporting to a main Task. The following happens in the script: * This script initializes a main Task and spawns subprocesses, each for an instances of that Task. * Each Task in a subprocess references the main Task by calling [`Task.current_task()`](../../references/sdk/task.md#taskcurrent_task), diff --git a/docs/guides/frameworks/autokeras/autokeras_imdb_example.md b/docs/guides/frameworks/autokeras/autokeras_imdb_example.md index 6b9b30f2..ceb7d42a 100644 --- a/docs/guides/frameworks/autokeras/autokeras_imdb_example.md +++ b/docs/guides/frameworks/autokeras/autokeras_imdb_example.md @@ -1,7 +1,7 @@ --- title: AutoKeras --- -The [autokeras_imdb_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) example +The [autokeras_imdb_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) example script demonstrates the integration of ClearML into code, which uses [autokeras](https://github.com/keras-team/autokeras). The example does the following: diff --git a/docs/guides/frameworks/catboost/catboost.md b/docs/guides/frameworks/catboost/catboost.md index ce698c45..2c6b0aeb 100644 --- a/docs/guides/frameworks/catboost/catboost.md +++ b/docs/guides/frameworks/catboost/catboost.md @@ -2,7 +2,7 @@ title: CatBoost --- -The [catboost_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/catboost/catboost_example.py) +The [catboost_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/catboost/catboost_example.py) example demonstrates the integration of ClearML into code that uses [CatBoost](https://catboost.ai). The example script does the following: diff --git a/docs/guides/frameworks/fastai/fastai_with_tensorboard.md b/docs/guides/frameworks/fastai/fastai_with_tensorboard.md index 6e37a046..1b58978e 100644 --- a/docs/guides/frameworks/fastai/fastai_with_tensorboard.md +++ b/docs/guides/frameworks/fastai/fastai_with_tensorboard.md @@ -1,11 +1,11 @@ --- title: Fast.ai --- -The [fastai_with_tensorboard_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/fastai/legacy/fastai_with_tensorboard_example.py) +The [fastai_with_tensorboard_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/fastai/legacy/fastai_with_tensorboard_example.py) example demonstrates the integration of ClearML into code that uses FastAI v1 and TensorBoard. :::note FastAI V2 -The ClearML repository also includes [examples using FastAI v2](https://github.com/allegroai/clearml/tree/master/examples/frameworks/fastai). +The ClearML repository also includes [examples using FastAI v2](https://github.com/clearml/clearml/tree/master/examples/frameworks/fastai). ::: diff --git a/docs/guides/frameworks/huggingface/transformers.md b/docs/guides/frameworks/huggingface/transformers.md index 03358fe0..c095f3a9 100644 --- a/docs/guides/frameworks/huggingface/transformers.md +++ b/docs/guides/frameworks/huggingface/transformers.md @@ -2,7 +2,7 @@ title: Transformers --- -The [Hugging Face Transformers example](https://github.com/allegroai/clearml/blob/master/examples/frameworks/huggingface/transformers.ipynb) +The [Hugging Face Transformers example](https://github.com/clearml/clearml/blob/master/examples/frameworks/huggingface/transformers.ipynb) demonstrates how to integrate ClearML into your Transformer's [Trainer](https://huggingface.co/docs/transformers/v4.34.1/en/main_classes/trainer) code. The Hugging Face Trainer automatically uses the built-in [`ClearMLCallback`](https://huggingface.co/docs/transformers/v4.34.1/en/main_classes/callback#transformers.integrations.ClearMLCallback) if the `clearml` package is already installed, to log Transformers models, parameters, scalars, and more. diff --git a/docs/guides/frameworks/keras/jupyter.md b/docs/guides/frameworks/keras/jupyter.md index f4dcd064..005e2414 100644 --- a/docs/guides/frameworks/keras/jupyter.md +++ b/docs/guides/frameworks/keras/jupyter.md @@ -2,7 +2,7 @@ title: Keras with Matplotlib - Jupyter Notebook --- -The [jupyter.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/jupyter.ipynb) example +The [jupyter.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/jupyter.ipynb) example demonstrates ClearML's automatic logging of code running in a Jupyter Notebook that uses Keras and Matplotlib. The example does the following: diff --git a/docs/guides/frameworks/keras/keras_tensorboard.md b/docs/guides/frameworks/keras/keras_tensorboard.md index 8330f8ed..f07f2c5e 100644 --- a/docs/guides/frameworks/keras/keras_tensorboard.md +++ b/docs/guides/frameworks/keras/keras_tensorboard.md @@ -3,11 +3,11 @@ title: Keras with TensorBoard --- The example below demonstrates the integration of ClearML into code which uses Keras and TensorBoard. -View it in [script](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) -or in [Jupyter Notebook](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/jupyter_keras_TB_example.ipynb). +View it in [script](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) +or in [Jupyter Notebook](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/jupyter_keras_TB_example.ipynb). :::note -The example in [Jupyter Notebook](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/jupyter_keras_TB_example.ipynb) +The example in [Jupyter Notebook](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/jupyter_keras_TB_example.ipynb) includes a clickable icon to open the notebook in Google Colab. ::: diff --git a/docs/guides/frameworks/lightgbm/lightgbm_example.md b/docs/guides/frameworks/lightgbm/lightgbm_example.md index 7b35d69b..5530d77f 100644 --- a/docs/guides/frameworks/lightgbm/lightgbm_example.md +++ b/docs/guides/frameworks/lightgbm/lightgbm_example.md @@ -2,7 +2,7 @@ title: LightGBM --- -The [lightgbm_example](https://github.com/allegroai/clearml/blob/master/examples/frameworks/lightgbm/lightgbm_example.py) +The [lightgbm_example](https://github.com/clearml/clearml/blob/master/examples/frameworks/lightgbm/lightgbm_example.py) script demonstrates the integration of ClearML into code that uses LightGBM. The example script does the following: diff --git a/docs/guides/frameworks/matplotlib/matplotlib_example.md b/docs/guides/frameworks/matplotlib/matplotlib_example.md index 857f1e2f..d8686aa1 100644 --- a/docs/guides/frameworks/matplotlib/matplotlib_example.md +++ b/docs/guides/frameworks/matplotlib/matplotlib_example.md @@ -5,11 +5,11 @@ title: Matplotlib The example below demonstrates integrating ClearML into code that uses `matplotlib` to plot scatter diagrams, and show images. ClearML automatically logs the diagrams and images. -View the example in [script](https://github.com/allegroai/clearml/blob/master/examples/frameworks/matplotlib/matplotlib_example.py) -or in [Jupyter Notebook](https://github.com/allegroai/clearml/blob/master/examples/frameworks/matplotlib/jupyter_matplotlib_example.ipynb). +View the example in [script](https://github.com/clearml/clearml/blob/master/examples/frameworks/matplotlib/matplotlib_example.py) +or in [Jupyter Notebook](https://github.com/clearml/clearml/blob/master/examples/frameworks/matplotlib/jupyter_matplotlib_example.ipynb). :::note -The example in [Jupyter Notebook](https://github.com/allegroai/clearml/blob/master/examples/frameworks/matplotlib/jupyter_matplotlib_example.ipynb) +The example in [Jupyter Notebook](https://github.com/clearml/clearml/blob/master/examples/frameworks/matplotlib/jupyter_matplotlib_example.ipynb) includes a clickable icon to open the notebook in Google Colab. ::: diff --git a/docs/guides/frameworks/megengine/megengine_mnist.md b/docs/guides/frameworks/megengine/megengine_mnist.md index 786abb2c..a2f000e3 100644 --- a/docs/guides/frameworks/megengine/megengine_mnist.md +++ b/docs/guides/frameworks/megengine/megengine_mnist.md @@ -2,7 +2,7 @@ title: MegEngine --- -The [megengine_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/megengine/megengine_mnist.py) +The [megengine_mnist.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/megengine/megengine_mnist.py) example demonstrates the integration of ClearML into code that uses [MegEngine](https://github.com/MegEngine/MegEngine) and [TensorBoardX](https://github.com/lanpa/tensorboardX). ClearML automatically captures models saved with `megengine`. diff --git a/docs/guides/frameworks/pytorch/model_updating.md b/docs/guides/frameworks/pytorch/model_updating.md index 6d9471da..4822eed8 100644 --- a/docs/guides/frameworks/pytorch/model_updating.md +++ b/docs/guides/frameworks/pytorch/model_updating.md @@ -2,7 +2,7 @@ title: PyTorch Model Updating --- -The [pytorch_model_update.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_model_update.py) +The [pytorch_model_update.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_model_update.py) example demonstrates training a model and logging it using the [OutputModel](../../../references/sdk/model_outputmodel.md) class. diff --git a/docs/guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K.md b/docs/guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K.md index bff461f2..396cb0c2 100644 --- a/docs/guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K.md +++ b/docs/guides/frameworks/pytorch/notebooks/audio/audio_classification_UrbanSound8K.md @@ -2,7 +2,7 @@ title: Audio Classification - Jupyter Notebooks --- -The [audio_classification_UrbanSound8K.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_classifier_UrbanSound8K.ipynb) example script demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the UrbanSound8K dataset for audio classification. The example calls TensorBoard methods in training and testing to report scalars, audio debug samples, and spectrogram visualizations. The spectrogram visualizations are plotted by calling Matplotlib methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates a task named `audio classification UrbanSound8K` in the `Audio Example` project. +The [audio_classification_UrbanSound8K.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_classifier_UrbanSound8K.ipynb) example script demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the UrbanSound8K dataset for audio classification. The example calls TensorBoard methods in training and testing to report scalars, audio debug samples, and spectrogram visualizations. The spectrogram visualizations are plotted by calling Matplotlib methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates a task named `audio classification UrbanSound8K` in the `Audio Example` project. ## Scalars diff --git a/docs/guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.md b/docs/guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.md index c6447cfd..7c0661d9 100644 --- a/docs/guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.md +++ b/docs/guides/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.md @@ -2,7 +2,7 @@ title: Audio Preprocessing - Jupyter Notebook --- -The example [audio_preprocessing_example.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.ipynb) +The example [audio_preprocessing_example.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/audio/audio_preprocessing_example.ipynb) demonstrates integrating ClearML into a Jupyter Notebook which uses PyTorch and preprocesses audio samples. ClearML automatically logs spectrogram visualizations reported by calling Matplotlib methods, and audio samples reported by calling TensorBoard methods. The example also demonstrates connecting parameters to a Task and logging them. When the script runs, it creates a task named `data pre-processing` in the `Audio Example` project. ## Plots diff --git a/docs/guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md b/docs/guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md index 6a106a9c..b93ddd94 100644 --- a/docs/guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md +++ b/docs/guides/frameworks/pytorch/notebooks/image/hyperparameter_search.md @@ -2,7 +2,7 @@ title: Image Hyperparameter Optimization - Jupyter Notebook --- -[hyperparameter_search.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/hyperparameter_search.ipynb) +[hyperparameter_search.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/hyperparameter_search.ipynb) demonstrates using ClearML's [HyperParameterOptimizer](../../../../../references/sdk/hpo_optimization_hyperparameteroptimizer.md) class to perform automated hyperparameter optimization (HPO). @@ -12,7 +12,7 @@ The example maximizes total accuracy by finding an optimal number of epochs, bat automatically logs the optimization's top performing tasks. The task whose hyperparameters are optimized is named `image_classification_CIFAR10`. It is created by running another -ClearML example, [image_classification_CIFAR10.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.ipynb), +ClearML example, [image_classification_CIFAR10.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.ipynb), which must run before `hyperparameter_search.ipynb`. The optimizer Task, `Hyperparameter Optimization`, and the tasks appear individually in the [ClearML Web UI](../../../../../webapp/webapp_overview.md). diff --git a/docs/guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.md b/docs/guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.md index 0e2c87c4..fd8f0d82 100644 --- a/docs/guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.md +++ b/docs/guides/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.md @@ -2,7 +2,7 @@ title: Image Classification - Jupyter Notebook --- -The example [image_classification_CIFAR10.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.ipynb) +The example [image_classification_CIFAR10.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/image/image_classification_CIFAR10.ipynb) demonstrates integrating ClearML into a Jupyter Notebook, which uses PyTorch, TensorBoard, and TorchVision to train a neural network on the CIFAR10 dataset for image classification. ClearML automatically logs the example script's calls to TensorBoard methods in training and testing which report scalars and image debug samples, as well as the model diff --git a/docs/guides/frameworks/pytorch/notebooks/table/download_and_preprocessing.md b/docs/guides/frameworks/pytorch/notebooks/table/download_and_preprocessing.md index 21a9676e..94fd9f6d 100644 --- a/docs/guides/frameworks/pytorch/notebooks/table/download_and_preprocessing.md +++ b/docs/guides/frameworks/pytorch/notebooks/table/download_and_preprocessing.md @@ -2,9 +2,9 @@ title: Tabular Data Downloading and Preprocessing - Jupyter Notebook --- -The [download_and_preprocessing.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_preprocessing.ipynb) example demonstrates ClearML storing preprocessed tabular data as artifacts, and explicitly reporting the tabular data in the **ClearML Web UI**. When the script runs, it creates a task named `tabular preprocessing` in the `Table Example` project. +The [download_and_preprocessing.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_preprocessing.ipynb) example demonstrates ClearML storing preprocessed tabular data as artifacts, and explicitly reporting the tabular data in the **ClearML Web UI**. When the script runs, it creates a task named `tabular preprocessing` in the `Table Example` project. -This tabular data is prepared for another script, [train_tabular_predictor.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb), which trains a network with it. +This tabular data is prepared for another script, [train_tabular_predictor.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb), which trains a network with it. ## Artifacts diff --git a/docs/guides/frameworks/pytorch/notebooks/table/tabular_training_pipeline.md b/docs/guides/frameworks/pytorch/notebooks/table/tabular_training_pipeline.md index 936fe304..e1b58f7d 100644 --- a/docs/guides/frameworks/pytorch/notebooks/table/tabular_training_pipeline.md +++ b/docs/guides/frameworks/pytorch/notebooks/table/tabular_training_pipeline.md @@ -9,17 +9,17 @@ class. The pipeline uses four Tasks (each Task is created using a different notebook): -* The pipeline controller Task ([tabular_ml_pipeline.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/tabular_ml_pipeline.ipynb)) -* A data preprocessing Task ([preprocessing_and_encoding.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/preprocessing_and_encoding.ipynb)) -* A training Task ([train_tabular_predictor.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb)) -* A better model comparison Task ([pick_best_model.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/pick_best_model.ipynb)) +* The pipeline controller Task ([tabular_ml_pipeline.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/tabular_ml_pipeline.ipynb)) +* A data preprocessing Task ([preprocessing_and_encoding.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/preprocessing_and_encoding.ipynb)) +* A training Task ([train_tabular_predictor.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb)) +* A better model comparison Task ([pick_best_model.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/pick_best_model.ipynb)) The `PipelineController` class includes functionality to create a pipeline controller, add steps to the pipeline, pass data from one step to another, control the dependencies of a step beginning only after other steps complete, run the pipeline, wait for it to complete, and cleanup afterwards. In this pipeline example, the data preprocessing Task and training Task are each added to the pipeline twice (each is in two steps). When the pipeline runs, the data preprocessing Task and training Task are cloned twice, and the newly cloned Tasks execute. The Task they are cloned from, called the base Task, does not execute. The pipeline controller passes different data to each cloned Task by overriding parameters. In this way, the same Task can run more than once in the pipeline, but with different data. :::note Download Data -The data download Task is not a step in the pipeline, see [download_and_split](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_split.ipynb). +The data download Task is not a step in the pipeline, see [download_and_split](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_split.ipynb). ::: ## Pipeline Controller and Steps @@ -246,17 +246,17 @@ By hovering over a step or path between nodes, you can view information about it **To run the pipeline:** -1. Download the data by running the notebook [download_and_split.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_split.ipynb). +1. Download the data by running the notebook [download_and_split.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/download_and_split.ipynb). 1. Run the script for each of the steps, if the script has not run once before. - * [preprocessing_and_encoding.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/preprocessing_and_encoding.ipynb) - * [train_tabular_predictor.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb) - * [pick_best_model.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/pick_best_model.ipynb). + * [preprocessing_and_encoding.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/preprocessing_and_encoding.ipynb) + * [train_tabular_predictor.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/train_tabular_predictor.ipynb) + * [pick_best_model.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/pick_best_model.ipynb). 1. Run the pipeline controller one of the following two ways: - * Run the notebook [tabular_ml_pipeline.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/tabular_ml_pipeline.ipynb). + * Run the notebook [tabular_ml_pipeline.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/table/tabular_ml_pipeline.ipynb). * Remotely execute the Task - If the Task `tabular training pipeline` which is associated with the project `Tabular Example` already exists in ClearML Server, clone it and enqueue it to execute. diff --git a/docs/guides/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.md b/docs/guides/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.md index ea75df2e..692f1b27 100644 --- a/docs/guides/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.md +++ b/docs/guides/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.md @@ -2,7 +2,7 @@ title: Text Classification - Jupyter Notebook --- -The example [text_classification_AG_NEWS.ipynb](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.ipynb) +The example [text_classification_AG_NEWS.ipynb](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/notebooks/text/text_classification_AG_NEWS.ipynb) demonstrates using Jupyter Notebook for ClearML, and the integration of ClearML into code which trains a network to classify text in the `torchtext` [AG_NEWS](https://pytorch.org/text/stable/datasets.html#ag-news) dataset, and then applies the model to predict the classification of sample text. diff --git a/docs/guides/frameworks/pytorch/pytorch_abseil.md b/docs/guides/frameworks/pytorch/pytorch_abseil.md index 2ff8f0e6..da1a4cf7 100644 --- a/docs/guides/frameworks/pytorch/pytorch_abseil.md +++ b/docs/guides/frameworks/pytorch/pytorch_abseil.md @@ -2,7 +2,7 @@ title: PyTorch Abseil --- -The [pytorch_abseil.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_abseil.py) +The [pytorch_abseil.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_abseil.py) example demonstrates the integration of ClearML into code that uses PyTorch and [`absl.flags`](https://abseil.io/docs/python/guides/flags). The example script does the following: diff --git a/docs/guides/frameworks/pytorch/pytorch_distributed_example.md b/docs/guides/frameworks/pytorch/pytorch_distributed_example.md index b73ae9ca..31f63da4 100644 --- a/docs/guides/frameworks/pytorch/pytorch_distributed_example.md +++ b/docs/guides/frameworks/pytorch/pytorch_distributed_example.md @@ -2,7 +2,7 @@ title: PyTorch Distributed --- -The [pytorch_distributed_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_distributed_example.py) +The [pytorch_distributed_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_distributed_example.py) script demonstrates integrating ClearML into a code that uses the [PyTorch Distributed Communications Package](https://pytorch.org/docs/stable/distributed.html) (`torch.distributed`). diff --git a/docs/guides/frameworks/pytorch/pytorch_matplotlib.md b/docs/guides/frameworks/pytorch/pytorch_matplotlib.md index e6444f86..afc20955 100644 --- a/docs/guides/frameworks/pytorch/pytorch_matplotlib.md +++ b/docs/guides/frameworks/pytorch/pytorch_matplotlib.md @@ -2,7 +2,7 @@ title: PyTorch with Matplotlib --- -The [pytorch_matplotlib.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_matplotlib.py) +The [pytorch_matplotlib.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_matplotlib.py) example demonstrates the integration of ClearML into code that uses PyTorch and Matplotlib. The example does the following: diff --git a/docs/guides/frameworks/pytorch/pytorch_mnist.md b/docs/guides/frameworks/pytorch/pytorch_mnist.md index d0ffb6ef..ec46c3e6 100644 --- a/docs/guides/frameworks/pytorch/pytorch_mnist.md +++ b/docs/guides/frameworks/pytorch/pytorch_mnist.md @@ -2,7 +2,7 @@ title: PyTorch MNIST --- -The [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) example +The [pytorch_mnist.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) example demonstrates the integration of ClearML into code that uses PyTorch. The example script does the following: diff --git a/docs/guides/frameworks/pytorch/pytorch_tensorboard.md b/docs/guides/frameworks/pytorch/pytorch_tensorboard.md index 451ffc8c..2dec7c79 100644 --- a/docs/guides/frameworks/pytorch/pytorch_tensorboard.md +++ b/docs/guides/frameworks/pytorch/pytorch_tensorboard.md @@ -2,7 +2,7 @@ title: PyTorch with TensorBoard --- -The [pytorch_tensorboard.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_tensorboard.py) +The [pytorch_tensorboard.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_tensorboard.py) example demonstrates the integration of ClearML into code that uses PyTorch and TensorBoard. The example does the following: diff --git a/docs/guides/frameworks/pytorch/pytorch_tensorboardx.md b/docs/guides/frameworks/pytorch/pytorch_tensorboardx.md index 25704917..6bc91d63 100644 --- a/docs/guides/frameworks/pytorch/pytorch_tensorboardx.md +++ b/docs/guides/frameworks/pytorch/pytorch_tensorboardx.md @@ -2,7 +2,7 @@ title: PyTorch TensorBoardX --- -The [pytorch_tensorboardX.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorboardx/pytorch_tensorboardX.py) +The [pytorch_tensorboardX.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorboardx/pytorch_tensorboardX.py) example demonstrates the integration of ClearML into code that uses PyTorch and TensorBoardX. The example does the following: diff --git a/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md b/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md index caf376b3..dcde66a8 100644 --- a/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md +++ b/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md @@ -2,7 +2,7 @@ title: PyTorch TensorBoard Toy --- -The [tensorboard_toy_pytorch.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/tensorboard_toy_pytorch.py) +The [tensorboard_toy_pytorch.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/tensorboard_toy_pytorch.py) example demonstrates the integration of ClearML into code, which creates a TensorBoard `SummaryWriter` object to log debug sample images. When the script runs, it creates a task named `pytorch tensorboard toy example`, which is associated with the `examples` project. diff --git a/docs/guides/frameworks/pytorch_ignite/integration_pytorch_ignite.md b/docs/guides/frameworks/pytorch_ignite/integration_pytorch_ignite.md index 95d1f498..3e56e835 100644 --- a/docs/guides/frameworks/pytorch_ignite/integration_pytorch_ignite.md +++ b/docs/guides/frameworks/pytorch_ignite/integration_pytorch_ignite.md @@ -2,7 +2,7 @@ title: PyTorch Ignite TensorboardLogger --- -The [cifar_ignite.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py) example +The [cifar_ignite.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py) example script integrates ClearML into code that uses [PyTorch Ignite](https://github.com/pytorch/ignite). The example script does the following: diff --git a/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md b/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md index 6b1e4b75..951ffa62 100644 --- a/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md +++ b/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md @@ -2,7 +2,7 @@ title: PyTorch Lightning --- -The [pytorch-lightning](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch-lightning/pytorch_lightning_example.py) +The [pytorch-lightning](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch-lightning/pytorch_lightning_example.py) script demonstrates the integration of ClearML into code that uses [PyTorch Lightning](https://www.pytorchlightning.ai/). The example script does the following: diff --git a/docs/guides/frameworks/scikit-learn/sklearn_joblib_example.md b/docs/guides/frameworks/scikit-learn/sklearn_joblib_example.md index e27dac1a..5d6e7a19 100644 --- a/docs/guides/frameworks/scikit-learn/sklearn_joblib_example.md +++ b/docs/guides/frameworks/scikit-learn/sklearn_joblib_example.md @@ -2,7 +2,7 @@ title: scikit-learn with Joblib --- -The [sklearn_joblib_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py) +The [sklearn_joblib_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py) demonstrates the integration of ClearML into code that uses `scikit-learn` and `joblib` to store a model and model snapshots, and `matplotlib` to create a scatter diagram. When the script runs, it creates a task named `scikit-learn joblib example` in the `examples` project. diff --git a/docs/guides/frameworks/scikit-learn/sklearn_matplotlib_example.md b/docs/guides/frameworks/scikit-learn/sklearn_matplotlib_example.md index 173b4fc5..aec6536b 100644 --- a/docs/guides/frameworks/scikit-learn/sklearn_matplotlib_example.md +++ b/docs/guides/frameworks/scikit-learn/sklearn_matplotlib_example.md @@ -2,7 +2,7 @@ title: scikit-learn with Matplotlib --- -The [sklearn_matplotlib_example.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_matplotlib_example.py) +The [sklearn_matplotlib_example.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_matplotlib_example.py) script demonstrates the integration of ClearML into code that uses `scikit-learn` and `matplotlib`. The example does the following: diff --git a/docs/guides/frameworks/tensorboardx/tensorboardx.md b/docs/guides/frameworks/tensorboardx/tensorboardx.md index 0566ff0c..57dae9de 100644 --- a/docs/guides/frameworks/tensorboardx/tensorboardx.md +++ b/docs/guides/frameworks/tensorboardx/tensorboardx.md @@ -2,7 +2,7 @@ title: TensorBoardX with PyTorch --- -The [pytorch_tensorboardX.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorboardx/pytorch_tensorboardX.py) +The [pytorch_tensorboardX.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorboardx/pytorch_tensorboardX.py) example demonstrates the integration of ClearML into code that uses PyTorch and TensorBoardX. The script does the following: diff --git a/docs/guides/frameworks/tensorboardx/video_tensorboardx.md b/docs/guides/frameworks/tensorboardx/video_tensorboardx.md index 8c89f734..258e1eb1 100644 --- a/docs/guides/frameworks/tensorboardx/video_tensorboardx.md +++ b/docs/guides/frameworks/tensorboardx/video_tensorboardx.md @@ -2,7 +2,7 @@ title: TensorBoardX Video --- -The [moveiepy_tensorboardx.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorboardx/moviepy_tensorboardx.py) +The [moveiepy_tensorboardx.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorboardx/moviepy_tensorboardx.py) example demonstrates the integration of ClearML into code, which creates a TensorBoardX `SummaryWriter` object to log video data. diff --git a/docs/guides/frameworks/tensorflow/integration_keras_tuner.md b/docs/guides/frameworks/tensorflow/integration_keras_tuner.md index 2168abd0..4635afd9 100644 --- a/docs/guides/frameworks/tensorflow/integration_keras_tuner.md +++ b/docs/guides/frameworks/tensorflow/integration_keras_tuner.md @@ -8,12 +8,12 @@ instructions. ::: Integrate ClearML into code that uses [Keras Tuner](https://www.tensorflow.org/tutorials/keras/keras_tuner). By -specifying `ClearMLTunerLogger` (see [kerastuner.py](https://github.com/allegroai/clearml/blob/master/clearml/external/kerastuner.py)) +specifying `ClearMLTunerLogger` (see [kerastuner.py](https://github.com/clearml/clearml/blob/master/clearml/external/kerastuner.py)) as the Keras Tuner logger, ClearML automatically logs scalars and hyperparameter optimization. ## ClearMLTunerLogger -Take a look at [keras_tuner_cifar.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/kerastuner/keras_tuner_cifar.py) +Take a look at [keras_tuner_cifar.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/kerastuner/keras_tuner_cifar.py) example script, which demonstrates the integration of ClearML in a code that uses Keras Tuner. The script does the following: diff --git a/docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md b/docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md index 35852cd8..3a62f229 100644 --- a/docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md +++ b/docs/guides/frameworks/tensorflow/tensorboard_pr_curve.md @@ -2,7 +2,7 @@ title: TensorBoard PR Curve --- -The [tensorboard_pr_curve.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_pr_curve.py) +The [tensorboard_pr_curve.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_pr_curve.py) example demonstrates the integration of ClearML into code that uses TensorFlow and TensorBoard. The example script does the following: diff --git a/docs/guides/frameworks/tensorflow/tensorboard_toy.md b/docs/guides/frameworks/tensorflow/tensorboard_toy.md index 96fdbaf2..d112a4b9 100644 --- a/docs/guides/frameworks/tensorflow/tensorboard_toy.md +++ b/docs/guides/frameworks/tensorflow/tensorboard_toy.md @@ -2,7 +2,7 @@ title: TensorBoard Toy --- -The [tensorboard_toy.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_toy.py) +The [tensorboard_toy.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_toy.py) example demonstrates ClearML's automatic logging of TensorBoard scalars, histograms, images, and text, as well as all other console output and TensorFlow Definitions. diff --git a/docs/guides/frameworks/tensorflow/tensorflow_mnist.md b/docs/guides/frameworks/tensorflow/tensorflow_mnist.md index 2fe86f64..263e7da6 100644 --- a/docs/guides/frameworks/tensorflow/tensorflow_mnist.md +++ b/docs/guides/frameworks/tensorflow/tensorflow_mnist.md @@ -2,7 +2,7 @@ title: TensorFlow MNIST --- -The [tensorflow_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py) +The [tensorflow_mnist.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py) example demonstrates the integration of ClearML into code that uses TensorFlow and Keras to train a neural network on the Keras built-in [MNIST](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/mnist) handwritten digits dataset. diff --git a/docs/guides/frameworks/xgboost/xgboost_metrics.md b/docs/guides/frameworks/xgboost/xgboost_metrics.md index a613e7a5..93886191 100644 --- a/docs/guides/frameworks/xgboost/xgboost_metrics.md +++ b/docs/guides/frameworks/xgboost/xgboost_metrics.md @@ -2,7 +2,7 @@ title: XGBoost Metrics --- -The [xgboost_metrics.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/xgboost/xgboost_metrics.py) +The [xgboost_metrics.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/xgboost/xgboost_metrics.py) example demonstrates the integration of ClearML into code that uses XGBoost to train a network on the scikit-learn [iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html#sklearn.datasets.load_iris) classification dataset. ClearML automatically captures models and scalars logged with XGBoost. diff --git a/docs/guides/frameworks/xgboost/xgboost_sample.md b/docs/guides/frameworks/xgboost/xgboost_sample.md index b1fa8c50..cc861cca 100644 --- a/docs/guides/frameworks/xgboost/xgboost_sample.md +++ b/docs/guides/frameworks/xgboost/xgboost_sample.md @@ -2,7 +2,7 @@ title: XGBoost and scikit-learn --- -The [xgboost_sample.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/xgboost/xgboost_sample.py) +The [xgboost_sample.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/xgboost/xgboost_sample.py) example demonstrates integrating ClearML into code that uses [XGBoost](https://xgboost.readthedocs.io/en/stable/). The example does the following: diff --git a/docs/guides/ide/google_colab.md b/docs/guides/ide/google_colab.md index ad2f96cb..49163696 100644 --- a/docs/guides/ide/google_colab.md +++ b/docs/guides/ide/google_colab.md @@ -20,7 +20,7 @@ and running, users can send Tasks to be executed on Google Colab's hardware. 1. Run the first cell, which installs all the necessary packages: ``` - !pip install git+https://github.com/allegroai/clearml + !pip install git+https://github.com/clearml/clearml !pip install clearml-agent ``` 1. Run the second cell, which exports this environment variable: diff --git a/docs/guides/ide/integration_pycharm.md b/docs/guides/ide/integration_pycharm.md index d8ebde0f..20fd7c15 100644 --- a/docs/guides/ide/integration_pycharm.md +++ b/docs/guides/ide/integration_pycharm.md @@ -17,7 +17,7 @@ private credentials (assuming the entire code base, including `.git` already exi ## Installation -1. Download the latest plugin version from the [Releases page](https://github.com/allegroai/clearml-pycharm-plugin/releases). +1. Download the latest plugin version from the [Releases page](https://github.com/clearml/clearml-pycharm-plugin/releases). 1. Install the plugin in PyCharm from local disk: diff --git a/docs/guides/ide/remote_jupyter_tutorial.md b/docs/guides/ide/remote_jupyter_tutorial.md index 69d464cb..fcf519cb 100644 --- a/docs/guides/ide/remote_jupyter_tutorial.md +++ b/docs/guides/ide/remote_jupyter_tutorial.md @@ -93,13 +93,13 @@ Now, let's execute some code in the remote session! 1. Open up a new Notebook. -1. In the first cell of the notebook, clone the [ClearML repository](https://github.com/allegroai/clearml): +1. In the first cell of the notebook, clone the [ClearML repository](https://github.com/clearml/clearml): ``` - !git clone https://github.com/allegroai/clearml.git + !git clone https://github.com/clearml/clearml.git ``` -1. In the second cell of the notebook, run this [script](https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) +1. In the second cell of the notebook, run this [script](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py) from the cloned repository: ``` diff --git a/docs/guides/main.md b/docs/guides/main.md index def8fe29..89143186 100644 --- a/docs/guides/main.md +++ b/docs/guides/main.md @@ -7,7 +7,7 @@ slug: /guides To help learn and use ClearML, we provide example scripts that demonstrate how to use ClearML's various features. -Examples scripts are in the [examples](https://github.com/allegroai/clearml/tree/master/examples) folder of the GitHub `clearml` +Examples scripts are in the [examples](https://github.com/clearml/clearml/tree/master/examples) folder of the GitHub `clearml` repository. They are also preloaded in the **ClearML Server**. Each examples folder in the GitHub ``clearml`` repository contains a ``requirements.txt`` file for example scripts in that folder. diff --git a/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md b/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md index cb788f24..2b385640 100644 --- a/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md +++ b/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt.md @@ -2,7 +2,7 @@ title: Hyperparameter Optimization --- -The [hyper_parameter_optimizer.py](https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py) +The [hyper_parameter_optimizer.py](https://github.com/clearml/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py) example script demonstrates hyperparameter optimization (HPO), which is automated by using ClearML. ## Set the Search Strategy for Optimization @@ -24,7 +24,7 @@ The following search strategies can be used: * Random uniform sampling of hyperparameter strategy - [`automation.RandomSearch`](../../../references/sdk/hpo_optimization_randomsearch.md) * Full grid sampling strategy of every hyperparameter combination - [`automation.GridSearch`](../../../references/sdk/hpo_optimization_gridsearch.md). -* Custom - Use a custom class and inherit from the ClearML automation base strategy class, [`SearchStrategy`](https://github.com/allegroai/clearml/blob/master/clearml/automation/optimization.py#L310) +* Custom - Use a custom class and inherit from the ClearML automation base strategy class, [`SearchStrategy`](https://github.com/clearml/clearml/blob/master/clearml/automation/optimization.py#L310) The search strategy class that is chosen will be passed to the [`automation.HyperParameterOptimizer`](../../../references/sdk/hpo_optimization_hyperparameteroptimizer.md) object later. diff --git a/docs/guides/pipeline/pipeline_controller.md b/docs/guides/pipeline/pipeline_controller.md index 24e8b36f..2cddb681 100644 --- a/docs/guides/pipeline/pipeline_controller.md +++ b/docs/guides/pipeline/pipeline_controller.md @@ -2,20 +2,20 @@ title: Pipeline from Tasks --- -The [pipeline_from_tasks.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_tasks.py) +The [pipeline_from_tasks.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/pipeline_from_tasks.py) example demonstrates a simple pipeline, where each step is a [ClearML Task](../../fundamentals/task.md). The pipeline is implemented using the [PipelineController](../../references/sdk/automation_controller_pipelinecontroller.md) class. Steps are added to a PipelineController object, which launches and monitors the steps when executed. This example incorporates four tasks, each of which is created using a different script: -* **Controller Task** ([pipeline_from_tasks.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_tasks.py)) - +* **Controller Task** ([pipeline_from_tasks.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/pipeline_from_tasks.py)) - Implements the pipeline controller, adds the steps (tasks) to the pipeline, and runs the pipeline. -* **Step 1** ([step1_dataset_artifact.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/step1_dataset_artifact.py)) - +* **Step 1** ([step1_dataset_artifact.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/step1_dataset_artifact.py)) - Downloads data and stores the data as an artifact. -* **Step 2** ([step2_data_processing.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/step2_data_processing.py)) - +* **Step 2** ([step2_data_processing.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/step2_data_processing.py)) - Loads the stored data (from Step 1), processes it, and stores the processed data as artifacts. -* **Step 3** ([step3_train_model.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/step3_train_model.py)) - +* **Step 3** ([step3_train_model.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/step3_train_model.py)) - Loads the processed data (from Step 2) and trains a network. When the controller task is executed, it clones the step tasks, and enqueues the newly cloned tasks for execution. Note @@ -100,7 +100,7 @@ The sections below describe in more detail what happens in the controller task a ## Step 1 - Downloading the Data -The pipeline's first step ([step1_dataset_artifact.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/step1_dataset_artifact.py)) +The pipeline's first step ([step1_dataset_artifact.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/step1_dataset_artifact.py)) does the following: 1. Download data using [`StorageManager.get_local_copy()`](../../references/sdk/storage.md#storagemanagerget_local_copy): @@ -108,7 +108,7 @@ does the following: ```python # simulate local dataset, download one, so we have something local local_iris_pkl = StorageManager.get_local_copy( - remote_url='https://github.com/allegroai/events/raw/master/odsc20-east/generic/iris_dataset.pkl' + remote_url='https://github.com/clearml/events/raw/master/odsc20-east/generic/iris_dataset.pkl' ) ``` 1. Store the data as an artifact named `dataset` using [`Task.upload_artifact()`](../../references/sdk/task.md#upload_artifact): @@ -119,7 +119,7 @@ does the following: ## Step 2 - Processing the Data -The pipeline's second step ([step2_data_processing.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/step2_data_processing.py)) +The pipeline's second step ([step2_data_processing.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/step2_data_processing.py)) does the following: 1. Connect its configuration parameters with the ClearML task: @@ -154,7 +154,7 @@ does the following: ## Step 3 - Training the Network -The pipeline's third step ([step3_train_model.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/step3_train_model.py)) +The pipeline's third step ([step3_train_model.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/step3_train_model.py)) does the following: 1. Connect its configuration parameters with the ClearML task. This allows the [pipeline controller](#the-pipeline-controller) to override the `dataset_task_id` value as the pipeline is run. diff --git a/docs/guides/pipeline/pipeline_decorator.md b/docs/guides/pipeline/pipeline_decorator.md index f8f55ef6..4c5a3213 100644 --- a/docs/guides/pipeline/pipeline_decorator.md +++ b/docs/guides/pipeline/pipeline_decorator.md @@ -2,7 +2,7 @@ title: Pipeline from Decorators --- -The [pipeline_from_decorator.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py) +The [pipeline_from_decorator.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py) example demonstrates the creation of a pipeline in ClearML using the [`PipelineDecorator`](../../references/sdk/automation_controller_pipelinecontroller.md#class-automationcontrollerpipelinedecorator) class. @@ -58,7 +58,7 @@ PipelineDecorator.set_default_execution_queue('default') # PipelineDecorator.debug_pipeline() executing_pipeline( - pickle_url='https://github.com/allegroai/events/raw/master/odsc20-east/generic/iris_dataset.pkl', + pickle_url='https://github.com/clearml/events/raw/master/odsc20-east/generic/iris_dataset.pkl', ) ``` diff --git a/docs/guides/pipeline/pipeline_functions.md b/docs/guides/pipeline/pipeline_functions.md index 2dcf17ad..0ac60a55 100644 --- a/docs/guides/pipeline/pipeline_functions.md +++ b/docs/guides/pipeline/pipeline_functions.md @@ -2,7 +2,7 @@ title: Pipeline from Functions --- -The [pipeline_from_functions.py](https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_functions.py) +The [pipeline_from_functions.py](https://github.com/clearml/clearml/blob/master/examples/pipeline/pipeline_from_functions.py) example script demonstrates the creation of a pipeline using the [PipelineController](../../references/sdk/automation_controller_pipelinecontroller.md) class. @@ -45,7 +45,7 @@ the function will be automatically logged as required packages for the pipeline pipe.add_parameter( name='url', description='url to pickle file', - default='https://github.com/allegroai/events/raw/master/odsc20-east/generic/iris_dataset.pkl' + default='https://github.com/clearml/events/raw/master/odsc20-east/generic/iris_dataset.pkl' ) ``` diff --git a/docs/guides/reporting/3d_plots_reporting.md b/docs/guides/reporting/3d_plots_reporting.md index 7f286257..080c6fdf 100644 --- a/docs/guides/reporting/3d_plots_reporting.md +++ b/docs/guides/reporting/3d_plots_reporting.md @@ -2,7 +2,7 @@ title: 3D Plots Reporting --- -The [3d_plots_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/3d_plots_reporting.py) +The [3d_plots_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/3d_plots_reporting.py) example demonstrates reporting a series as a surface plot and as a 3D scatter plot. When the script runs, it creates a task named `3D plot reporting` in the `examples` project. diff --git a/docs/guides/reporting/artifacts.md b/docs/guides/reporting/artifacts.md index 0ac20f0b..8c764169 100644 --- a/docs/guides/reporting/artifacts.md +++ b/docs/guides/reporting/artifacts.md @@ -2,7 +2,7 @@ title: Artifacts Reporting --- -The [artifacts.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/artifacts.py) example demonstrates +The [artifacts.py](https://github.com/clearml/clearml/blob/master/examples/reporting/artifacts.py) example demonstrates uploading objects (other than models) to storage as task artifacts. These artifacts include: diff --git a/docs/guides/reporting/clearml_logging_example.md b/docs/guides/reporting/clearml_logging_example.md index 1931e2ac..760152b5 100644 --- a/docs/guides/reporting/clearml_logging_example.md +++ b/docs/guides/reporting/clearml_logging_example.md @@ -2,7 +2,7 @@ title: Using Logger - Jupyter Notebook --- -The [jupyter_logging_example.ipynb](https://github.com/allegroai/clearml/blob/master/examples/reporting/jupyter_logging_example.ipynb) +The [jupyter_logging_example.ipynb](https://github.com/clearml/clearml/blob/master/examples/reporting/jupyter_logging_example.ipynb) script demonstrates the integration of ClearML's explicit reporting module, `Logger`, in a Jupyter Notebook. All ClearML explicit reporting works with Jupyter Notebook. diff --git a/docs/guides/reporting/explicit_reporting.md b/docs/guides/reporting/explicit_reporting.md index 714dd6f2..dcf3ea53 100644 --- a/docs/guides/reporting/explicit_reporting.md +++ b/docs/guides/reporting/explicit_reporting.md @@ -4,7 +4,7 @@ title: Explicit Reporting Tutorial In this tutorial, learn how to extend ClearML automagical capturing of inputs and outputs with explicit reporting. -In this example, you will add the following to the [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) +In this example, you will add the following to the [pytorch_mnist.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) example script from ClearML's GitHub repo: * Setting an output destination for model checkpoints (snapshots). @@ -14,12 +14,12 @@ example script from ClearML's GitHub repo: ## Prerequisites -* The [clearml](https://github.com/allegroai/clearml) repository is cloned. +* The [clearml](https://github.com/clearml/clearml) repository is cloned. * The `clearml` package is installed. ## Before Starting -Make a copy of [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) +Make a copy of [pytorch_mnist.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) to add explicit reporting to it. ```bash diff --git a/docs/guides/reporting/html_reporting.md b/docs/guides/reporting/html_reporting.md index 9ead5c27..1842e941 100644 --- a/docs/guides/reporting/html_reporting.md +++ b/docs/guides/reporting/html_reporting.md @@ -2,7 +2,7 @@ title: HTML Reporting --- -The [html_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py) example +The [html_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/html_reporting.py) example demonstrates reporting local HTML files and HTML by URL using [`Logger.report_media()`](../../references/sdk/logger.md#report_media). ClearML reports these HTML debug samples in the **ClearML Web UI** **>** task's **DEBUG SAMPLES** tab. @@ -15,7 +15,7 @@ When the script runs, it creates a task named `html samples reporting` in the `e Report HTML by URL using [`Logger.report_media()`](../../references/sdk/logger.md#report_media)'s `url` parameter. -See the example script's [`report_html_url`](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L16) +See the example script's [`report_html_url`](https://github.com/clearml/clearml/blob/master/examples/reporting/html_reporting.py#L16) function, which reports the ClearML documentation's home page. ```python @@ -37,7 +37,7 @@ Report the following using `Logger.report_media()`'s `local_path` parameter: ### Interactive HTML -See the example script's [`report_html_periodic_table`](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L26) function, which reports a file created from Bokeh sample data. +See the example script's [`report_html_periodic_table`](https://github.com/clearml/clearml/blob/master/examples/reporting/html_reporting.py#L26) function, which reports a file created from Bokeh sample data. ```python Logger.current_logger().report_media( title="html", @@ -49,7 +49,7 @@ Logger.current_logger().report_media( ### Bokeh GroupBy HTML -See the example script's [`report_html_groupby`](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L117) function, which reports a Pandas GroupBy with nested HTML, created from Bokeh sample data. +See the example script's [`report_html_groupby`](https://github.com/clearml/clearml/blob/master/examples/reporting/html_reporting.py#L117) function, which reports a Pandas GroupBy with nested HTML, created from Bokeh sample data. ```python Logger.current_logger().report_media( title="html", @@ -61,7 +61,7 @@ Logger.current_logger().report_media( ### Bokeh Graph HTML -See the example script's [`report_html_graph`](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L162) function, which reports a Bokeh plot created from Bokeh sample data. +See the example script's [`report_html_graph`](https://github.com/clearml/clearml/blob/master/examples/reporting/html_reporting.py#L162) function, which reports a Bokeh plot created from Bokeh sample data. ```python Logger.current_logger().report_media( @@ -74,7 +74,7 @@ Logger.current_logger().report_media( ### Bokeh Image HTML -See the example script's [`report_html_image`](https://github.com/allegroai/clearml/blob/master/examples/reporting/html_reporting.py#L195) function, which reports an image created from Bokeh sample data. +See the example script's [`report_html_image`](https://github.com/clearml/clearml/blob/master/examples/reporting/html_reporting.py#L195) function, which reports an image created from Bokeh sample data. ```python Logger.current_logger().report_media( diff --git a/docs/guides/reporting/hyper_parameters.md b/docs/guides/reporting/hyper_parameters.md index d890167b..db6fdae6 100644 --- a/docs/guides/reporting/hyper_parameters.md +++ b/docs/guides/reporting/hyper_parameters.md @@ -2,7 +2,7 @@ title: Hyperparameters Reporting --- -The [hyper_parameters.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/hyper_parameters.py) example +The [hyper_parameters.py](https://github.com/clearml/clearml/blob/master/examples/reporting/hyper_parameters.py) example script demonstrates: * ClearML's automatic logging of `argparse` command line options and TensorFlow Definitions * Logging user-defined hyperparameters with a parameter dictionary and connecting the dictionary to a Task. diff --git a/docs/guides/reporting/image_reporting.md b/docs/guides/reporting/image_reporting.md index 1450ddf9..eb61136d 100644 --- a/docs/guides/reporting/image_reporting.md +++ b/docs/guides/reporting/image_reporting.md @@ -2,7 +2,7 @@ title: Image Reporting --- -The [image_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/image_reporting.py) example +The [image_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/image_reporting.py) example demonstrates reporting (uploading) images in several formats, including: * NumPy arrays * uint8 diff --git a/docs/guides/reporting/manual_matplotlib_reporting.md b/docs/guides/reporting/manual_matplotlib_reporting.md index a84e9a8c..3e0ab33f 100644 --- a/docs/guides/reporting/manual_matplotlib_reporting.md +++ b/docs/guides/reporting/manual_matplotlib_reporting.md @@ -2,7 +2,7 @@ title: Manual Matplotlib Reporting --- -The [matplotlib_manual_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/matplotlib_manual_reporting.py) +The [matplotlib_manual_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/matplotlib_manual_reporting.py) example demonstrates using ClearML to log plots and images generated by Matplotlib and Seaborn. ## Plots diff --git a/docs/guides/reporting/media_reporting.md b/docs/guides/reporting/media_reporting.md index 8cbf50ee..6e5b79ec 100644 --- a/docs/guides/reporting/media_reporting.md +++ b/docs/guides/reporting/media_reporting.md @@ -2,7 +2,7 @@ title: Media Reporting --- -The [media_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/media_reporting.py) example +The [media_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/media_reporting.py) example demonstrates reporting (uploading) images, audio, and video. Use [`Logger.report_media()`](../../references/sdk/logger.md#report_media) to upload from: * Local path diff --git a/docs/guides/reporting/model_config.md b/docs/guides/reporting/model_config.md index 7f21d696..2fada138 100644 --- a/docs/guides/reporting/model_config.md +++ b/docs/guides/reporting/model_config.md @@ -2,7 +2,7 @@ title: Model Reporting --- -The [model_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/model_reporting.py) example +The [model_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/model_reporting.py) example demonstrates logging a model using the [OutputModel](../../references/sdk/model_outputmodel.md) class. diff --git a/docs/guides/reporting/pandas_reporting.md b/docs/guides/reporting/pandas_reporting.md index c13e2371..2e1edbfd 100644 --- a/docs/guides/reporting/pandas_reporting.md +++ b/docs/guides/reporting/pandas_reporting.md @@ -2,7 +2,7 @@ title: Tables Reporting (Pandas and CSV Files) --- -The [pandas_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/pandas_reporting.py) example demonstrates reporting tabular data from Pandas DataFrames and CSV files as tables. +The [pandas_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/pandas_reporting.py) example demonstrates reporting tabular data from Pandas DataFrames and CSV files as tables. ClearML reports these tables, and displays them in the **ClearML Web UI** **>** task's **PLOTS** tab. diff --git a/docs/guides/reporting/plotly_reporting.md b/docs/guides/reporting/plotly_reporting.md index 7926c0c5..79954808 100644 --- a/docs/guides/reporting/plotly_reporting.md +++ b/docs/guides/reporting/plotly_reporting.md @@ -2,7 +2,7 @@ title: Plotly Reporting --- -The [plotly_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/plotly_reporting.py) example +The [plotly_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/plotly_reporting.py) example demonstrates ClearML's Plotly integration and reporting. Report Plotly plots in ClearML by calling the [`Logger.report_plotly()`](../../references/sdk/logger.md#report_plotly) method, and passing a complex diff --git a/docs/guides/reporting/scalar_reporting.md b/docs/guides/reporting/scalar_reporting.md index e1873f6d..bee2d8cf 100644 --- a/docs/guides/reporting/scalar_reporting.md +++ b/docs/guides/reporting/scalar_reporting.md @@ -2,7 +2,7 @@ title: Scalars Reporting --- -The [scalar_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/scalar_reporting.py) script +The [scalar_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/scalar_reporting.py) script demonstrates explicit scalar reporting. ClearML reports scalars in the **ClearML Web UI** **>** task's **SCALARS** tab. When the script runs, it creates a task named `scalar reporting` in the `examples` project. diff --git a/docs/guides/reporting/scatter_hist_confusion_mat_reporting.md b/docs/guides/reporting/scatter_hist_confusion_mat_reporting.md index 15bead0e..410b281b 100644 --- a/docs/guides/reporting/scatter_hist_confusion_mat_reporting.md +++ b/docs/guides/reporting/scatter_hist_confusion_mat_reporting.md @@ -2,7 +2,7 @@ title: 2D Plots Reporting --- -The [scatter_hist_confusion_mat_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/scatter_hist_confusion_mat_reporting.py) +The [scatter_hist_confusion_mat_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/scatter_hist_confusion_mat_reporting.py) example demonstrates reporting series data in the following 2D formats: * [Histograms](#histograms) * [Confusion matrices](#confusion-matrices) diff --git a/docs/guides/reporting/text_reporting.md b/docs/guides/reporting/text_reporting.md index 05c61770..1b7eba72 100644 --- a/docs/guides/reporting/text_reporting.md +++ b/docs/guides/reporting/text_reporting.md @@ -2,7 +2,7 @@ title: Text Reporting --- -The [text_reporting.py](https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py) script +The [text_reporting.py](https://github.com/clearml/clearml/blob/master/examples/reporting/text_reporting.py) script demonstrates reporting explicit text by calling [`Logger.report_text()`](../../references/sdk/logger.md#report_text). ClearML reports the text in the **ClearML Web UI**, in the task's **CONSOLE** tab. diff --git a/docs/guides/reporting/using_artifacts.md b/docs/guides/reporting/using_artifacts.md index b57c2360..f470d971 100644 --- a/docs/guides/reporting/using_artifacts.md +++ b/docs/guides/reporting/using_artifacts.md @@ -2,7 +2,7 @@ title: Using Artifacts --- -The [using_artifacts_example](https://github.com/allegroai/clearml/blob/master/examples/reporting/using_artifacts_example.py) +The [using_artifacts_example](https://github.com/clearml/clearml/blob/master/examples/reporting/using_artifacts_example.py) script demonstrates uploading a data file to a task as an artifact and then accessing and utilizing the artifact in a different task. When the script runs it creates two tasks, `create artifact` and `use artifact from other task`, both of which are associated diff --git a/docs/guides/services/aws_autoscaler.md b/docs/guides/services/aws_autoscaler.md index 512cf01c..c761b49b 100644 --- a/docs/guides/services/aws_autoscaler.md +++ b/docs/guides/services/aws_autoscaler.md @@ -2,8 +2,8 @@ title: ClearML AWS Autoscaler Service --- -The ClearML [AWS autoscaler example](https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py) -demonstrates how to use the [`clearml.automation.auto_scaler`](https://github.com/allegroai/clearml/blob/master/clearml/automation/auto_scaler.py) +The ClearML [AWS autoscaler example](https://github.com/clearml/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py) +demonstrates how to use the [`clearml.automation.auto_scaler`](https://github.com/clearml/clearml/blob/master/clearml/automation/auto_scaler.py) module to implement a service that optimizes AWS EC2 instance scaling according to a defined instance budget. The autoscaler periodically polls your AWS cluster and automatically stops idle instances based on a defined maximum idle time or spins @@ -12,7 +12,7 @@ up new instances when there aren't enough to execute pending tasks. ## Running the ClearML AWS Autoscaler Run the ClearML AWS autoscaler in one of these ways: -* Run the [aws_autoscaler.py](https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py) +* Run the [aws_autoscaler.py](https://github.com/clearml/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py) script locally * Launch through your [`services` queue](../../clearml_agent/clearml_agent_services_mode.md) diff --git a/docs/guides/services/cleanup_service.md b/docs/guides/services/cleanup_service.md index 700bca59..fbf75776 100644 --- a/docs/guides/services/cleanup_service.md +++ b/docs/guides/services/cleanup_service.md @@ -2,7 +2,7 @@ title: Cleanup Service --- -The [cleanup service](https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py) +The [cleanup service](https://github.com/clearml/clearml/blob/master/examples/services/cleanup/cleanup_service.py) demonstrates how to use the `clearml.backend_api.session.client.APIClient` class to implement a service that deletes old archived tasks and their associated files: model checkpoints, other artifacts, and debug samples. @@ -44,7 +44,7 @@ This is followed by details from the cleanup. ## The Cleanup Service Code -[cleanup_service.py](https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py) creates +[cleanup_service.py](https://github.com/clearml/clearml/blob/master/examples/services/cleanup/cleanup_service.py) creates an `APIClient` object that establishes a session with the ClearML Server, and accomplishes the cleanup by calling: * [`Tasks.get_all`](../../references/api/tasks.md#post-tasksget_all) to get a list of Tasks to delete, providing the following parameters: * `system_tags` - Get only Tasks tagged as `archived`. diff --git a/docs/guides/services/slack_alerts.md b/docs/guides/services/slack_alerts.md index 7c65a554..e7bcb251 100644 --- a/docs/guides/services/slack_alerts.md +++ b/docs/guides/services/slack_alerts.md @@ -2,7 +2,7 @@ title: Monitoring Service Posting Slack Alerts --- -The [Slack alerts example](https://github.com/allegroai/clearml/blob/master/examples/services/monitoring/slack_alerts.py) +The [Slack alerts example](https://github.com/clearml/clearml/blob/master/examples/services/monitoring/slack_alerts.py) demonstrates how to use the `clearml.automation.monitor` class to implement a service that monitors the completion and failure of tasks, and posts alert messages on a Slack channel. diff --git a/docs/guides/ui/building_leader_board.md b/docs/guides/ui/building_leader_board.md index 120fcc46..1d8cffe5 100644 --- a/docs/guides/ui/building_leader_board.md +++ b/docs/guides/ui/building_leader_board.md @@ -18,7 +18,7 @@ Bookmark the leaderboard in the browser to reuse and share the leaderboard URL w For this tutorial, use one of the following as a project: * Recommended: One of your projects with more than one task. -* Clone the [clearml](https://github.com/allegroai/clearml) repository and execute some of the example scripts. +* Clone the [clearml](https://github.com/clearml/clearml) repository and execute some of the example scripts. ## Step 1: Select a Project diff --git a/docs/guides/ui/tuning_exp.md b/docs/guides/ui/tuning_exp.md index 1b3954d6..c94d95de 100644 --- a/docs/guides/ui/tuning_exp.md +++ b/docs/guides/ui/tuning_exp.md @@ -2,13 +2,13 @@ title: Tuning Tasks --- -In this tutorial, learn how to tune a task. The task that will be tuned is created by the [pytorch_mnist.py](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) +In this tutorial, learn how to tune a task. The task that will be tuned is created by the [pytorch_mnist.py](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py) example script. ## Prerequisites -* Clone the [clearml](https://github.com/allegroai/clearml) repository. -* Install the [requirements](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/requirements.txt) +* Clone the [clearml](https://github.com/clearml/clearml) repository. +* Install the [requirements](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/requirements.txt) for the TensorFlow examples. * Have **ClearML Agent** [installed and configured](../../clearml_agent/clearml_agent_setup.md#installation). diff --git a/docs/hyperdatasets/code_examples.md b/docs/hyperdatasets/code_examples.md index 232edd03..5af4a163 100644 --- a/docs/hyperdatasets/code_examples.md +++ b/docs/hyperdatasets/code_examples.md @@ -6,9 +6,9 @@ The following examples demonstrate registering, retrieving, and ingesting your d interface. ## Registering your Data -* [register_dataset_with_roi.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-registration/register_dataset_with_roi.py) - Demonstrates +* [register_dataset_with_roi.py](https://github.com/clearml/clearml/blob/master/examples/hyperdatasets/data-registration/register_dataset_with_roi.py) - Demonstrates creating a new DatasetVersion and adding to it frames, supporting ROI annotations and metadata -* [register_dataset_masks.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-registration/register_dataset_masks.py) - Demonstrates +* [register_dataset_masks.py](https://github.com/clearml/clearml/blob/master/examples/hyperdatasets/data-registration/register_dataset_masks.py) - Demonstrates creating a new DatasetVersion and adding to it frames containing masks. This example also demonstrates the DatasetVersion-level [pixel segmentation masks](masks.md#pixel-segmentation-masks). @@ -16,8 +16,8 @@ After executing either of these scripts, you can view your DatasetVersion conten ## Using your Data ### Dataviews -The [dataview_example_framegroup.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-ingestion/dataview_example_framegroup.py) -and [dataview_example_singleframe.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-ingestion/dataview_example_singleframe.py) +The [dataview_example_framegroup.py](https://github.com/clearml/clearml/blob/master/examples/hyperdatasets/data-ingestion/dataview_example_framegroup.py) +and [dataview_example_singleframe.py](https://github.com/clearml/clearml/blob/master/examples/hyperdatasets/data-ingestion/dataview_example_singleframe.py) examples demonstrate how to use a [DataView](dataviews.md) to retrieve your data as SingleFrames and FrameGroups as part of a running task. This is done by creating a DataView query and then retrieving the corresponding frames. @@ -25,9 +25,9 @@ DataView details are displayed in the UI in a task's **DATAVIEWS** tab. ### Data Ingestion -The [pytorch_dataset_example.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-ingestion/pytorch_dataset_example.py) +The [pytorch_dataset_example.py](https://github.com/clearml/clearml/blob/master/examples/hyperdatasets/data-ingestion/pytorch_dataset_example.py) example demonstrates how to feed your DataViews to an ML framework by creating a DataView query and wrapping it as a PyTorch Dataset. -The [pytorch_dataset_example_with_masks.py](https://github.com/allegroai/clearml/blob/master/examples/hyperdatasets/data-ingestion/pytorch_dataset_example_with_masks.py) +The [pytorch_dataset_example_with_masks.py](https://github.com/clearml/clearml/blob/master/examples/hyperdatasets/data-ingestion/pytorch_dataset_example_with_masks.py) example demonstrates the additional actions required when your frames contain masks. \ No newline at end of file diff --git a/docs/integrations/click.md b/docs/integrations/click.md index c105e264..cf9298bd 100644 --- a/docs/integrations/click.md +++ b/docs/integrations/click.md @@ -47,5 +47,5 @@ When this code is executed, ClearML logs your command-line arguments, which you In the UI, you can clone the task multiple times and set the clones' parameter values for execution by the [ClearML Agent](../clearml_agent.md). When the clone is executed, the executing agent will use the new parameter values as if set by the command-line. -See [code examples](https://github.com/allegroai/clearml/blob/master/examples/frameworks/click) demonstrating integrating +See [code examples](https://github.com/clearml/clearml/blob/master/examples/frameworks/click) demonstrating integrating ClearML with code that uses `click`. \ No newline at end of file diff --git a/docs/integrations/hydra.md b/docs/integrations/hydra.md index 66bb7cb5..d8a05c04 100644 --- a/docs/integrations/hydra.md +++ b/docs/integrations/hydra.md @@ -67,4 +67,4 @@ according to your added parameters. ## Code Example -See example which demonstrates integrating ClearML into script that uses Hydra [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/hydra/hydra_example.py). +See example which demonstrates integrating ClearML into script that uses Hydra [here](https://github.com/clearml/clearml/blob/master/examples/frameworks/hydra/hydra_example.py). diff --git a/docs/integrations/ignite.md b/docs/integrations/ignite.md index 098bcc7b..9b2de832 100644 --- a/docs/integrations/ignite.md +++ b/docs/integrations/ignite.md @@ -29,7 +29,7 @@ uncommitted code, Python environment, all information logged through `Tensorboar Visualize all the captured information in the task's page in ClearML's [WebApp](#webapp). -See a code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py). +See a code example [here](https://github.com/clearml/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py). ## ClearMLLogger PyTorch Ignite supports a ClearML Logger to log metrics, text, model/optimizer parameters, plots, and model checkpoints diff --git a/docs/integrations/jsonargparse.md b/docs/integrations/jsonargparse.md index 779e32fb..8f348e45 100644 --- a/docs/integrations/jsonargparse.md +++ b/docs/integrations/jsonargparse.md @@ -64,5 +64,5 @@ task is enqueued, the task will use the new values during execution. ## Code Examples -See [code examples](https://github.com/allegroai/clearml/blob/master/examples/frameworks/jsonargparse) demonstrating integrating +See [code examples](https://github.com/clearml/clearml/blob/master/examples/frameworks/jsonargparse) demonstrating integrating ClearML with code that uses `jsonargparse`. diff --git a/docs/integrations/mmcv.md b/docs/integrations/mmcv.md index 8702f27e..8c77ca70 100644 --- a/docs/integrations/mmcv.md +++ b/docs/integrations/mmcv.md @@ -50,4 +50,4 @@ You can view the captured metrics in the task's **Scalars** tab in the [WebApp]( ![OpenMMLab scalars](../img/integration_openmmlab_scalars.png) -See MMCV code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/openmmlab/openmmlab_cifar10.py). \ No newline at end of file +See MMCV code example [here](https://github.com/clearml/clearml/blob/master/examples/frameworks/openmmlab/openmmlab_cifar10.py). \ No newline at end of file diff --git a/docs/integrations/python_fire.md b/docs/integrations/python_fire.md index 1167014a..88c30733 100644 --- a/docs/integrations/python_fire.md +++ b/docs/integrations/python_fire.md @@ -21,7 +21,7 @@ When the code runs, ClearML logs your command-line arguments, which you can view In the UI, you can clone the task multiple times and set the clones' parameter values for execution by the [ClearML Agent](../clearml_agent.md). When the clone is executed, the executing agent will use the new parameter values as if set by the command-line. -See [code examples](https://github.com/allegroai/clearml/blob/master/examples/frameworks/fire) demonstrating integrating +See [code examples](https://github.com/clearml/clearml/blob/master/examples/frameworks/fire) demonstrating integrating ClearML with code that uses `fire`. diff --git a/docs/integrations/seaborn.md b/docs/integrations/seaborn.md index c4861d2b..ca2e1a2c 100644 --- a/docs/integrations/seaborn.md +++ b/docs/integrations/seaborn.md @@ -23,4 +23,4 @@ in the task's **Plots** tab. ![Seaborn plot](../img/integrations_seaborn_plots.png) -View code example [here](https://github.com/allegroai/clearml/blob/master/examples/frameworks/matplotlib/matplotlib_example.py). +View code example [here](https://github.com/clearml/clearml/blob/master/examples/frameworks/matplotlib/matplotlib_example.py). diff --git a/docs/integrations/tensorflow.md b/docs/integrations/tensorflow.md index 72aa4a9b..3bdaee58 100644 --- a/docs/integrations/tensorflow.md +++ b/docs/integrations/tensorflow.md @@ -85,7 +85,7 @@ model checkpoints, TensorFlow definitions, and scalars logged using TensorFlow m logging of TensorBoard output and TensorFlow definitions. * [TensorBoard Toy](../guides/frameworks/tensorflow/tensorboard_toy.md) - Demonstrates ClearML's automatic logging of TensorBoard scalars, histograms, images, and text, as well as all console output and TensorFlow Definitions. -* [Absl flags](https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/absl_flags.py) - Demonstrates +* [Absl flags](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/absl_flags.py) - Demonstrates ClearML's automatic logging of parameters defined using `absl.flags` ## Remote Execution diff --git a/docs/pipelines/pipelines_sdk_tasks.md b/docs/pipelines/pipelines_sdk_tasks.md index 7f28668f..36eefe5e 100644 --- a/docs/pipelines/pipelines_sdk_tasks.md +++ b/docs/pipelines/pipelines_sdk_tasks.md @@ -29,7 +29,7 @@ You can define parameters for controlling different pipeline runs: pipe.add_parameter( name='pickle_url', description='url to pickle file', - default='https://github.com/allegroai/events/raw/master/odsc20-east/generic/iris_dataset.pkl' + default='https://github.com/clearml/events/raw/master/odsc20-east/generic/iris_dataset.pkl' ) ``` diff --git a/docs/release_notes/clearml_agent/ver_0_12.md b/docs/release_notes/clearml_agent/ver_0_12.md index 1e0f5399..5b2f1d51 100644 --- a/docs/release_notes/clearml_agent/ver_0_12.md +++ b/docs/release_notes/clearml_agent/ver_0_12.md @@ -22,8 +22,8 @@ title: Version 0.12 * Windows support! YES, you can now have Windows machines as part of your cluster (notice --docker is not supported on Windows). * Add initial Conda package manager support (still in beta). * Add --gpus and --cpu-only for easier GPU control when running multiple `trains-agent` instances on the same machine. -* [python_binary](https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf#L35) can now be specified (the default is the same Python binary executing the `trains-agent`). -* Fix Issue ([GitHub Issue #2](https://github.com/allegroai/trains-agent/issues/2)). +* [python_binary](https://github.com/clearml/clearml-agent/blob/master/docs/trains.conf#L35) can now be specified (the default is the same Python binary executing the `trains-agent`). +* Fix Issue ([GitHub Issue #2](https://github.com/clearml/clearml-agent/issues/2)). ### Trains Agent 0.12.0 @@ -36,4 +36,4 @@ TRAINS Agent - Initial Release * Support for standalone scripts and Jupyter Notebooks. * TRAINS-server v0.12 and above. * TRAINS v0.11.3 and above. -* Sample configuration file available [here](https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf). +* Sample configuration file available [here](https://github.com/clearml/clearml-agent/blob/master/docs/trains.conf). diff --git a/docs/release_notes/clearml_agent/ver_0_14.md b/docs/release_notes/clearml_agent/ver_0_14.md index bf2c4c7c..6d85214b 100644 --- a/docs/release_notes/clearml_agent/ver_0_14.md +++ b/docs/release_notes/clearml_agent/ver_0_14.md @@ -23,10 +23,10 @@ title: Version 0.14 **Features and Bug Fixes** * Add support for `trains-agent execute --id --docker` that allows executing a specific experiment inside a docker container. * Add support for `trains-agent execute --id --clone` that clones the provided experiment and executes the cloned experiment. -* Add support for `APIClient.models.delete` to allow programmatically deleting a model ([GitHub trains-server Issue #32](https://github.com/allegroai/trains-server/issues/32)). +* Add support for `APIClient.models.delete` to allow programmatically deleting a model ([GitHub trains-server Issue #32](https://github.com/clearml/clearml-server/issues/32)). * Add daemon support for passing storage-related OS environment variables to experiments executed inside a Docker container (supported by trains>=0.13.3): * AWS: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_DEFAULT_REGION` * Azure: `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_KEY` * Google: `GOOGLE_APPLICATION_CREDENTIALS` -* Fix Git checkout with submodules ([GitHub Issue #112](https://github.com/allegroai/trains/issues/112)). +* Fix Git checkout with submodules ([GitHub Issue #112](https://github.com/clearml/clearml/issues/112)). * Docker image in command line takes precedence over the Docker image specified in an experiment. \ No newline at end of file diff --git a/docs/release_notes/clearml_agent/ver_0_15.md b/docs/release_notes/clearml_agent/ver_0_15.md index 87c6fc5d..60848b44 100644 --- a/docs/release_notes/clearml_agent/ver_0_15.md +++ b/docs/release_notes/clearml_agent/ver_0_15.md @@ -15,9 +15,9 @@ title: Version 0.15 **Bug Fixes** * Fix initialization wizard (allow at most two verification retries, then print error). -* Add warning on `--gpus` with no detected CUDA version ([GitHub trains-agent Issue 24](https://github.com/allegroai/trains-agent/issues/24)). -* Add `agent.force_git_ssh_protocol` configuration option to force all Git links to `ssh://` ([GitHub trains-agent Issue 16](https://github.com/allegroai/trains-agent/issues/16)). -* Add Git user / pass permission into pip package installation from Git repository ([GitHub trains-agent Issue 22](https://github.com/allegroai/trains-agent/issues/22)). +* Add warning on `--gpus` with no detected CUDA version ([GitHub trains-agent Issue 24](https://github.com/clearml/clearml-agent/issues/24)). +* Add `agent.force_git_ssh_protocol` configuration option to force all Git links to `ssh://` ([GitHub trains-agent Issue 16](https://github.com/clearml/clearml-agent/issues/16)). +* Add Git user / pass permission into pip package installation from Git repository ([GitHub trains-agent Issue 22](https://github.com/clearml/clearml-agent/issues/22)). ### Trains Agent 0.15.0 @@ -37,7 +37,7 @@ title: Version 0.15 **Bug Fixes** -* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/allegroai/trains-agent/issues/19)). +* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/clearml/clearml-agent/issues/19)). * Improve docker host-mount support, use `TRAINS_AGENT_DOCKER_HOST_MOUNT` environment variable. * Support pip v20.1 local/http package reference in pip freeze. * Fix detached mode to correctly use cache folder slots. diff --git a/docs/release_notes/clearml_agent/ver_0_16.md b/docs/release_notes/clearml_agent/ver_0_16.md index 05108d22..c02e1890 100644 --- a/docs/release_notes/clearml_agent/ver_0_16.md +++ b/docs/release_notes/clearml_agent/ver_0_16.md @@ -23,7 +23,7 @@ title: Version 0.16 - Allow selecting external `trains.conf` file for the pod itself. - Allow providing pod template, extra bash init script, alternate SSH server port, gateway address (k8s ingress / ELB). -- Allow specifying `cudatoolkit` version in the "installed packages" section when using conda as package manager (GitHub trains Issue 229). +- Allow specifying `cudatoolkit` version in the "installed packages" section when using conda as package manager (GitHub trains Issue 229). - Add `agent.package_manager.force_repo_requirements_txt`. If True, "Installed Packages" on Task are ignored, and only repository `requirements.txt` is used. - Pass `TRAINS_DOCKER_IMAGE` into docker for interactive sessions. - Add `torchcsprng` and `torchtext` to PyTorch resolving. @@ -64,7 +64,7 @@ title: Version 0.16 **Bug Fixes** * Fix pre-installed packages are ignored when installing a git package wheel. Reinstalling a `git+http` link is enough - to make sure all requirements are met / installed ([GitHub Issue #196](https://github.com/allegroai/trains/issues/196)). + to make sure all requirements are met / installed ([GitHub Issue #196](https://github.com/clearml/clearml/issues/196)). * Fix incorrect check for spaces in current execution folder. * Fix requirements detection: * Update torch version after using downloaded / system pre-installed version. @@ -84,7 +84,7 @@ title: Version 0.16 **Bug Fixes** -* Fix GPU Windows monitoring support ([GitHub Issue #177](https://github.com/allegroai/trains/issues/177)). +* Fix GPU Windows monitoring support ([GitHub Issue #177](https://github.com/clearml/clearml/issues/177)). * Fix `.git-credentials` and `.gitconfig` mapping into docker. * Fix non-root docker image usage. * Fix docker to use `UTF-8` encoding, so prints won't break it. diff --git a/docs/release_notes/clearml_agent/ver_0_17.md b/docs/release_notes/clearml_agent/ver_0_17.md index 13e59170..329ca47f 100644 --- a/docs/release_notes/clearml_agent/ver_0_17.md +++ b/docs/release_notes/clearml_agent/ver_0_17.md @@ -9,18 +9,18 @@ title: Version 0.17 - Add virtual environment caching: * Supports venv caching both in standard and docker mode. * Configurable using the `agent.venvs_cache` configuration section. - * Disabled by default, enable [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L101). + * Disabled by default, enable [here](https://github.com/clearml/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L101). - Add support for `--services-mode` with venvs. -- Add `agent.force_git_ssh_user` configuration value (default `git`, see [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L29)) - ([clearml-agent GitHub Issue 42](https://github.com/allegroai/clearml-agent/issues/42)). +- Add `agent.force_git_ssh_user` configuration value (default `git`, see [here](https://github.com/clearml/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L29)) + ([clearml-agent GitHub Issue 42](https://github.com/clearml/clearml-agent/issues/42)). - Add `agent.ignore_requested_python_version` configuration option for multi Python environments (default `false`). - Add `agent.enable_task_env` configuration option to set the OS environment based on the Environment section of the Task (default `false`, see - [here](https://github.com/allegroai/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L151)). + [here](https://github.com/clearml/clearml-agent/blob/205f9dd81620fcec5aa155991afbbf711f3cd648/docs/clearml.conf#L151)). - K8s glue: * Add support for detecting and deleting k8s pods that fail to start. * Allow providing namespace in k8s glue and k8s glue example. * Add base-pod-number parameter to k8s glue and example. -- Change `agent.default_docker.image` to `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` (see [here](https://github.com/allegroai/clearml-agent/blob/8a46dc6b03860bc1cc022fd19e86bf4524986a0e/docs/clearml.conf#L144)). +- Change `agent.default_docker.image` to `nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04` (see [here](https://github.com/clearml/clearml-agent/blob/8a46dc6b03860bc1cc022fd19e86bf4524986a0e/docs/clearml.conf#L144)). - Use shared git cache for multiple agents on the same machine. - Upgrade pynvml add detect CUDA version from driver level. - Update agent and services docker files. @@ -43,4 +43,4 @@ title: Version 0.17 ### ClearML Agent 0.17.0 -* Renamed `trains-agent` as `clearml-agent` (see [clearml-agent](https://github.com/allegroai/clearml-agent) in GitHub). +* Renamed `trains-agent` as `clearml-agent` (see [clearml-agent](https://github.com/clearml/clearml-agent) in GitHub). diff --git a/docs/release_notes/clearml_agent/ver_1_0.md b/docs/release_notes/clearml_agent/ver_1_0.md index c6702440..769ae8df 100644 --- a/docs/release_notes/clearml_agent/ver_1_0.md +++ b/docs/release_notes/clearml_agent/ver_1_0.md @@ -17,7 +17,7 @@ title: Version 1.0 * Verify `docker` command exists when running in docker mode * Add support for terminating dockers on `sig_term` in dynamic mode * Add stopping message on Task process termination -* Add `agent.docker_install_opencv_libs` configuration option to enable automatic opencv libs install for faster docker spin-up (default: `true`, see [here](https://github.com/allegroai/clearml-agent/blob/4f18bb7ea0600db6ee63ecb5ad0ea8d048a272ca/docs/clearml.conf#L139)) +* Add `agent.docker_install_opencv_libs` configuration option to enable automatic opencv libs install for faster docker spin-up (default: `true`, see [here](https://github.com/clearml/clearml-agent/blob/4f18bb7ea0600db6ee63ecb5ad0ea8d048a272ca/docs/clearml.conf#L139)) * Add support for new container base setup script feature * Bump virtualenv dependency version (support `v>=16,<21`) * Add support for dynamic gpus opportunistic scheduling (with min/max gpus per queue) @@ -26,8 +26,8 @@ title: Version 1.0 **Bug Fixes** -* Fix agent can return non-zero error code and pods will end up restarting forever ([clearml-agent GitHub Issue 56](https://github.com/allegroai/clearml-agent/issues/56)) -* Fix poetry support ([clearml-agent GitHub Issue 57](https://github.com/allegroai/clearml-agent/issues/57)) +* Fix agent can return non-zero error code and pods will end up restarting forever ([clearml-agent GitHub Issue 56](https://github.com/clearml/clearml-agent/issues/56)) +* Fix poetry support ([clearml-agent GitHub Issue 57](https://github.com/clearml/clearml-agent/issues/57)) * Fix cuda version from driver does not return minor version * Fix requirements local path replace back when using cache * Fix k8s glue diff --git a/docs/release_notes/clearml_agent/ver_1_1.md b/docs/release_notes/clearml_agent/ver_1_1.md index 0d91ce9b..67c3a0c0 100644 --- a/docs/release_notes/clearml_agent/ver_1_1.md +++ b/docs/release_notes/clearml_agent/ver_1_1.md @@ -14,7 +14,7 @@ title: Version 1.1 - Add support for truncating task log file after reporting to server using `agent.truncate_task_output_files` configuration setting - Fix PyJWT resiliency support -- Fix `--stop` checking default queue tag ([ClearML Agent GitHub issue #80](https://github.com/allegroai/clearml-agent/issues/80)) +- Fix `--stop` checking default queue tag ([ClearML Agent GitHub issue #80](https://github.com/clearml/clearml-agent/issues/80)) - Fix queue tag `default` does not exist and `--queue` not specified (try queue named `"default"`) - Fix Python 3.5 compatibility - Fix PY2.7 support for PyTorch @@ -30,17 +30,17 @@ ClearML k8s glue default pod label was changed to `CLEARML=agent` (instead of `T **Features** -- Add poetry cache into docker mapping ([ClearML Agent GitHub issue #74](https://github.com/allegroai/clearml-agent/issues/74)) -- Allow rewriting SSH URLs (see [here](https://github.com/allegroai/clearml-agent/commit/9456e493ac6d6495310ee084db906f9cdca8218c)), refers to [ClearML Agent GitHub PR #72](https://github.com/allegroai/clearml-agent/pull/72) ([ClearML Agent GitHub issue #42](https://github.com/allegroai/clearml-agent/issues/42)) -- Add docker environment arguments log masking support, customizable using the `agent.hide_docker_command_env_vars` configuration value (see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L172)) ([ClearML Agent GitHub issue #67](https://github.com/allegroai/clearml-agent/issues/67)) -- Add support for naming docker containers using the `agent.docker_container_name_format` configuration option to set a name format (disabled by default) ([ClearML issue #412](https://github.com/allegroai/clearml/issues/412)) +- Add poetry cache into docker mapping ([ClearML Agent GitHub issue #74](https://github.com/clearml/clearml-agent/issues/74)) +- Allow rewriting SSH URLs (see [here](https://github.com/clearml/clearml-agent/commit/9456e493ac6d6495310ee084db906f9cdca8218c)), refers to [ClearML Agent GitHub PR #72](https://github.com/clearml/clearml-agent/pull/72) ([ClearML Agent GitHub issue #42](https://github.com/clearml/clearml-agent/issues/42)) +- Add docker environment arguments log masking support, customizable using the `agent.hide_docker_command_env_vars` configuration value (see [here](https://github.com/clearml/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L172)) ([ClearML Agent GitHub issue #67](https://github.com/clearml/clearml-agent/issues/67)) +- Add support for naming docker containers using the `agent.docker_container_name_format` configuration option to set a name format (disabled by default) ([ClearML issue #412](https://github.com/clearml/clearml/issues/412)) - k8s glue - - Remove queue name from pod name, add queue name and ID to pod labels ([ClearML Agent GitHub issue #64](https://github.com/allegroai/clearml-agent/issues/64)) + - Remove queue name from pod name, add queue name and ID to pod labels ([ClearML Agent GitHub issue #64](https://github.com/clearml/clearml-agent/issues/64)) - Update task `status_message` for non-responsive or hanging pods - Support the `agent.docker_force_pull` configuration option for scheduled pods - Add docker example for running the k8s glue as a pod in a k8s cluster -- Add `agent.ignore_requested_python_version` configuration option to ignore any requested Python version (default false, see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L45)) -- Add `agent.docker_internal_mounts` configuration option to control containers internal mounts (non-root containers, see [here](https://github.com/allegroai/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L184)) +- Add `agent.ignore_requested_python_version` configuration option to ignore any requested Python version (default false, see [here](https://github.com/clearml/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L45)) +- Add `agent.docker_internal_mounts` configuration option to control containers internal mounts (non-root containers, see [here](https://github.com/clearml/clearml-agent/blob/db57441c5dda43d8e38f01d7f52f047913e95ba5/docs/clearml.conf#L184)) - Add support for `-r requirements.txt` in the Installed Packages section - Add support for `CLEARML_AGENT_INITIAL_CONNECT_RETRY_OVERRIDE` environment variable to override initial server connection behavior (defaults to true, allows boolean value or an explicit number specifying the number of connect retries) - Add support for `CLEARML_AGENT_DISABLE_SSH_MOUNT` environment variable allowing to disable the auto `.ssh` mount into the docker @@ -54,9 +54,9 @@ ClearML k8s glue default pod label was changed to `CLEARML=agent` (instead of `T **Bug Fixes** -- Fix auto mount `SSH_AUTH_SOCK` into docker ([ClearML Agent GitHub issue #45](https://github.com/allegroai/clearml-agent/issues/45)) -- Fix package manager configuration documentation ([ClearML Agent GitHub issue #78](https://github.com/allegroai/clearml-agent/issues/78)) -- Fix support for spaces in docker arguments ([ClearML GitHub issue #358](https://github.com/allegroai/clearml/issues/358)) +- Fix auto mount `SSH_AUTH_SOCK` into docker ([ClearML Agent GitHub issue #45](https://github.com/clearml/clearml-agent/issues/45)) +- Fix package manager configuration documentation ([ClearML Agent GitHub issue #78](https://github.com/clearml/clearml-agent/issues/78)) +- Fix support for spaces in docker arguments ([ClearML GitHub issue #358](https://github.com/clearml/clearml/issues/358)) - Fix standalone script with pre-exiting conda venv - Fix PyYAML v5.4, v5.4.1 versions not supported - Fix parsing VCS links starting with `git+git@` (notice `git+git://` was already supported) diff --git a/docs/release_notes/clearml_agent/ver_1_2.md b/docs/release_notes/clearml_agent/ver_1_2.md index 3455f697..75de3d48 100644 --- a/docs/release_notes/clearml_agent/ver_1_2.md +++ b/docs/release_notes/clearml_agent/ver_1_2.md @@ -22,14 +22,14 @@ title: Version 1.2 **New Features and Improvements** -- Update S3 bucket verify option for minio ([ClearML Agent GitHub PR #83](https://github.com/allegroai/clearml-agent/pull/83)) -- Add environment variable for request method ([ClearML Agent GitHub PR #91](https://github.com/allegroai/clearml-agent/pull/91)) -- Add additional k8s-glue dockerfiles ([ClearML Agent GitHub PR #94](https://github.com/allegroai/clearml-agent/pull/94) ) +- Update S3 bucket verify option for minio ([ClearML Agent GitHub PR #83](https://github.com/clearml/clearml-agent/pull/83)) +- Add environment variable for request method ([ClearML Agent GitHub PR #91](https://github.com/clearml/clearml-agent/pull/91)) +- Add additional k8s-glue dockerfiles ([ClearML Agent GitHub PR #94](https://github.com/clearml/clearml-agent/pull/94) ) - Update default docker image to `nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04` -- Add support for custom docker image resolving using the `agent.default_docker.match_rules` configuration setting (see [here](https://github.com/allegroai/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L169)) +- Add support for custom docker image resolving using the `agent.default_docker.match_rules` configuration setting (see [here](https://github.com/clearml/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L169)) - Add `agent.force_git_root_python_path` configuration setting to force adding the git repository root folder to the `PYTHONPATH` (if set working directory is not added to the `PYHTONPATH`) - Add `build --force-docker` command line argument to the to allow ignoring task container data -- Add `agent.poetry_version` configuration setting to specify poetry version (and force installation of poetry if missing, see [here](https://github.com/allegroai/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L65)) +- Add `agent.poetry_version` configuration setting to specify poetry version (and force installation of poetry if missing, see [here](https://github.com/clearml/clearml-agent/blob/8712c5e636d9a02e939a9759348d29521a3939a9/docs/clearml.conf#L65)) - Add custom build script support - Add extra configurations when starting daemon - Add `agent.package_manager.force_original_requirements` configuration option, allowing to only use original requirements produced by local execution (note that using this configuration option prevents editing installed packages using the UI) @@ -38,10 +38,10 @@ title: Version 1.2 **Bug Fixes** -- Fix virtualenv Python interpreter used ([ClearML Agent GitHub PR #98](https://github.com/allegroai/clearml-agent/pull/98)) -- Fix typing package incorrectly required for Python>3.5 ([ClearML Agent GitHub PR #103](https://github.com/allegroai/clearml-agent/pull/103)) -- Fix symbolic links not copied from cached VCS into working copy (windows platform will result with default copy content instead of original symbolic link) ([ClearML Agent GitHub PR #89](https://github.com/allegroai/clearml-agent/pull/89)) -- Fix agent fails to check out code from main branch when branch/commit is not explicitly specified ([ClearML GitHub issue #551](https://github.com/allegroai/clearml/issues/551)) +- Fix virtualenv Python interpreter used ([ClearML Agent GitHub PR #98](https://github.com/clearml/clearml-agent/pull/98)) +- Fix typing package incorrectly required for Python>3.5 ([ClearML Agent GitHub PR #103](https://github.com/clearml/clearml-agent/pull/103)) +- Fix symbolic links not copied from cached VCS into working copy (windows platform will result with default copy content instead of original symbolic link) ([ClearML Agent GitHub PR #89](https://github.com/clearml/clearml-agent/pull/89)) +- Fix agent fails to check out code from main branch when branch/commit is not explicitly specified ([ClearML GitHub issue #551](https://github.com/clearml/clearml/issues/551)) - Fix `git+git://` requirements - Fix `default_python` calculation (and verbosity) - Fix using deprecated `abc` support (Python 3.10 compatibility) diff --git a/docs/release_notes/clearml_agent/ver_1_3.md b/docs/release_notes/clearml_agent/ver_1_3.md index c1de4e0d..a8b2abf1 100644 --- a/docs/release_notes/clearml_agent/ver_1_3.md +++ b/docs/release_notes/clearml_agent/ver_1_3.md @@ -4,7 +4,7 @@ title: Version 1.3 ### ClearML Agent 1.3.0 **New Features and Improvements** -* Support private repos from `requirements.txt` file ([ClearML Agent GitHub PR #107](https://github.com/allegroai/clearml-agent/pull/107)) +* Support private repos from `requirements.txt` file ([ClearML Agent GitHub PR #107](https://github.com/clearml/clearml-agent/pull/107)) * Bump PyJWT version due to "*Key confusion through non-blocklisted public key formats*" vulnerability * Add support for additional command line arguments in k8s glue example * Add Python 3.10 support diff --git a/docs/release_notes/clearml_agent/ver_1_4.md b/docs/release_notes/clearml_agent/ver_1_4.md index f02d7027..66444063 100644 --- a/docs/release_notes/clearml_agent/ver_1_4.md +++ b/docs/release_notes/clearml_agent/ver_1_4.md @@ -42,4 +42,4 @@ title: Version 1.4 * Fix template namespace should override default namespace * Fix `extra_bash_init_cmd` location in initial bash script * Fix debug mode -* Fixed documentation ([ClearML Agent PR #117](https://github.com/allegroai/clearml-agent/pull/117)) +* Fixed documentation ([ClearML Agent PR #117](https://github.com/clearml/clearml-agent/pull/117)) diff --git a/docs/release_notes/clearml_agent/ver_1_5.md b/docs/release_notes/clearml_agent/ver_1_5.md index 911e985e..a771a1d4 100644 --- a/docs/release_notes/clearml_agent/ver_1_5.md +++ b/docs/release_notes/clearml_agent/ver_1_5.md @@ -5,14 +5,14 @@ title: Version 1.5 ### ClearML Agent 1.5.2 **New Features and Improvements** -* Switch services agent entrypoint shell from sh to bash ([ClearML Agent GitHub issue #140](https://github.com/allegroai/clearml-agent/issues/140)) +* Switch services agent entrypoint shell from sh to bash ([ClearML Agent GitHub issue #140](https://github.com/clearml/clearml-agent/issues/140)) * Improve poetry support -* Add poetry cwd support ([ClearML Agent GitHub issue #138](https://github.com/allegroai/clearml-agent/issues/138)) +* Add poetry cwd support ([ClearML Agent GitHub issue #138](https://github.com/clearml/clearml-agent/issues/138)) * Add `agent.package_manager.poetry_install_extra_args` configuration option * Do not allow request exceptions (keep retrying, throw error only on the initial login call) **Bug Fixes** -* Fix agent update version ([ClearML Agent GitHub PR #132](https://github.com/allegroai/clearml-agent/pull/132)) +* Fix agent update version ([ClearML Agent GitHub PR #132](https://github.com/clearml/clearml-agent/pull/132)) * Fix login uses `GET` with payload, which breaks when trying to connect a server running in GCP * Fix `clearml-agent build --docker` stuck on certain containers * Fix build fails when target is relative path @@ -25,7 +25,7 @@ title: Version 1.5 ### ClearML Agent 1.5.1 **New Features and Improvements** -* Upgrade requirements for `attrs`, `jsonschema`, `pyparsing`, `six`, and `pyjwt` ([ClearML Agent GitHub issue #129](https://github.com/allegroai/clearml-agent/issues/129)) +* Upgrade requirements for `attrs`, `jsonschema`, `pyparsing`, `six`, and `pyjwt` ([ClearML Agent GitHub issue #129](https://github.com/clearml/clearml-agent/issues/129)) * Add default output URI selection to `clearml-agent init` * Add `agent.disable_task_docker_override` configuration option to disable docker override specified in executing tasks * Add `CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGES` env var (default `true`) to allow overriding default `system_site_packages: true` @@ -39,7 +39,7 @@ since 4.0.0) ### ClearML Agent 1.5.0 **New Features and Improvements** -* Add option to crash agent on exception using `agent.crash_on_exception` configuration setting ([ClearML Agent GitHub issue #122](https://github.com/allegroai/clearml-agent/issues/122)) +* Add option to crash agent on exception using `agent.crash_on_exception` configuration setting ([ClearML Agent GitHub issue #122](https://github.com/clearml/clearml-agent/issues/122)) * Improve venv cache disabled message * Upgrade packages for better Python 3.10 support * Remove future package dependency (Python 2 is not supported for `clearml-agent`) diff --git a/docs/release_notes/clearml_agent/ver_1_6.md b/docs/release_notes/clearml_agent/ver_1_6.md index e4854526..39b55014 100644 --- a/docs/release_notes/clearml_agent/ver_1_6.md +++ b/docs/release_notes/clearml_agent/ver_1_6.md @@ -10,10 +10,10 @@ title: Version 1.6 ### ClearML Agent 1.6.0 **New Features and Improvements** -* Upgrade `requests` library ([ClearML Agent GitHub issue #161](https://github.com/allegroai/clearml-agent/issues/161)) +* Upgrade `requests` library ([ClearML Agent GitHub issue #161](https://github.com/clearml/clearml-agent/issues/161)) * Add support for controlling PyTorch resolving mode using the `CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE` environment variable and `agent.package_manager.pytorch_resolve configuration` setting with `none` (no resolving), `pip` (sets extra -index based on cuda and lets pip resolve) or `direct` (the previous parsing algorithm that does the matching and downloading). Default is `pip` ([ClearML Agent GitHub issue #152](https://github.com/allegroai/clearml-agent/issues/152)) +index based on cuda and lets pip resolve) or `direct` (the previous parsing algorithm that does the matching and downloading). Default is `pip` ([ClearML Agent GitHub issue #152](https://github.com/clearml/clearml-agent/issues/152)) * Add backwards compatibility in standalone mode using the `CLEARML_AGENT_STANDALONE_CONFIG_BC` environment variable * Add `CLEARML_AGENT_DOCKER_AGENT_REPO` alias for the `FORCE_CLEARML_AGENT_REPO` environment variable * Show a better message for `agent init` when an existing `clearml.conf` is found diff --git a/docs/release_notes/clearml_agent/ver_1_7.md b/docs/release_notes/clearml_agent/ver_1_7.md index d93724eb..5ca1c5b5 100644 --- a/docs/release_notes/clearml_agent/ver_1_7.md +++ b/docs/release_notes/clearml_agent/ver_1_7.md @@ -22,7 +22,7 @@ log files (daemon and execution) * Set worker ID in k8s pod execution **Bug Fixes** -* Fix `agent.package_manager.poetry_install_extra_args` are used in all Poetry commands and not just in `install` ([ClearML Agent GitHub issue #173](https://github.com/allegroai/clearml-agent/issues/173)) +* Fix `agent.package_manager.poetry_install_extra_args` are used in all Poetry commands and not just in `install` ([ClearML Agent GitHub issue #173](https://github.com/clearml/clearml-agent/issues/173)) * Fix if process return code is SIGKILL (-9 or 137) and abort callback was called, do not mark as failed but as aborted * Fix `agent.git_host` setting will cause git@domain URLs to not be replaced by SSH URLs since furl cannot parse them to obtain host diff --git a/docs/release_notes/clearml_agent/ver_1_8.md b/docs/release_notes/clearml_agent/ver_1_8.md index a8603d90..158f0b06 100644 --- a/docs/release_notes/clearml_agent/ver_1_8.md +++ b/docs/release_notes/clearml_agent/ver_1_8.md @@ -5,7 +5,7 @@ title: Version 1.8 ### ClearML Agent 1.8.1 **New Features** -* Add option to set daemon polling interval ([ClearML Agent GitHub PR #197](https://github.com/allegroai/clearml-agent/pull/197)) +* Add option to set daemon polling interval ([ClearML Agent GitHub PR #197](https://github.com/clearml/clearml-agent/pull/197)) * Add Python 3.12 support **Bug Fixes** @@ -31,9 +31,9 @@ specified and the code is running a preinstalled docker * Improve GPU monitoring for MIGs **Bug Fixes** -* Use correct Python version in Poetry init ([ClearML Agent GitHub PR #179](https://github.com/allegroai/clearml-agent/pull/179)) -* Fix queue handling in `K8sIntegration` and `k8s_glue_example.py` ([ClearML Agent GitHub PR #183](https://github.com/allegroai/clearml-agent/pull/183)) -* Fix `FileNotFoundException` crash in `find_python_executable_for_version` ([ClearML Agent GitHub issue #164](https://github.com/allegroai/clearml-agent/issues/164)) +* Use correct Python version in Poetry init ([ClearML Agent GitHub PR #179](https://github.com/clearml/clearml-agent/pull/179)) +* Fix queue handling in `K8sIntegration` and `k8s_glue_example.py` ([ClearML Agent GitHub PR #183](https://github.com/clearml/clearml-agent/pull/183)) +* Fix `FileNotFoundException` crash in `find_python_executable_for_version` ([ClearML Agent GitHub issue #164](https://github.com/clearml/clearml-agent/issues/164)) * Fix delete temp console pipe log files after Task execution is completed (important for long-lasting services agents to avoid collecting temp files on host machine) * Fix `agent.enable_git_ask_pass` does not show in configuration dump * Fix `pip` is returned as a pip version if no value exists in `agent.package_manager.pip_version` diff --git a/docs/release_notes/clearml_agent/ver_1_9.md b/docs/release_notes/clearml_agent/ver_1_9.md index 68af729d..a4902a75 100644 --- a/docs/release_notes/clearml_agent/ver_1_9.md +++ b/docs/release_notes/clearml_agent/ver_1_9.md @@ -23,7 +23,7 @@ those matching these filters to be used when running containers ### ClearML Agent 1.9.2 **New Features and Bug Fixes** -* Handle OSError when checking for `is_file` ([ClearML GitHub issue #1337](https://github.com/allegroai/clearml/issues/1337)) +* Handle OSError when checking for `is_file` ([ClearML GitHub issue #1337](https://github.com/clearml/clearml/issues/1337)) * Add support for pip legacy resolver for versions specified using the `agent.package_manager.pip_legacy_resolver` configuration option * Add skip existing packages * Fix report index not advancing in resource monitoring causes more than one GPU not to be reported @@ -38,7 +38,7 @@ those matching these filters to be used when running containers ### ClearML Agent 1.9.0 **New Features and Improvements** -* Add `NO_DOCKER` flag to `clearml-agent-services` entrypoint ([ClearML Agent GitHub PR #206](https://github.com/allegroai/clearml-agent/pull/206)) +* Add `NO_DOCKER` flag to `clearml-agent-services` entrypoint ([ClearML Agent GitHub PR #206](https://github.com/clearml/clearml-agent/pull/206)) * Use `venv` module if `virtualenv` is not supported * Find the correct Python version when using a pre-installed python environment * Add `/bin/bash` support in the task's `script.binary` property @@ -56,7 +56,7 @@ by the `working_dir` setting in the `:` format (e.g. `:standal * Add support for skipping container apt installs using `CLEARML_AGENT_SKIP_CONTAINER_APT` env var in k8s **Bug Fixes** -* Fix git fetch did not update new tags ([ClearML Agent GitHub issue #209](https://github.com/allegroai/clearml-agent/issues/209)) +* Fix git fetch did not update new tags ([ClearML Agent GitHub issue #209](https://github.com/clearml/clearml-agent/issues/209)) * Fix file mode should be optional in configuration `files` section * Fix `-m module $env` to support parsing `$env` before launching * Fix setting tasks that were just marked as `aborted` to `started` - only force task to `started` after dequeueing it, diff --git a/docs/release_notes/clearml_server/enterprise/ver_3_23.md b/docs/release_notes/clearml_server/enterprise/ver_3_23.md index 96e9272b..c3e42865 100644 --- a/docs/release_notes/clearml_server/enterprise/ver_3_23.md +++ b/docs/release_notes/clearml_server/enterprise/ver_3_23.md @@ -49,16 +49,16 @@ user access to these objects or their containing project. * Add UI edit of default Project default output destination **Bug Fixes** -* Fix broken download links to artifacts stored in Azure ([ClearML Server GitHub issue #247](https://github.com/allegroai/clearml-server/issues/247)) +* Fix broken download links to artifacts stored in Azure ([ClearML Server GitHub issue #247](https://github.com/clearml/clearml-server/issues/247)) * Fix Cross-site Scripting (XSS) vulnerability (CWE-79) -* Fix UI experiment textual comparison diff showing contextual diffs as different contents ([ClearML GitHub issue #646](https://github.com/allegroai/clearml/issues/646)) -* Fix Model API calls fail when its creating task has been deleted ([ClearML GitHub issue #1299](https://github.com/allegroai/clearml/issues/1299)) -* Fix UI plots display "Iteration 0" when it is the only reported iteration ([ClearML GitHub issue #1267](https://github.com/allegroai/clearml/issues/1267)) -* Fix scalar series starting with angle bracket (`<`) causes UI scalar plot legend to display raw html ([ClearML GitHub issue #1292](https://github.com/allegroai/clearml/issues/1292)) -* Fix auto-refresh modifies UI experiment debug sample view ([ClearML GitHub issue #1259](https://github.com/allegroai/clearml/issues/1259)) +* Fix UI experiment textual comparison diff showing contextual diffs as different contents ([ClearML GitHub issue #646](https://github.com/clearml/clearml/issues/646)) +* Fix Model API calls fail when its creating task has been deleted ([ClearML GitHub issue #1299](https://github.com/clearml/clearml/issues/1299)) +* Fix UI plots display "Iteration 0" when it is the only reported iteration ([ClearML GitHub issue #1267](https://github.com/clearml/clearml/issues/1267)) +* Fix scalar series starting with angle bracket (`<`) causes UI scalar plot legend to display raw html ([ClearML GitHub issue #1292](https://github.com/clearml/clearml/issues/1292)) +* Fix auto-refresh modifies UI experiment debug sample view ([ClearML GitHub issue #1259](https://github.com/clearml/clearml/issues/1259)) * Fix Hydra parameters not displaying correctly in UI experiment comparison parallel coordinates plot -* Fix UI pipeline DAG display to be "bottoms-up" so all final steps appear in the bottom of the diagram ([ClearML Web GitHub PR #86](https://github.com/allegroai/clearml-web/pull/86)) -* Fix UI experiment comparison does not overlay box plots and histograms ([ClearML GitHub issue #1298](https://github.com/allegroai/clearml/issues/1298)) +* Fix UI pipeline DAG display to be "bottoms-up" so all final steps appear in the bottom of the diagram ([ClearML Web GitHub PR #86](https://github.com/clearml/clearml-web/pull/86)) +* Fix UI experiment comparison does not overlay box plots and histograms ([ClearML GitHub issue #1298](https://github.com/clearml/clearml/issues/1298)) * Fix modifying UI experiment input models does not work * Fix UI Reports image upload not working * Fix active masks labels not displaying in UI annotation task frame viewer diff --git a/docs/release_notes/clearml_server/enterprise/ver_3_24.md b/docs/release_notes/clearml_server/enterprise/ver_3_24.md index 174b8bfb..0dc36446 100644 --- a/docs/release_notes/clearml_server/enterprise/ver_3_24.md +++ b/docs/release_notes/clearml_server/enterprise/ver_3_24.md @@ -11,7 +11,7 @@ title: Version 3.24 **New Features** -* New look and feel: Full light/dark themes ([ClearML GitHub issue #1297](https://github.com/allegroai/clearml/issues/1297)) +* New look and feel: Full light/dark themes ([ClearML GitHub issue #1297](https://github.com/clearml/clearml/issues/1297)) * New UI task creation options * Support bash as well as python scripts * Support file upload diff --git a/docs/release_notes/clearml_server/open_source/ver_0_10.md b/docs/release_notes/clearml_server/open_source/ver_0_10.md index 10fc0cc2..414ac03d 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_10.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_10.md @@ -9,13 +9,13 @@ title: Version 0.10 ### Trains Server 0.10.1 -* Add simple username / password login scheme, see [GitHub trains-server Issue 3](https://github.com/allegroai/trains-server/issues/3) +* Add simple username / password login scheme, see [GitHub trains-server Issue 3](https://github.com/clearml/clearml-server/issues/3) * Fixed User/Password configuration (see [instructions](../../../deploying_clearml/clearml_server_config.md#web-login-authentication)). * Replace the Python web-server with NGINX. -* Improve sub-domain support ([GitHub trains-server Issue 9](https://github.com/allegroai/trains-server/issues/9)). +* Improve sub-domain support ([GitHub trains-server Issue 9](https://github.com/clearml/clearml-server/issues/9)). * Extend configuration options diff --git a/docs/release_notes/clearml_server/open_source/ver_0_12.md b/docs/release_notes/clearml_server/open_source/ver_0_12.md index d318b391..70063a9a 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_12.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_12.md @@ -10,7 +10,7 @@ title: Version 0.12 Minor bug fixes and improvements * Add resource monitoring. -* Fix Web UI compare plots ([GitHub Issue #55](https://github.com/allegroai/clearml/issues/55)). +* Fix Web UI compare plots ([GitHub Issue #55](https://github.com/clearml/clearml/issues/55)). * Improve server upgrade checks/messages. ### Trains Server 0.12.0 diff --git a/docs/release_notes/clearml_server/open_source/ver_0_13.md b/docs/release_notes/clearml_server/open_source/ver_0_13.md index 7edbd8eb..134ce6f3 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_13.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_13.md @@ -11,10 +11,10 @@ title: Version 0.13 **Features and Bug Fixes** * Add parallel coordinates hyperparameter comparison, available under **Compare Experiments** **>** **HYPERPARAMETERS** - **>** **Parallel Coordinates** (in the drop-down) ([GitHub Issue #53](https://github.com/allegroai/trains/issues/53)). + **>** **Parallel Coordinates** (in the drop-down) ([GitHub Issue #53](https://github.com/clearml/clearml/issues/53)). * Add encoding of experiment table view settings in URL to allow sharing, using browser URL copy / paste. -* Add loguru (ANSI color) support ([GitHub Issue #29](https://github.com/allegroai/trains/issues/29)). -* Add support for special characters in hyperparameter keys (white-spaces, `.` and `$`) ([GitHub Issue #69](https://github.com/allegroai/trains/issues/69)). +* Add loguru (ANSI color) support ([GitHub Issue #29](https://github.com/clearml/clearml/issues/29)). +* Add support for special characters in hyperparameter keys (white-spaces, `.` and `$`) ([GitHub Issue #69](https://github.com/clearml/clearml/issues/69)). * Add optional anonymous daily usage statistics (help us improve Trains Server): * Disabled by default. * Requires user opt-in. @@ -23,5 +23,5 @@ title: Version 0.13 * Reports average workload per day (amount and average duration of queues, agents and experiments). * Improve experiment table filtering indication. * Improve model view to allow navigating to its generating experiment. -* Fix experiment comparison to distinguish between experiments with the same name ([GitHub Issue #52](https://github.com/allegroai/trains/issues/52)). -* Fix Web UI compare plots bug ([GitHub Issue #55](https://github.com/allegroai/trains/issues/55) and [GitHub Issue #73](https://github.com/allegroai/trains/issues/73)). +* Fix experiment comparison to distinguish between experiments with the same name ([GitHub Issue #52](https://github.com/clearml/clearml/issues/52)). +* Fix Web UI compare plots bug ([GitHub Issue #55](https://github.com/clearml/clearml/issues/55) and [GitHub Issue #73](https://github.com/clearml/clearml/issues/73)). diff --git a/docs/release_notes/clearml_server/open_source/ver_0_14.md b/docs/release_notes/clearml_server/open_source/ver_0_14.md index 0685701c..ba3b9a52 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_14.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_14.md @@ -17,14 +17,14 @@ title: Version 0.14 * Fix misaligned titles in experiment results, scalars, and plots (**RESULTS** tab, **SCALARS** and **PLOTS** sub-tabs). * Fix parallel coordinate comparison in comparing experiments. Adding dimension with no values removes all lines. * Fix resizing the experiments tables window. The preferences wheel jumped over the titles. -* Fix the browser zoom (**CTRL +**, **CTRL -**, and mouse wheel) in experiment **RESULTS** **>** **SCALARS** tab. It broke labels ([GitHub Issue #127](https://github.com/allegroai/trains/issues/127)). +* Fix the browser zoom (**CTRL +**, **CTRL -**, and mouse wheel) in experiment **RESULTS** **>** **SCALARS** tab. It broke labels ([GitHub Issue #127](https://github.com/clearml/clearml/issues/127)). ### Trains Server 0.14.1 **Features and Bug Fixes** * Support default **Trains Server** Kubernetes Helm ports in the **Trains Web-App (UI)**. -* Fix issue switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/allegroai/trains/issues/75)). +* Fix issue switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/clearml/clearml/issues/75)). * Do not reset view filtering when switching tabs in experiment view. ### Trains Server 0.14.0 @@ -32,8 +32,8 @@ title: Version 0.14 **Features and Bug Fixes** * Add debug image viewing for an unlimited number of iterations, filtering by metric, and comparing debug images for different iterations. -* Add debug image view zoom, fit to screen, resize, hover coordinates ([GitHub Issue #68](https://github.com/allegroai/trains/issues/68)). -* Add switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/allegroai/trains/issues/75)). +* Add debug image view zoom, fit to screen, resize, hover coordinates ([GitHub Issue #68](https://github.com/clearml/clearml/issues/68)). +* Add switching scalar plots between linear and logarithmic views ([GitHub Issue #75](https://github.com/clearml/clearml/issues/75)). * Add scalar minimums and maximums to comparing experiments. * Add opening HTML artifact files in the browser. * Add Web-App notification for version updates. @@ -44,8 +44,8 @@ title: Version 0.14 * Allow editing model description. * Allow editing experiment uncommitted changes and installed packages separately. * Allow deleting an input model from an experiment whose status is *Draft*. -* Fix dialog elements which are now accessible in non-100% screen scale. ([GitHub Issue #89](https://github.com/allegroai/trains/issues/89)). -* Fix custom columns in the experiments table not properly loaded from browser store ([GitHub Issue #106](https://github.com/allegroai/trains/issues/106)). -* Fix issue with showing 2D graphs containing sub-plots ([trains #slack](https://github.com/allegroai/trains/issues/106)) +* Fix dialog elements which are now accessible in non-100% screen scale. ([GitHub Issue #89](https://github.com/clearml/clearml/issues/89)). +* Fix custom columns in the experiments table not properly loaded from browser store ([GitHub Issue #106](https://github.com/clearml/clearml/issues/106)). +* Fix issue with showing 2D graphs containing sub-plots ([trains #slack](https://github.com/clearml/clearml/issues/106)) * Fix issues with vertical scrolling of experiment logs. * Fix issue with image plots sometimes not visible in experiments comparison. diff --git a/docs/release_notes/clearml_server/open_source/ver_0_15.md b/docs/release_notes/clearml_server/open_source/ver_0_15.md index 9846b0b2..cd0a43e6 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_15.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_15.md @@ -10,8 +10,8 @@ title: Version 0.15 **Bug Fixes** -* Fix experiment clone dialog resets while typing ([GitHub trains-server Issue 44](https://github.com/allegroai/trains-server/issues/44)). -* Fix image-only plots are not displayed ([GitHub Issue #140](https://github.com/allegroai/trains/issues/140)). +* Fix experiment clone dialog resets while typing ([GitHub trains-server Issue 44](https://github.com/clearml/clearml-server/issues/44)). +* Fix image-only plots are not displayed ([GitHub Issue #140](https://github.com/clearml/clearml/issues/140)). * Fix last item in plots list is hidden. * Fix input model edit view contains a redundant save button. * Fix credentials popup shows incorrect URL for fileserver. @@ -30,12 +30,12 @@ title: Version 0.15 **Features** -* Add support for debug media (images, audio, video) ([GitHub Issue #120](https://github.com/allegroai/trains/issues/120)). +* Add support for debug media (images, audio, video) ([GitHub Issue #120](https://github.com/clearml/clearml/issues/120)). * Add support for previewing HTML debug samples. -* Add experiment and model tag management ([GitHub Issue #46](https://github.com/allegroai/trains/issues/46)). +* Add experiment and model tag management ([GitHub Issue #46](https://github.com/clearml/clearml/issues/46)). * Add support for more task types (see Trains v0.15.0 and Trains Agent v0.15.0 for related support and services mode). * Experiment comparison: - * Add Last / Max (maximal) / Min (minimal) scalar values to parallel coordinates in experiment comparison ([GitHub Issue #131](https://github.com/allegroai/trains/issues/131)). + * Add Last / Max (maximal) / Min (minimal) scalar values to parallel coordinates in experiment comparison ([GitHub Issue #131](https://github.com/clearml/clearml/issues/131)). * Add deep comparison support to experiment comparison, including uncommitted changes by file with line diff and search of all text. * Enhance table columns: * Add dynamic column ordering to the experiments and models tables. @@ -59,14 +59,14 @@ title: Version 0.15 **Bug Fixes** -* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/allegroai/trains-agent/issues/19)). +* Fix init wizard, correctly display the input servers ([GitHub Issue #19](https://github.com/clearml/clearml-agent/issues/19)). * Improve docker host-mount support, use `TRAINS_AGENT_DOCKER_HOST_MOUNT` environment variable. * Support pip v20.1 local / http package reference in pip freeze. * Fix detached mode to correctly use cache folder slots. * Fix `CUDA_VISIBLE_DEVICES` should never be set to "all". * Do not monitor GPU when running with `--cpu-only` * Fix experiment comparison issues: - * Scalar comparison type does not reflect displayed comparison ([GitHub Issue #40](https://github.com/allegroai/trains/issues/40)). + * Scalar comparison type does not reflect displayed comparison ([GitHub Issue #40](https://github.com/clearml/clearml/issues/40)). * Comparison animations overlapping other fields. * Cannot choose a metric after clearing it in experiment comparison parallel coordinates. * Adding a dimension without a value removes all lines in experiment comparison parallel coordinates. @@ -77,14 +77,14 @@ title: Version 0.15 * Changing experiment table / info panel split redraws scalars and plots. * Wrong download URL shown for S3 artifact links with special characters. * Fix experiment results view issues: - * Browser zoom breaks labels (Ctrl + / Ctrl - / mouse wheel) in experiment results **SCALARS** tab ([GitHub Issue #127](https://github.com/allegroai/trains/issues/127)). + * Browser zoom breaks labels (Ctrl + / Ctrl - / mouse wheel) in experiment results **SCALARS** tab ([GitHub Issue #127](https://github.com/clearml/clearml/issues/127)). * Debug image name not shown if file cannot be loaded. * minio images with spaces in the path not displaying correctly * Plots should use the full panel space. * Plot relative time displaying incorrectly in experiment results **SCALARS** tab. * Wall time display in local time in experiment results **SCALARS** tab. * Fix experiment table issues: - * Hyperparameters sorted as strings, not as numbers ([GitHub Issue #124](https://github.com/allegroai/trains/issues/124)). + * Hyperparameters sorted as strings, not as numbers ([GitHub Issue #124](https://github.com/clearml/clearml/issues/124)). * Prevent auto refresh reordering when the info panel is open. * Table resizing made the preferences wheel to jump over titles. * Fix experiment reset when both the input and output model are the same causes model deletion. diff --git a/docs/release_notes/clearml_server/open_source/ver_0_16.md b/docs/release_notes/clearml_server/open_source/ver_0_16.md index 0b397a6b..80e45dff 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_16.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_16.md @@ -16,7 +16,7 @@ Upgrading to this version requires a manual [data migration](../../../deploying_ * Fix model page issue causing N/A to show after switching tabs (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597672446125300). * Removed experiments comparison limit (only 10 were allowed). Limit is now 100, configurable using `services.tasks.multi_task_histogram_limit`. (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597755746141500). * Fix scalar plots sometimes not calculated by the server in lower iteration values (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597321653085900). -* Fix error while retrieving experiment log when only a few lines were reported ([GitHub trains-server Issue 59](https://github.com/allegroai/trains-server/issues/59)). +* Fix error while retrieving experiment log when only a few lines were reported ([GitHub trains-server Issue 59](https://github.com/clearml/clearml-server/issues/59)). * Update Fixed User full-name on restart (Trains Slack channel [thread)](https://clearml.slack.com/archives/CTK20V944/p1597906492003300). * Fix project ordering issue. * When loading plots, display a spinner and don't show "no data". @@ -41,13 +41,13 @@ Upgrading to this version requires a manual [data migration](../../../deploying_ * Model design is in the **ARTIFACTS** tab. * Experiment model description is in the **CONFIGURATION OBJECTS** section in the **CONFIGURATION** tab. * Improve experiment comparison: - * In hyperparameter parallel coordinate comparison, hover over an experiment name to highlight it on plot ([GitHub Issue #53](https://github.com/allegroai/trains/issues/53)). + * In hyperparameter parallel coordinate comparison, hover over an experiment name to highlight it on plot ([GitHub Issue #53](https://github.com/clearml/clearml/issues/53)). * Remove fields providing no additional information from comparison. * Improve the model framework filter. Filter contains only frameworks used by models in the project. * Add configurable **Trains** services examples. * Add support for text debug samples in the **DEBUG SAMPLES** section in the **RESULTS** tab. * Add legend on / off toggle control for every plot. -* Add clear button for text areas ([GitHub trains-server Issue #42](https://github.com/allegroai/trains-server/issues/42)). +* Add clear button for text areas ([GitHub trains-server Issue #42](https://github.com/clearml/clearml-server/issues/42)). * Reinstate the bottom bar **Archive** button. * Add **Trains** community links to left bar. * Add Hi-DPI display support. diff --git a/docs/release_notes/clearml_server/open_source/ver_0_17.md b/docs/release_notes/clearml_server/open_source/ver_0_17.md index 3ae0abe5..1de8d386 100644 --- a/docs/release_notes/clearml_server/open_source/ver_0_17.md +++ b/docs/release_notes/clearml_server/open_source/ver_0_17.md @@ -14,11 +14,11 @@ title: Version 0.17 * Add experiment active duration custom column to experiment table. * Improve comparison experiment consistency in plots for experiment name and color. * Improve project last update time; updated whenever an experiment or model in the project is updated. -* Improve table multi-select UX: shift+mouse-click ([clearml GitHub Issue 61](https://github.com/allegroai/trains/issues/61)). +* Improve table multi-select UX: shift+mouse-click ([clearml GitHub Issue 61](https://github.com/clearml/clearml/issues/61)). * Add experiment parent to experiment details INFO tab. * Improve UI experiment archiving; archiving queued experiments automatically dequeues them. * Improve experiment comparison; use available screen width. -* Add debug sample history tracking ([clearml GitHub Issue 81](https://github.com/allegroai/trains/issues/81)). +* Add debug sample history tracking ([clearml GitHub Issue 81](https://github.com/clearml/clearml/issues/81)). * Replace "Download" action with "Copy to Clipboard" for local files. * Add ClearML usage tips. * Add Task and Model, Clone and Move to allow creating new project on the fly. @@ -32,13 +32,13 @@ title: Version 0.17 **Bug Fixes** * Fix HiDPI table issues. -* Fix results / plots not scaling properly ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)). +* Fix results / plots not scaling properly ([clearml GitHub Issue 247](https://github.com/clearml/clearml/issues/247)). * Fix experiment comparison: * Cannot compare regular Tasks to example Tasks. * Cannot unhide scalars. * Scalars wall time view and relative view will not auto-refresh. * Fix log scrolling in Chrome. -* Fix UI adds to plots legend labels unnecessary task ID string ([clearml GitHub Issue 247](https://github.com/allegroai/trains/issues/247)). +* Fix UI adds to plots legend labels unnecessary task ID string ([clearml GitHub Issue 247](https://github.com/clearml/clearml/issues/247)). * Fix parallel coordinates: dimensions are not sorted by value. * Fix scalar auto color selection provides too similar colors. * Fix reordering a custom column does not restore column location. diff --git a/docs/release_notes/clearml_server/open_source/ver_1_0.md b/docs/release_notes/clearml_server/open_source/ver_1_0.md index c5fdd291..17529965 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_0.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_0.md @@ -6,7 +6,7 @@ title: Version 1.0 **Bug Fixes** -- Fix Task container does not accept `null` values ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1622119047293300) [ClearML GitHub issue 365](https://github.com/allegroai/clearml/issues/365)) +- Fix Task container does not accept `null` values ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1622119047293300) [ClearML GitHub issue 365](https://github.com/clearml/clearml/issues/365)) - Fix debug images exception in Results page - Fix a typo in Worker Setup help popup @@ -15,7 +15,7 @@ title: Version 1.0 **Bug Fixes** - Fix clearing experiment requirements causes "empty" requirements (as opposed to "no requirements") -- Fix logout fails with `endpoint not found` error ([ClearML GitHub issue 349](https://github.com/allegroai/clearml/issues/349)) +- Fix logout fails with `endpoint not found` error ([ClearML GitHub issue 349](https://github.com/clearml/clearml/issues/349)) - Fix hamburger side menu `Manage Queues` does nothing and returns console error ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1620308724418100)) - Fix broken config dir backwards compatibility (`/opt/trains/config` should also be supported) @@ -26,12 +26,12 @@ title: Version 1.0 * Add Multi-level project hierarchy - Projects can now contain sub-projects * Add markdown editor for project overview * Add support for multiple models per experiment -* Add Context menu batch operations ([ClearML GitHub issue 286](https://github.com/allegroai/clearml/issues/286)) -* Add global search regex support ([ClearML GitHub issue 81](https://github.com/allegroai/clearml/issues/81#issuecomment-735003956)) +* Add Context menu batch operations ([ClearML GitHub issue 286](https://github.com/clearml/clearml/issues/286)) +* Add global search regex support ([ClearML GitHub issue 81](https://github.com/clearml/clearml/issues/81#issuecomment-735003956)) * Add support for extended container (docker) configuration and init script * Add special character support ("." (dot) and "$") in Hyperparameters * Add new context menu icons -* Add deleting experiments and models through UI (ClearML GitHub issues [53](https://github.com/allegroai/clearml-server/issues/53), [61](https://github.com/allegroai/clearml-server/issues/61), and [81](https://github.com/allegroai/clearml/issues/81#issuecomment-706907718)) +* Add deleting experiments and models through UI (ClearML GitHub issues [53](https://github.com/clearml/clearml-server/issues/53), [61](https://github.com/clearml/clearml-server/issues/61), and [81](https://github.com/clearml/clearml/issues/81#issuecomment-706907718)) * Improve Project deletion: deletes project's experiments and models. * Add experiment description access from info title * Improve experiment table columns @@ -41,7 +41,7 @@ title: Version 1.0 * Add secondary nested sorting * Add `worker<->queue` clickable reference in workers and queues tables * Add presenting all metric debug samples concurrently -* Add Full project name display in project cards ([ClearML GitHub issue 81](https://github.com/allegroai/clearml/issues/81#issuecomment-823303842)) +* Add Full project name display in project cards ([ClearML GitHub issue 81](https://github.com/clearml/clearml/issues/81#issuecomment-823303842)) * Add option for continuing aborted tasks **Bug Fixes** @@ -54,15 +54,15 @@ title: Version 1.0 * Fix UI missing default selection of artifacts in full screen view * Fix UI variant name not shown in plots * Fix UI missing tooltip for truncated column headers -* Fix UI custom columns choice does not persist per project ([ClearML GitHub issue 314](https://github.com/allegroai/clearml/issues/314)) +* Fix UI custom columns choice does not persist per project ([ClearML GitHub issue 314](https://github.com/clearml/clearml/issues/314)) * Fix API plot_str not returned for compressed plots * Fix UI plots color picker consistency * Fix API ```Tasks.reset``` marking parent id as 'deleted' in its children * Fix UI missing queue selection on queue delete * Fix UI debug image history slider not shown when there's only a single iteration -* Fix UI X-axis labels are being cut in plots ([ClearML GitHub issue 264](https://github.com/allegroai/clearml/issues/264)) +* Fix UI X-axis labels are being cut in plots ([ClearML GitHub issue 264](https://github.com/clearml/clearml/issues/264)) * Fix UI scalar color choice dialog toggles between screen positions every time you open it -* Fix UI hovering on legend clears parallel coordinates graph filters ([ClearML GitHub issue 259](https://github.com/allegroai/clearml/issues/259)) +* Fix UI hovering on legend clears parallel coordinates graph filters ([ClearML GitHub issue 259](https://github.com/clearml/clearml/issues/259)) * Fix UI experiment comparison 'hide identical fields' button disable doesn't work * Fix UI auto refresh removes model ID in model panel * Fix UI debug samples download image opens new tab instead of downloading diff --git a/docs/release_notes/clearml_server/open_source/ver_1_1.md b/docs/release_notes/clearml_server/open_source/ver_1_1.md index 90637d7a..8048121f 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_1.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_1.md @@ -22,9 +22,9 @@ title: Version 1.1 **Bug Fixes** -- Fix experiment details UI failure opening hyperparameter sections beginning with `#` ([ClearML Server GitHub issue #79](https://github.com/allegroai/clearml-server/issues/79)) +- Fix experiment details UI failure opening hyperparameter sections beginning with `#` ([ClearML Server GitHub issue #79](https://github.com/clearml/clearml-server/issues/79)) - Fix performance issues with UI comparison of large experiments ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1621698235159800)) -- Fix filtering on hyperparameters ([ClearML GitHub issue #385](https://github.com/allegroai/clearml/issues/385) [Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1626600582284700)) +- Fix filtering on hyperparameters ([ClearML GitHub issue #385](https://github.com/clearml/clearml/issues/385) [Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1626600582284700)) - Fix profile page user options toggle control area of effect - Fix browser resizing affecting plot zoom - Fix UI dataview filter resetting version on filter addition diff --git a/docs/release_notes/clearml_server/open_source/ver_1_10.md b/docs/release_notes/clearml_server/open_source/ver_1_10.md index 4367048c..394f6bc1 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_10.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_10.md @@ -5,19 +5,19 @@ title: Version 1.10 ### ClearML Server 1.10.1 **Bug Fix** -* Fix WebApp does not connect to the server when using non-default ports ([ClearML Server GitHub issue #191](https://github.com/allegroai/clearml-server/issues/191)) +* Fix WebApp does not connect to the server when using non-default ports ([ClearML Server GitHub issue #191](https://github.com/clearml/clearml-server/issues/191)) ### ClearML Server 1.10.0 **New Features and Improvements** * Additional model information - * Add Lineage tab to UI model page, which displays experiment that created model and lists experiments that use the model ([ClearML GitHub issue #810](https://github.com/allegroai/clearml/issues/810)) + * Add Lineage tab to UI model page, which displays experiment that created model and lists experiments that use the model ([ClearML GitHub issue #810](https://github.com/clearml/clearml/issues/810)) * Add Plots and Scalars tabs to UI Model page -* Add project hierarchy view for ClearML resources (pipelines, datasets, hyper-datasets, reports) ([ClearML GitHub issue #724](https://github.com/allegroai/clearml/issues/724)): +* Add project hierarchy view for ClearML resources (pipelines, datasets, hyper-datasets, reports) ([ClearML GitHub issue #724](https://github.com/clearml/clearml/issues/724)): * Flat view - all objects displayed side-by-side * Nested view - objects grouped by containing projects. -* Add archived experiment inclusion in UI experiment comparison ([ClearML GitHub issue #795](https://github.com/allegroai/clearml/issues/795)) -* Update ElasticSearch image to 7.17.7 ([ClearML Server GitHub issue #183](https://github.com/allegroai/clearml-server/issues/183)) +* Add archived experiment inclusion in UI experiment comparison ([ClearML GitHub issue #795](https://github.com/clearml/clearml/issues/795)) +* Update ElasticSearch image to 7.17.7 ([ClearML Server GitHub issue #183](https://github.com/clearml/clearml-server/issues/183)) * Add reported plot support for multiple color formats * Add running tasks to UI project overview metric snapshot * Improve UI Reports: @@ -27,7 +27,7 @@ title: Version 1.10 * Add Experiment comparison UI parallel coordinate plots full screen view **Bug Fixes** -* Fix UI scalar plot smoothing is biased towards initial value ([ClearML Server GitHub issue #180](https://github.com/allegroai/clearml-server/issues/180)) +* Fix UI scalar plot smoothing is biased towards initial value ([ClearML Server GitHub issue #180](https://github.com/clearml/clearml-server/issues/180)) * Fix UI experiment plot comparison not merging plots * Fix UI reports search bar doesn't search report descriptions * Fix UI reports search bar does not close after result is selected diff --git a/docs/release_notes/clearml_server/open_source/ver_1_11.md b/docs/release_notes/clearml_server/open_source/ver_1_11.md index f312d2e0..6c18ed44 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_11.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_11.md @@ -7,7 +7,7 @@ title: Version 1.11 **New Features and Improvements** * New model comparison UI: compare model configuration, metrics, and plots * Add Preview tab to UI pipeline for viewing debug samples and plots attached to a pipeline -* Add dynamic side panel width in UI experiment artifacts and configuration ([ClearML Server GitHub issue #160](https://github.com/allegroai/clearml-server/issues/160)) +* Add dynamic side panel width in UI experiment artifacts and configuration ([ClearML Server GitHub issue #160](https://github.com/clearml/clearml-server/issues/160)) * Add support for embedding following UI resources in reports: * Single-value-scalar table plots * Application dashboard graphs @@ -21,11 +21,11 @@ title: Version 1.11 **Bug Fixes** -* Fix UI experiment plot history navigation not working correctly ([ClearML GitHub issue #955](https://github.com/allegroai/clearml/issues/955)) -* Support browser native context menu (via right click) in UI experiment hyperparameters ([ClearML GitHub issue #990](https://github.com/allegroai/clearml/issues/990)) -* Fix `series` argument not in UI experiment debug samples when using `logger.report_media` ([ClearML GitHub issue #685](https://github.com/allegroai/clearml/issues/685)) -* Fix smoothing bar not displayed in UI experiment comparison scalars in full screen mode ([ClearML GitHub issue #996](https://github.com/allegroai/clearml/issues/996)) -* Fix uploading image in UI report raises error ([ClearML GitHub issue #1020](https://github.com/allegroai/clearml/issues/1020)) +* Fix UI experiment plot history navigation not working correctly ([ClearML GitHub issue #955](https://github.com/clearml/clearml/issues/955)) +* Support browser native context menu (via right click) in UI experiment hyperparameters ([ClearML GitHub issue #990](https://github.com/clearml/clearml/issues/990)) +* Fix `series` argument not in UI experiment debug samples when using `logger.report_media` ([ClearML GitHub issue #685](https://github.com/clearml/clearml/issues/685)) +* Fix smoothing bar not displayed in UI experiment comparison scalars in full screen mode ([ClearML GitHub issue #996](https://github.com/clearml/clearml/issues/996)) +* Fix uploading image in UI report raises error ([ClearML GitHub issue #1020](https://github.com/clearml/clearml/issues/1020)) * Add mobile device support for UI experiment tab selection * Fix deleting models doesn't remove their weight files from fileserver * Fix "Load More" button does not work in UI experiment comparison "Add Experiment" modal diff --git a/docs/release_notes/clearml_server/open_source/ver_1_12.md b/docs/release_notes/clearml_server/open_source/ver_1_12.md index 0bd5ad5d..e15b15fc 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_12.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_12.md @@ -5,7 +5,7 @@ title: Version 1.12 ### ClearML Server 1.12.1 **Bug Fixes** -* Fix UI experiment scalars and plots not updated on auto-refresh ([ClearML GitHub issue #1086](https://github.com/allegroai/clearml/issues/1086)) +* Fix UI experiment scalars and plots not updated on auto-refresh ([ClearML GitHub issue #1086](https://github.com/clearml/clearml/issues/1086)) * Fix pipeline parameters not displayed in UI pipeline "New Run" modal * Fix UI pipeline project cards display incorrect pipeline run counts @@ -13,14 +13,14 @@ title: Version 1.12 **New Features and Improvements** * Additional UI cloud storage access options: - * Support for AWS S3 temporary access tokens ([ClearML GitHub issue #200](https://github.com/allegroai/clearml-server/issues/200) and [ClearML Web GitHub issue #52](https://github.com/allegroai/clearml-web/issues/52)) + * Support for AWS S3 temporary access tokens ([ClearML GitHub issue #200](https://github.com/clearml/clearml-server/issues/200) and [ClearML Web GitHub issue #52](https://github.com/clearml/clearml-web/issues/52)) * Support credentials for private GCS buckets -* Add multiple smoothing algorithms to UI scalar plots ([ClearML GitHub issue #996](https://github.com/allegroai/clearml/issues/996)) +* Add multiple smoothing algorithms to UI scalar plots ([ClearML GitHub issue #996](https://github.com/clearml/clearml/issues/996)) * Running average * Exponential running average * Gaussian -* Add exporting UI object table data in CSV format ([ClearML GitHub issue #882](https://github.com/allegroai/clearml/issues/882)) -* Increase console log page size ([ClearML Web GitHub PR #58](https://github.com/allegroai/clearml-web/pull/58)) +* Add exporting UI object table data in CSV format ([ClearML GitHub issue #882](https://github.com/clearml/clearml/issues/882)) +* Increase console log page size ([ClearML Web GitHub PR #58](https://github.com/clearml/clearml-web/pull/58)) * Add Python executable to UI experiment execution tab * Improve naming scheme for UI plots download as PNG: files are saved with same name as plot object * Add keyboard arrow-key controls for UI "Add tag" menu @@ -30,9 +30,9 @@ title: Version 1.12 * Add iteration specification to plots in task comparison plots page **Bug Fixes** -* Fix handling of the subpaths with nginx templating ([ClearML Server GitHub PR #204](https://github.com/allegroai/clearml-server/pull/204)) -* Fix fullscreen single-iteration plot displays history slider ([ClearML GitHub issue #909](https://github.com/allegroai/clearml/issues/909)) -* Fix UI new project modal does not accept Azure URLs for default output destination field ([ClearML GitHub issue #1047](https://github.com/allegroai/clearml/issues/1047)) +* Fix handling of the subpaths with nginx templating ([ClearML Server GitHub PR #204](https://github.com/clearml/clearml-server/pull/204)) +* Fix fullscreen single-iteration plot displays history slider ([ClearML GitHub issue #909](https://github.com/clearml/clearml/issues/909)) +* Fix UI new project modal does not accept Azure URLs for default output destination field ([ClearML GitHub issue #1047](https://github.com/clearml/clearml/issues/1047)) * Fix missing "Description" column in UI dataset version table * Fix UI scalar plots notation for small and large scale values * Fix UI experiment comparison does not display experiments' last reported iterations diff --git a/docs/release_notes/clearml_server/open_source/ver_1_13.md b/docs/release_notes/clearml_server/open_source/ver_1_13.md index 8f5f67e7..281d29cb 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_13.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_13.md @@ -12,22 +12,22 @@ title: Version 1.13 **New Features and Improvements** * Improve UI comparison - * Add visual comparison of single value scalars ([ClearML Server GitHub issue #202](https://github.com/allegroai/clearml-server/issues/202)) - * Scalar value comparison: metrics are presented in a single table ([ClearML GitHub issue #760](https://github.com/allegroai/clearml/issues/760)) - * Add global experiment legend: Experiment name, tags and color ([ClearML GitHub issue #81](https://github.com/allegroai/clearml/issues/81)) - * Clickable URLs/object IDs in experiment details comparison ([ClearML GitHub issue #1073](https://github.com/allegroai/clearml/issues/1073)) + * Add visual comparison of single value scalars ([ClearML Server GitHub issue #202](https://github.com/clearml/clearml-server/issues/202)) + * Scalar value comparison: metrics are presented in a single table ([ClearML GitHub issue #760](https://github.com/clearml/clearml/issues/760)) + * Add global experiment legend: Experiment name, tags and color ([ClearML GitHub issue #81](https://github.com/clearml/clearml/issues/81)) + * Clickable URLs/object IDs in experiment details comparison ([ClearML GitHub issue #1073](https://github.com/clearml/clearml/issues/1073)) * UI Search improvements * Dashboard search results ordered by update time * Top bar search query saved in URL -* Improve UI experiment debug sample iteration fetching speed ([ClearML Server GitHub issue #170](https://github.com/allegroai/clearml-server/issues/170)) +* Improve UI experiment debug sample iteration fetching speed ([ClearML Server GitHub issue #170](https://github.com/clearml/clearml-server/issues/170)) * Add UI cloud credentials support for GCS signed URLs with HMAC authentication * Add Experiment plot UI support for displaying subplots **Bug Fixes** -* Fix UI single-value scalars table does not display all single-value scalars ([ClearML GitHub issue #1111](https://github.com/allegroai/clearml/issues/1111)) -* Fix UI pipeline step duration does not update during runtime ([ClearML GitHub issue #1090](https://github.com/allegroai/clearml/issues/1090)) -* Fix UI experiment comparison parallel coordinates plot crops the metric and parameter names ([ClearML GitHub issue #1111](https://github.com/allegroai/clearml/issues/1111)) -* Fix hidden experiments appear in UI All Experiments search results ([ClearML GitHub issue #1139](https://github.com/allegroai/clearml/issues/1139)) +* Fix UI single-value scalars table does not display all single-value scalars ([ClearML GitHub issue #1111](https://github.com/clearml/clearml/issues/1111)) +* Fix UI pipeline step duration does not update during runtime ([ClearML GitHub issue #1090](https://github.com/clearml/clearml/issues/1090)) +* Fix UI experiment comparison parallel coordinates plot crops the metric and parameter names ([ClearML GitHub issue #1111](https://github.com/clearml/clearml/issues/1111)) +* Fix hidden experiments appear in UI All Experiments search results ([ClearML GitHub issue #1139](https://github.com/clearml/clearml/issues/1139)) * Fix disabled graphs in UI scalars reappear in full-screen mode * Fix UI auto-refresh sometimes causes many error message to be raised * Fix "All" tag filter not working in UI object pages diff --git a/docs/release_notes/clearml_server/open_source/ver_1_14.md b/docs/release_notes/clearml_server/open_source/ver_1_14.md index a726ad9d..9f49c089 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_14.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_14.md @@ -9,7 +9,7 @@ title: Version 1.14 * Change default cookie `SameSite` setting to `Lax` **Bug Fixes** -* Fix clicking on UI Metric Snapshot datapoint does not navigate to experiment ([ClearML Server GitHub issue #228](https://github.com/allegroai/clearml-server/issues/228)) +* Fix clicking on UI Metric Snapshot datapoint does not navigate to experiment ([ClearML Server GitHub issue #228](https://github.com/clearml/clearml-server/issues/228)) * Fix some APIs not sanitizing query parameters * Fix downloading UI experiment console log sometimes returns an older file * Fix UI experiment plot comparison displays duplicates of image plots @@ -20,7 +20,7 @@ title: Version 1.14 **New Features and Improvements** * New UI experiment table comparative view: compare plots and scalars of all selected experiments -* Add UI experiment display of original Python requirements along with actual packages used ([ClearML GitHub issue #793](https://github.com/allegroai/clearml/issues/793)) +* Add UI experiment display of original Python requirements along with actual packages used ([ClearML GitHub issue #793](https://github.com/clearml/clearml/issues/793)) * Add UI project metric snapshot support for multiple metrics * Add compressed UI experiment table info panel mode displaying only experiment name and status * Add "x unified" hover mode to UI plots diff --git a/docs/release_notes/clearml_server/open_source/ver_1_15.md b/docs/release_notes/clearml_server/open_source/ver_1_15.md index 27b7f5df..a75fe2b0 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_15.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_15.md @@ -5,23 +5,23 @@ title: Version 1.15 ### ClearML Server 1.15.1 **Bug Fix** -* Fix restarting ClearML Server deletes app credentials ([ClearML Server GitHub issue #240](https://github.com/allegroai/clearml-server/issues/240) and ClearML GitHub issues [#1247](https://github.com/allegroai/clearml/issues/1257) & [#1262](https://github.com/allegroai/clearml/issues/1262)) +* Fix restarting ClearML Server deletes app credentials ([ClearML Server GitHub issue #240](https://github.com/clearml/clearml-server/issues/240) and ClearML GitHub issues [#1247](https://github.com/clearml/clearml/issues/1257) & [#1262](https://github.com/clearml/clearml/issues/1262)) ### ClearML Server 1.15.0 **New Features and Improvements** * UI experiment hyperparameter comparison * Add Parallel coordinate view support for multiple metrics - * Add scatter plot mode to UI experiment hyperparameter comparison ([ClearML GitHub issue #984](https://github.com/allegroai/clearml/issues/984)) + * Add scatter plot mode to UI experiment hyperparameter comparison ([ClearML GitHub issue #984](https://github.com/clearml/clearml/issues/984)) * Add model lineage to UI model comparison -* Update nginx version to Fix 1-Byte Memory Overwrite RCE vulnerability ([ClearML Server GitHub issue #230](https://github.com/allegroai/clearml-server/issues/230)) +* Update nginx version to Fix 1-Byte Memory Overwrite RCE vulnerability ([ClearML Server GitHub issue #230](https://github.com/clearml/clearml-server/issues/230)) **Bug Fixes** -* Fix Heap Buffer Overflow vulnerability (CVE-2023-38545) ([ClearML Server GitHub issue #234](https://github.com/allegroai/clearml-server/issues/234)) +* Fix Heap Buffer Overflow vulnerability (CVE-2023-38545) ([ClearML Server GitHub issue #234](https://github.com/clearml/clearml-server/issues/234)) * Fix insufficient session expiration vulnerability (CWE-613) * Fix XSS vulnerability (CWE-79) in UI markdown * Fix UI reports vulnerable to XSS attack -* Fix UI generating undefined credentials for Jupyter Notebook ([ClearML Web GitHub issue #67](https://github.com/allegroai/clearml-web/issues/67)) +* Fix UI generating undefined credentials for Jupyter Notebook ([ClearML Web GitHub issue #67](https://github.com/clearml/clearml-web/issues/67)) * Fix "No tags" filter raises an error * Fix slow loading speed for UI experiment comparison scalars * Fix some UI experiment sometimes missing plots diff --git a/docs/release_notes/clearml_server/open_source/ver_1_16.md b/docs/release_notes/clearml_server/open_source/ver_1_16.md index 2d81b2f7..c74b28ab 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_16.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_16.md @@ -10,7 +10,7 @@ title: Version 1.16 ### ClearML Server 1.16.1 **Bug Fix** -* Fix user credentials reset on API server restart ([ClearML Server GitHub issue #250](https://github.com/allegroai/clearml-server/issues/250)) +* Fix user credentials reset on API server restart ([ClearML Server GitHub issue #250](https://github.com/clearml/clearml-server/issues/250)) ### ClearML Server 1.16.0 @@ -26,8 +26,8 @@ configuration setting * Fix UI scalar “Export to CSV” button returns an empty CSV file * Fix large number of tags in UI tables' tag filter causes UI to crash * Fix searching in UI object tables' tag filter causes UI to crash -* Fix consistent plot colors not maintained when embedded in UI Reports ([ClearML GitHub issue #1239](https://github.com/allegroai/clearml/issues/1239)) -* Fix UI plot legends and tooltips wrapping ([ClearML Server GitHub issue #241](https://github.com/allegroai/clearml-server/issues/241)) +* Fix consistent plot colors not maintained when embedded in UI Reports ([ClearML GitHub issue #1239](https://github.com/clearml/clearml/issues/1239)) +* Fix UI plot legends and tooltips wrapping ([ClearML Server GitHub issue #241](https://github.com/clearml/clearml-server/issues/241)) * Fix multiple parameters can be selected for plot axes in scatter plot view in UI experiment comparison * Fix image names change in UI task debug sample comparison when a different metric is selected * Fix numeric range filters of UI table columns not working diff --git a/docs/release_notes/clearml_server/open_source/ver_1_17.md b/docs/release_notes/clearml_server/open_source/ver_1_17.md index e6c9873a..51917272 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_17.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_17.md @@ -5,7 +5,7 @@ title: Version 1.17 ### ClearML Server 1.17.1 **Bug Fixes** -* Fix invalid mongodb connection string if `CLEARML_MONGODB_SERVICE_CONNECTION_STRING` is specified ([ClearML Server GitHub issue #252](https://github.com/allegroai/clearml-server/issues/252)) +* Fix invalid mongodb connection string if `CLEARML_MONGODB_SERVICE_CONNECTION_STRING` is specified ([ClearML Server GitHub issue #252](https://github.com/clearml/clearml-server/issues/252)) * Fix UI dataset Preview tab does not display dataset contents * Fix resetting UI experiments sometimes does not clear scalar plots @@ -14,7 +14,7 @@ title: Version 1.17 **New Features** * New ClearML Model dashboard: View all live model endpoints in a single location, complete with real time metrics reporting. * New UI pipeline run table comparative view: compare plots and scalars of selected pipeline runs -* Improve services agent behavior: If no credentials are specified, agent uses default credentials ([ClearML Server GitHub issue #140](https://github.com/allegroai/clearml-server/issues/140)) +* Improve services agent behavior: If no credentials are specified, agent uses default credentials ([ClearML Server GitHub issue #140](https://github.com/clearml/clearml-server/issues/140)) * Add UI re-enqueue of failed tasks * Add UI experiment scalar results table view * Add "Block running user's scripts in the browser" UI setting option for added security @@ -22,18 +22,18 @@ title: Version 1.17 * Add UI edit of default Project default output destination **Bug Fixes** -* Fix broken download links to artifacts stored in Azure ([ClearML Server GitHub issue #247](https://github.com/allegroai/clearml-server/issues/247)) +* Fix broken download links to artifacts stored in Azure ([ClearML Server GitHub issue #247](https://github.com/clearml/clearml-server/issues/247)) * Fix Cross-site Scripting (XSS) vulnerability (CWE-79) -* Fix UI experiment textual comparison diff showing contextual diffs as different contents ([ClearML GitHub issue #646](https://github.com/allegroai/clearml/issues/646)) -* Fix UI experiment comparison does not overlay box plots and histograms ([ClearML GitHub issue #1298](https://github.com/allegroai/clearml/issues/1298)) -* Fix UI plots display “Iteration 0” when it is the only reported iteration ([ClearML GitHub issue #1267](https://github.com/allegroai/clearml/issues/1267)) -* Fix scalar series starting with angle bracket (`<`) causes UI scalar plot legend to display raw html ([ClearML GitHub issue #1292](https://github.com/allegroai/clearml/issues/1292)) +* Fix UI experiment textual comparison diff showing contextual diffs as different contents ([ClearML GitHub issue #646](https://github.com/clearml/clearml/issues/646)) +* Fix UI experiment comparison does not overlay box plots and histograms ([ClearML GitHub issue #1298](https://github.com/clearml/clearml/issues/1298)) +* Fix UI plots display “Iteration 0” when it is the only reported iteration ([ClearML GitHub issue #1267](https://github.com/clearml/clearml/issues/1267)) +* Fix scalar series starting with angle bracket (`<`) causes UI scalar plot legend to display raw html ([ClearML GitHub issue #1292](https://github.com/clearml/clearml/issues/1292)) * Fix UI scalar plot not displayed if metric name includes a slash surrounded by spaces (`/`) -* Fix Model API calls fail when its creating task has been deleted ([ClearML GitHub issue #1299](https://github.com/allegroai/clearml/issues/1299)) -* Fix UI pipeline DAG display to be "bottom-up" so all final steps appear in the bottom of the diagram ([ClearML Web GitHub PR #86](https://github.com/allegroai/clearml-web/pull/86)) -* Fix invalid mongodb connection string if `CLEARML_MONGODB_SERVICE_CONNECTION_STRING` is specified ([ClearML Server GitHub issue #252](https://github.com/allegroai/clearml-server/issues/252)) -* Fix auto-refresh modifies UI experiment debug sample view ([ClearML GitHub issue #1259](https://github.com/allegroai/clearml/issues/1259)) -* Fix UI image plot retrieval from fileserver missing authentication cookie ([ClearML GitHub issue #1331](https://github.com/allegroai/clearml/issues/1331)) +* Fix Model API calls fail when its creating task has been deleted ([ClearML GitHub issue #1299](https://github.com/clearml/clearml/issues/1299)) +* Fix UI pipeline DAG display to be "bottom-up" so all final steps appear in the bottom of the diagram ([ClearML Web GitHub PR #86](https://github.com/clearml/clearml-web/pull/86)) +* Fix invalid mongodb connection string if `CLEARML_MONGODB_SERVICE_CONNECTION_STRING` is specified ([ClearML Server GitHub issue #252](https://github.com/clearml/clearml-server/issues/252)) +* Fix auto-refresh modifies UI experiment debug sample view ([ClearML GitHub issue #1259](https://github.com/clearml/clearml/issues/1259)) +* Fix UI image plot retrieval from fileserver missing authentication cookie ([ClearML GitHub issue #1331](https://github.com/clearml/clearml/issues/1331)) * Fix pipeline run version not set when re-executed via the UI * Fix metric and hyperparameter group string not searchable in UI table * Fix UI model hidden plot selection does not persist upon refresh diff --git a/docs/release_notes/clearml_server/open_source/ver_1_2.md b/docs/release_notes/clearml_server/open_source/ver_1_2.md index d4756af9..8bf17841 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_2.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_2.md @@ -6,7 +6,7 @@ title: Version 1.2 **New Features and Improvements** * Add functionality to "Abort all children" UI action for controller and optimizer tasks -* Add parameter search to task configuration UI section ([ClearML GitHub issue #467](https://github.com/allegroai/clearml/issues/467)) +* Add parameter search to task configuration UI section ([ClearML GitHub issue #467](https://github.com/clearml/clearml/issues/467)) * Add tag exclusion filters in UI experiment and model tables * Add "Clear Filters" functionality button to UI table filters * Improve full screen scalar graph to display all data points @@ -29,12 +29,12 @@ title: Version 1.2 **Bug Fixes** -* Fix UI debug sample viewer ignoring metric filter on auto-refresh ([ClearML GitHub issue #512](https://github.com/allegroai/clearml/issues/512)) -* Fix "Wall time" x-axis option in UI plots slowing down web app ([ClearML GitHub issue #441](https://github.com/allegroai/clearml/issues/441)) -* Fix UI table plot titles don't display their iteration number ([ClearML GitHub issue #474](https://github.com/allegroai/clearml/issues/474)) -* Fix auto-refresh resets vertical scroll in UI debug samples ([ClearML GitHub issue #474](https://github.com/allegroai/clearml/issues/474)) -* Fix manually run experiments cannot be edited in UI after being reset ([ClearML GitHub issue #449](https://github.com/allegroai/clearml/issues/449)) -* Fix UI Workers & Queues and Experiment Table pages display mismatching experiment runtime values ([ClearML Server GitHub issue #100](https://github.com/allegroai/clearml-server/issues/100)) +* Fix UI debug sample viewer ignoring metric filter on auto-refresh ([ClearML GitHub issue #512](https://github.com/clearml/clearml/issues/512)) +* Fix "Wall time" x-axis option in UI plots slowing down web app ([ClearML GitHub issue #441](https://github.com/clearml/clearml/issues/441)) +* Fix UI table plot titles don't display their iteration number ([ClearML GitHub issue #474](https://github.com/clearml/clearml/issues/474)) +* Fix auto-refresh resets vertical scroll in UI debug samples ([ClearML GitHub issue #474](https://github.com/clearml/clearml/issues/474)) +* Fix manually run experiments cannot be edited in UI after being reset ([ClearML GitHub issue #449](https://github.com/clearml/clearml/issues/449)) +* Fix UI Workers & Queues and Experiment Table pages display mismatching experiment runtime values ([ClearML Server GitHub issue #100](https://github.com/clearml/clearml-server/issues/100)) * Fix UI plots sometimes disappear after task execution completion * Fix UI experiment plots don't display some plots on a http server * Fix failure to add Azure credentials through UI Settings page by new users diff --git a/docs/release_notes/clearml_server/open_source/ver_1_3.md b/docs/release_notes/clearml_server/open_source/ver_1_3.md index fafb7f55..afec85be 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_3.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_3.md @@ -5,7 +5,7 @@ title: Version 1.3 ### ClearML Server 1.3.1 **Bug Fixes** -* Fix scalar graph smoothing ([ClearML Server GitHub issue #127](https://github.com/allegroai/clearml-server/issues/127)) +* Fix scalar graph smoothing ([ClearML Server GitHub issue #127](https://github.com/clearml/clearml-server/issues/127)) ### ClearML Server 1.3.0 @@ -14,9 +14,9 @@ title: Version 1.3 * Pipelines statistics dashboard * Pipeline run history: View pipeline structure, run configuration and outputs * Pipeline control: Abort/Run pipelines from UI -* Add support for exporting UI table plots in CSV formats ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400)) -* Improve UI breadcrumbs use of screen real-estate ([ClearML GitHub issue #529](https://github.com/allegroai/clearml/issues/529)) -* Add UI adherence to user specification of plot colors and labels ([ClearML GitHub issue #518](https://github.com/allegroai/clearml/issues/518)) +* Add support for exporting UI table plots in CSV formats ([ClearML GitHub issue #400](https://github.com/clearml/clearml/issues/400)) +* Improve UI breadcrumbs use of screen real-estate ([ClearML GitHub issue #529](https://github.com/clearml/clearml/issues/529)) +* Add UI adherence to user specification of plot colors and labels ([ClearML GitHub issue #518](https://github.com/clearml/clearml/issues/518)) * Add Model metadata to UI model information (Model table and details page) * Add multi-selection in UI experiment info-panel mode * Add labels to workspace credentials @@ -25,7 +25,7 @@ title: Version 1.3 * Add Previous/Next controls to UI text area search bars **Bug Fixes** -* Fix UI plots do not display Plotly `imshow texttemplate` ([ClearML GitHub issue #538](https://github.com/allegroai/clearml/issues/538)) +* Fix UI plots do not display Plotly `imshow texttemplate` ([ClearML GitHub issue #538](https://github.com/clearml/clearml/issues/538)) * Fix clicking F5 clears storage credentials from UI settings page * Fix UI plots "Wall time" horizontal axis option is not working * Fix disappearing UI full-screen scalar plot diff --git a/docs/release_notes/clearml_server/open_source/ver_1_4.md b/docs/release_notes/clearml_server/open_source/ver_1_4.md index bf3af082..62da14ca 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_4.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_4.md @@ -6,28 +6,28 @@ title: Version 1.4 **New Features and Improvements** -- Allow serving with a URL path prefix ([ClearML Server GitHub issue #119](https://github.com/allegroai/clearml-server/issues/119)) -- Allow overriding clearml web git url on build ([ClearML Server GitHub Pull Request #122](https://github.com/allegroai/clearml-server/pull/122)) -- Ensure agent-services waits for API server to be ready ([ClearML Server GitHub issue #113](https://github.com/allegroai/clearml-server/issues/113)) -- Add filter to UI experiment table "Project" column ([ClearML GitHub issue #529](https://github.com/allegroai/clearml/issues/529)) +- Allow serving with a URL path prefix ([ClearML Server GitHub issue #119](https://github.com/clearml/clearml-server/issues/119)) +- Allow overriding clearml web git url on build ([ClearML Server GitHub Pull Request #122](https://github.com/clearml/clearml-server/pull/122)) +- Ensure agent-services waits for API server to be ready ([ClearML Server GitHub issue #113](https://github.com/clearml/clearml-server/issues/113)) +- Add filter to UI experiment table "Project" column ([ClearML GitHub issue #529](https://github.com/clearml/clearml/issues/529)) - Add filters to UI model table custom metadata columns - Improve UI table object selection. Add buttons and context menu actions to switch between table and info panel mode -- Add task detail header to UI experiment comparison debug image section addressed in ([ClearML GitHub issue #81](https://github.com/allegroai/clearml/issues/81) ([comment](https://github.com/allegroai/clearml/issues/81#issuecomment-996841658))) -- Improve series name truncation behavior in UI scalar plots ([ClearML GitHub issue #562](https://github.com/allegroai/clearml/issues/562)) +- Add task detail header to UI experiment comparison debug image section addressed in ([ClearML GitHub issue #81](https://github.com/clearml/clearml/issues/81) ([comment](https://github.com/clearml/clearml/issues/81#issuecomment-996841658))) +- Improve series name truncation behavior in UI scalar plots ([ClearML GitHub issue #562](https://github.com/clearml/clearml/issues/562)) **Bug Fixes** -- Fix server limiting image and artifact upload size ([ClearML GitHub issue #606](https://github.com/allegroai/clearml/issues/606)) -- Fix server unnecessarily displays deletion error message ([ClearML Server GitHub issue #112](https://github.com/allegroai/clearml-server/issues/112)) -- Fix UI experiment comparison sections overly wide for many tag experiments ([ClearML GitHub issue #594](https://github.com/allegroai/clearml/issues/594)) +- Fix server limiting image and artifact upload size ([ClearML GitHub issue #606](https://github.com/clearml/clearml/issues/606)) +- Fix server unnecessarily displays deletion error message ([ClearML Server GitHub issue #112](https://github.com/clearml/clearml-server/issues/112)) +- Fix UI experiment comparison sections overly wide for many tag experiments ([ClearML GitHub issue #594](https://github.com/clearml/clearml/issues/594)) - Fix model name display in UI pipeline run info panel - Fix UI pipeline run info panel missing artifacts and models sections - Fix UI pipeline run info panel displays unneeded resource utilization metrics - Fix UI project cards missing task summary values - Fix pipeline tag colors can't be changed in UI pipeline page -- Fix pipeline run UI page not displaying pipeline steps ([ClearML GitHub issue #618](https://github.com/allegroai/clearml/issues/618)) +- Fix pipeline run UI page not displaying pipeline steps ([ClearML GitHub issue #618](https://github.com/clearml/clearml/issues/618)) - Fix breadcrumb links don't navigate anywhere - Fix horizontal scroll in UI experiment table causes column headers to jump - Fix removing UI table column creates a blank column -- Fix internal server error (500) on `events.get_scalar_metrics_and_variants` ([ClearML Server GitHub issue #120](https://github.com/allegroai/clearml-server/issues/120)) +- Fix internal server error (500) on `events.get_scalar_metrics_and_variants` ([ClearML Server GitHub issue #120](https://github.com/clearml/clearml-server/issues/120)) diff --git a/docs/release_notes/clearml_server/open_source/ver_1_5.md b/docs/release_notes/clearml_server/open_source/ver_1_5.md index 8183a714..12ff9728 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_5.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_5.md @@ -9,8 +9,8 @@ title: Version 1.5 * Redesign experiment status banner * Display all experiment tabs in single line * Move experiment action buttons to the top right -* Add UI adherence to user specified plot dimensions ([ClearML GitHub issue #587](https://github.com/allegroai/clearml/issues/587)) -* Add row highlight to experiment textual comparison UI ([ClearML GitHub issue #581](https://github.com/allegroai/clearml/issues/581)) +* Add UI adherence to user specified plot dimensions ([ClearML GitHub issue #587](https://github.com/clearml/clearml/issues/587)) +* Add row highlight to experiment textual comparison UI ([ClearML GitHub issue #581](https://github.com/clearml/clearml/issues/581)) * Add UI "Delete" action for Pipelines * Add UI indicator for task and pipeline progress * Add UI API credentials label "Edit" action @@ -19,8 +19,8 @@ title: Version 1.5 * Add pipeline results to UI dashboard search **Bug Fixes** -* Fix UI experiment debug samples disappearing after refresh ([ClearML Server GitHub issue #136](https://github.com/allegroai/clearml-server/issues/136)) -* Fix deleting tasks sometimes raises errors ([ClearML GitHub issue #632](https://github.com/allegroai/clearml/issues/632)) +* Fix UI experiment debug samples disappearing after refresh ([ClearML Server GitHub issue #136](https://github.com/clearml/clearml-server/issues/136)) +* Fix deleting tasks sometimes raises errors ([ClearML GitHub issue #632](https://github.com/clearml/clearml/issues/632)) * Fix only partial task log shown when running on ES with multiple shards * Fix move task to trash is not thread-safe * Fix UI Project overview metric snapshot not showing diff --git a/docs/release_notes/clearml_server/open_source/ver_1_6.md b/docs/release_notes/clearml_server/open_source/ver_1_6.md index 267dfeda..e0e779d2 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_6.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_6.md @@ -5,14 +5,14 @@ title: Version 1.6 ### ClearML Server 1.6.0 **New Features and Improvements** * New ClearML Datasets UI pages for tracking dataset versions and exploring version lineage and contents -* Add history navigation to experiments plots UI page ([ClearML GitHub issues #81](https://github.com/allegroai/clearml/issues/81) and [#255](https://github.com/allegroai/clearml/issues/255)): +* Add history navigation to experiments plots UI page ([ClearML GitHub issues #81](https://github.com/clearml/clearml/issues/81) and [#255](https://github.com/clearml/clearml/issues/255)): * Plots page shows last reported plot for each metric/variation combination * Single plot view provides history navigation slider * Add single value scalar reporting: Single value scalars are aggregated into a summary table in the experiment's scalars - UI page ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400)) -* Add "show hidden projects" control ([ClearML GitHub issue #694](https://github.com/allegroai/clearml/issues/694)) + UI page ([ClearML GitHub issue #400](https://github.com/clearml/clearml/issues/400)) +* Add "show hidden projects" control ([ClearML GitHub issue #694](https://github.com/clearml/clearml/issues/694)) * Improve UI projects page setting persistence - User's last chosen settings for recent/name sorting and team/personal - filter is saved ([ClearML GitHub issue #353](https://github.com/allegroai/clearml/issues/353)) + filter is saved ([ClearML GitHub issue #353](https://github.com/clearml/clearml/issues/353)) * Improve UI presentation for object IDs: * Display shortened ID next to task name * Display full ID in tooltip @@ -26,9 +26,9 @@ title: Version 1.6 * Maintain UI viewing mode when restoring archived items **Bug Fixes** -* Fix experiment selection with 'Shift' key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/allegroai/clearml-server/issues/139)) -* Fix UI search error message interferes with inputting search query ([ClearML Server GitHub issue #138](https://github.com/allegroai/clearml-server/issues/138)) -* Fix refresh breaks UI breadcrumbs ClearML Server ([ClearML Server GitHub issue #142](https://github.com/allegroai/clearml-server/issues/142)) +* Fix experiment selection with 'Shift' key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/clearml/clearml-server/issues/139)) +* Fix UI search error message interferes with inputting search query ([ClearML Server GitHub issue #138](https://github.com/clearml/clearml-server/issues/138)) +* Fix refresh breaks UI breadcrumbs ClearML Server ([ClearML Server GitHub issue #142](https://github.com/clearml/clearml-server/issues/142)) * Fix UI Workers and Queues page displays incorrect queue metrics * Fix failure to publish models whose generating task was reset * Fix listed models in UI pipeline run info panel doesn't link to model diff --git a/docs/release_notes/clearml_server/open_source/ver_1_7.md b/docs/release_notes/clearml_server/open_source/ver_1_7.md index d376cd11..2af9535c 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_7.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_7.md @@ -4,10 +4,10 @@ title: Version 1.7 ### ClearML Server 1.7.0 **New Features and Improvements** -* Add “Sync comparison” to UI experiment debug samples comparison: Control metric/iteration for all compared experiments ([ClearML GitHub issue #691](https://github.com/allegroai/clearml/issues/691)) -* Support serving UI from a non-root path of the ClearML Server ([ClearML Helm Charts issue #101](https://github.com/allegroai/clearml-helm-charts/issues/101) and [ClearML Server issue #135](https://github.com/allegroai/clearml-server/issues/135)) -* Add UI option for hiding “secret” experiment container arguments ([ClearML Server GitHub issue #146](https://github.com/allegroai/clearml-server/issues/146)) -* Add UI tables switch to detail mode through double-click ([ClearML Server GitHub issue #134](https://github.com/allegroai/clearml-server/issues/134)) +* Add “Sync comparison” to UI experiment debug samples comparison: Control metric/iteration for all compared experiments ([ClearML GitHub issue #691](https://github.com/clearml/clearml/issues/691)) +* Support serving UI from a non-root path of the ClearML Server ([ClearML Helm Charts issue #101](https://github.com/clearml/clearml-helm-charts/issues/101) and [ClearML Server issue #135](https://github.com/clearml/clearml-server/issues/135)) +* Add UI option for hiding “secret” experiment container arguments ([ClearML Server GitHub issue #146](https://github.com/clearml/clearml-server/issues/146)) +* Add UI tables switch to detail mode through double-click ([ClearML Server GitHub issue #134](https://github.com/clearml/clearml-server/issues/134)) * Add customizable user activity timeout for UI logout * Add UI navigation from experiment comparison back to originating experiment table * Improve UI scalar comparison graph configuration - Persist user's choice for viewing graph data. @@ -15,8 +15,8 @@ title: Version 1.7 * Add dataset description to UI dataset information display **Bug Fixes** -* Fix UI experiment Hyperparameter tab's inefficient use of screen real estate ([ClearML GitHub issue #705](https://github.com/allegroai/clearml/issues/705)) -* Fix navigating to an archived experiment's configuration causes UI to return to non-archived view ([ClearML Server GitHub issue #148](https://github.com/allegroai/clearml-server/issues/148)) +* Fix UI experiment Hyperparameter tab's inefficient use of screen real estate ([ClearML GitHub issue #705](https://github.com/clearml/clearml/issues/705)) +* Fix navigating to an archived experiment's configuration causes UI to return to non-archived view ([ClearML Server GitHub issue #148](https://github.com/clearml/clearml-server/issues/148)) * Fix metric storage failure when large amount of metrics logged * Fix UI plots downloaded as PNGs don't contain legends * Fix UI plot colors can't be changed when colors are specified in code diff --git a/docs/release_notes/clearml_server/open_source/ver_1_8.md b/docs/release_notes/clearml_server/open_source/ver_1_8.md index 83b0194c..cf4db4ce 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_8.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_8.md @@ -5,18 +5,18 @@ title: Version 1.8 ### ClearML Server 1.8.0 **New Features and Improvements** -* Support `/` delimited project names for on-the-fly nested project creation in UI operations ([ClearML Server GitHub issue #151](https://github.com/allegroai/clearml-server/issues/151)) +* Support `/` delimited project names for on-the-fly nested project creation in UI operations ([ClearML Server GitHub issue #151](https://github.com/clearml/clearml-server/issues/151)) * Add keyboard shortcut iteration navigation in full screen view of debug samples and plots -* Add "Don't show again" option for UI notification when archiving tasks ([ClearML Server GitHub issue #149](https://github.com/allegroai/clearml-server/issues/149)) +* Add "Don't show again" option for UI notification when archiving tasks ([ClearML Server GitHub issue #149](https://github.com/clearml/clearml-server/issues/149)) * Add task reset/delete modal automatically closes when no additional information is provided * Add parent project name in UI dataset and pipeline cards * Move UI plot legends to bottom of plot **Bug Fixes** -* Fix task artifacts not deleted from file server when task deleted via UI ([ClearML GitHub issue #801](https://github.com/allegroai/clearml/issues/801)) -* Fix deleting pipeline projects via UI fails ([ClearML GitHub issue #798](https://github.com/allegroai/clearml/issues/798)) -* Fix multiple selection with `Shift` key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/allegroai/clearml-server/issues/139)) -* Fix disappearing user filter in UI "All Experiments" table ([ClearML Web GitHub issue #33](https://github.com/allegroai/clearml-web/issues/33)) +* Fix task artifacts not deleted from file server when task deleted via UI ([ClearML GitHub issue #801](https://github.com/clearml/clearml/issues/801)) +* Fix deleting pipeline projects via UI fails ([ClearML GitHub issue #798](https://github.com/clearml/clearml/issues/798)) +* Fix multiple selection with `Shift` key in UI experiment table not working ([ClearML Server GitHub issue #139](https://github.com/clearml/clearml-server/issues/139)) +* Fix disappearing user filter in UI "All Experiments" table ([ClearML Web GitHub issue #33](https://github.com/clearml/clearml-web/issues/33)) * Fix UI Dashboard missing My Work / Team Work filter * Fix navigating to Scalars and Plots tabs in UI experiment comparison freezes browser * Fix project stats do not exclude dataset and annotation tasks diff --git a/docs/release_notes/clearml_server/open_source/ver_1_9.md b/docs/release_notes/clearml_server/open_source/ver_1_9.md index 07bcb60d..b677d6dd 100644 --- a/docs/release_notes/clearml_server/open_source/ver_1_9.md +++ b/docs/release_notes/clearml_server/open_source/ver_1_9.md @@ -5,8 +5,8 @@ title: Version 1.9 ### ClearML Server 1.9.2 **Bug Fixes** -* Fix "Unable to load image" error in UI experiment debug samples after entering storage credentials ([ClearML Web GitHub issue #46](https://github.com/allegroai/clearml-web/issues/46)) -* Fix incorrect default project name displayed in UI "Clone Experiment" modal ([ClearML Server GitHub issue #171](https://github.com/allegroai/clearml-server/issues/171)) +* Fix "Unable to load image" error in UI experiment debug samples after entering storage credentials ([ClearML Web GitHub issue #46](https://github.com/clearml/clearml-web/issues/46)) +* Fix incorrect default project name displayed in UI "Clone Experiment" modal ([ClearML Server GitHub issue #171](https://github.com/clearml/clearml-server/issues/171)) * Fix UI experiment debug sample viewer iteration slider not working * Fix UI experiment console log jumps backwards unnecessarily on refresh * Fix UI published report is not expandable @@ -20,7 +20,7 @@ title: Version 1.9 * Add in-app markdown guide for UI reports **Bug Fixes** -* Fix can't generate new pipeline runs from UI ([ClearML Server GitHub issue #169](https://github.com/allegroai/clearml-server/issues/169)) +* Fix can't generate new pipeline runs from UI ([ClearML Server GitHub issue #169](https://github.com/clearml/clearml-server/issues/169)) * Fix plot legend texts overlap in UI reports * Fix UI embedded plot colors are sometimes obscure * Fix `ctrl z` doesn't undo codeblocks in UI reports @@ -29,14 +29,14 @@ title: Version 1.9 ### ClearML Server 1.9.0 **New Features and Improvements** -* New ClearML Reports UI: Create and share rich MarkDown documents supporting embeddable online content ([ClearML GitHub issue #839](https://github.com/allegroai/clearml/issues/839)) -* Add user option to not show example content in the UI ([ClearML GitHub issue #774](https://github.com/allegroai/clearml/issues/774)) +* New ClearML Reports UI: Create and share rich MarkDown documents supporting embeddable online content ([ClearML GitHub issue #839](https://github.com/clearml/clearml/issues/839)) +* Add user option to not show example content in the UI ([ClearML GitHub issue #774](https://github.com/clearml/clearml/issues/774)) * Add tag filter to UI Pipeline, Dataset, and Hyper-Dataset pages * Extend UI experiment comparison up to 100 experiments * Allow project default output destination S3 URLs to include dots (`.`) and hyphens (`-`) **Bug Fixes** -* Fix long experiment names break UI experiment comparison scalar plot display ([ClearML GitHub issue #166](https://github.com/allegroai/clearml-server/issues/166)) +* Fix long experiment names break UI experiment comparison scalar plot display ([ClearML GitHub issue #166](https://github.com/clearml/clearml-server/issues/166)) * Fix plot legend definitions set by SDK don't override UI default settings * Fix comparing experiments with read-only components causes error * Fix UI model link doesn't preserve double spaces, breaking the URI diff --git a/docs/release_notes/clearml_server/open_source/ver_2_0.md b/docs/release_notes/clearml_server/open_source/ver_2_0.md index a6558899..2ca3a97e 100644 --- a/docs/release_notes/clearml_server/open_source/ver_2_0.md +++ b/docs/release_notes/clearml_server/open_source/ver_2_0.md @@ -10,11 +10,11 @@ MongoDB major version was upgraded from `v5.x` to `6.x`. Please note that if you Upgrading to ClearML Server v1.17 from a previous version: * If using `docker-compose`, use the following files: - * [docker-compose file](https://github.com/allegroai/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose.yml) - * [docker-compose file for Windows](https://github.com/allegroai/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose-win10.yml) + * [docker-compose file](https://github.com/clearml/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose.yml) + * [docker-compose file for Windows](https://github.com/clearml/clearml-server/blob/2976ce69cc91550a3614996e8a8d8cd799af2efd/upgrade/1_17_to_2_0/docker-compose-win10.yml) **New Features** -* New look and feel: Full light/dark themes ([ClearML GitHub issue #1297](https://github.com/allegroai/clearml/issues/1297)) +* New look and feel: Full light/dark themes ([ClearML GitHub issue #1297](https://github.com/clearml/clearml/issues/1297)) * New UI task creation options * Support bash as well as Python scripts * Support file upload diff --git a/docs/release_notes/clearml_serving/ver_1_2.md b/docs/release_notes/clearml_serving/ver_1_2.md index 0601e69d..2390462d 100644 --- a/docs/release_notes/clearml_serving/ver_1_2.md +++ b/docs/release_notes/clearml_serving/ver_1_2.md @@ -8,7 +8,7 @@ title: Version 1.2 * Improve GPU Performance, 50%-300% improvement over vanilla Triton * Improve performance on CPU, optimize uvloop + multi-processing * Add Hugging Face Transformer example -* Add binary input support ([ClearML Serving PR #37](https://github.com/allegroai/clearml-serving/pull/37) ) +* Add binary input support ([ClearML Serving PR #37](https://github.com/clearml/clearml-serving/pull/37) ) **Bug Fix** * stdout/stderr in inference service not logging to dedicated Task diff --git a/docs/release_notes/clearml_serving/ver_1_3.md b/docs/release_notes/clearml_serving/ver_1_3.md index 5a70b818..5f915a38 100644 --- a/docs/release_notes/clearml_serving/ver_1_3.md +++ b/docs/release_notes/clearml_serving/ver_1_3.md @@ -4,9 +4,9 @@ title: Version 1.3 ### ClearML Serving 1.3.1 **New Features and Bug Fixes** -* Add missing `await` ([ClearML Serving GitHub PR #55](https://github.com/allegroai/clearml-serving/pull/55)) -* Add traceback for failing to load `preprocess` class ([ClearML Serving GitHub issue #57](https://github.com/allegroai/clearml-serving/issues/57)) -* Fix Triton `config.pbtxt` for missing values or colliding specifications ([ClearML Serving GitHub issue #62](https://github.com/allegroai/clearml-serving/issues/62)) +* Add missing `await` ([ClearML Serving GitHub PR #55](https://github.com/clearml/clearml-serving/pull/55)) +* Add traceback for failing to load `preprocess` class ([ClearML Serving GitHub issue #57](https://github.com/clearml/clearml-serving/issues/57)) +* Fix Triton `config.pbtxt` for missing values or colliding specifications ([ClearML Serving GitHub issue #62](https://github.com/clearml/clearml-serving/issues/62)) * Add safer code for pulling from Kafka * Add `str` type to Triton type conversion * Fix ignore auto-detected platform when passing `config.pbtxt` with platform entry @@ -18,10 +18,10 @@ title: Version 1.3 ### ClearML Serving 1.3.0 **New Features and Improvements** * 20% Overall performance increase -* gRPC channel configuration ([ClearML Serving GitHub issue #49](https://github.com/allegroai/clearml-serving/issues/49)) +* gRPC channel configuration ([ClearML Serving GitHub issue #49](https://github.com/clearml/clearml-serving/issues/49)) * Hugging Face Transformer example **Bug fixes** -* Fix NumPy compatibility ([ClearML Serving GitHub issue #47](https://github.com/allegroai/clearml-serving/issues/47) and [#46](https://github.com/allegroai/clearml-serving/issues/46)) -* Fix Triton examples ([ClearML Serving GitHub issue #48](https://github.com/allegroai/clearml-serving/issues/48)) -* Add storage environment variables ([ClearML Serving GitHub PR #45](https://github.com/allegroai/clearml-serving/pull/45)) \ No newline at end of file +* Fix NumPy compatibility ([ClearML Serving GitHub issue #47](https://github.com/clearml/clearml-serving/issues/47) and [#46](https://github.com/clearml/clearml-serving/issues/46)) +* Fix Triton examples ([ClearML Serving GitHub issue #48](https://github.com/clearml/clearml-serving/issues/48)) +* Add storage environment variables ([ClearML Serving GitHub PR #45](https://github.com/clearml/clearml-serving/pull/45)) \ No newline at end of file diff --git a/docs/release_notes/sdk/open_source/ver_0_10.md b/docs/release_notes/sdk/open_source/ver_0_10.md index e8824f6b..2f6662ce 100644 --- a/docs/release_notes/sdk/open_source/ver_0_10.md +++ b/docs/release_notes/sdk/open_source/ver_0_10.md @@ -65,12 +65,12 @@ title: Version 0.10 **Features and Bug Fixes** -* Add scikit-learn support (load/store using joblib) ([GitHub Issue #20](https://github.com/allegroai/trains/issues/20)). -* Add xgboost support ([GitHub Issue #10](https://github.com/allegroai/trains/issues/10)). -* Add loguru support ([GitHub Issue #29](https://github.com/allegroai/trains/issues/29)). -* Add sub-domain support [trains.conf](https://github.com/allegroai/trains/blob/master/docs/trains.conf#L3) ([GitHub Issue #27](https://github.com/allegroai/trains/issues/27)). +* Add scikit-learn support (load/store using joblib) ([GitHub Issue #20](https://github.com/clearml/clearml/issues/20)). +* Add xgboost support ([GitHub Issue #10](https://github.com/clearml/clearml/issues/10)). +* Add loguru support ([GitHub Issue #29](https://github.com/clearml/clearml/issues/29)). +* Add sub-domain support [trains.conf](https://github.com/clearml/clearml/blob/master/docs/trains.conf#L3) ([GitHub Issue #27](https://github.com/clearml/clearml/issues/27)). * Fix sub-process support. -* Fix multiple TensorBoard writers ([GitHub Issue #26](https://github.com/allegroai/trains/issues/26)). +* Fix multiple TensorBoard writers ([GitHub Issue #26](https://github.com/clearml/clearml/issues/26)). ### Trains 0.10.2 @@ -78,7 +78,7 @@ title: Version 0.10 * Add Matplotlib SVG support. * Add Seaborn support. -* Add `TRAINS_LOG_ENVIRONMENT` environment logging ([GitHub trains Issue 17](https://github.com/allegroai/trains/issues/17#issuecomment-507398767)). +* Add `TRAINS_LOG_ENVIRONMENT` environment logging ([GitHub trains Issue 17](https://github.com/clearml/clearml/issues/17#issuecomment-507398767)). * Add Microsoft Azure notebook support. * Add Google Colab support. * Fix TensorBoard RGB channel order. diff --git a/docs/release_notes/sdk/open_source/ver_0_11.md b/docs/release_notes/sdk/open_source/ver_0_11.md index 648b9aa2..38934471 100644 --- a/docs/release_notes/sdk/open_source/ver_0_11.md +++ b/docs/release_notes/sdk/open_source/ver_0_11.md @@ -12,7 +12,7 @@ title: Version 0.11 **Features and Bug Fixes** * Resource-Monitor will only monitor active GPU devices based on environment variables: `NVIDIA_VISIBLE_DEVICES` or `CUDA_VISIBLE_DEVICES`. -* Fix issue ([GitHub Issue #48](https://github.com/allegroai/trains/issues/48)). +* Fix issue ([GitHub Issue #48](https://github.com/clearml/clearml/issues/48)). ### Trains 0.11.2 @@ -41,7 +41,7 @@ title: Version 0.11 **Features and Bug Fixes** * Full artifacts support (supported by trains-server >= 0.11.0). -* Artifacts include, Pandas.DataFrame, Numpy, PIL Image, local files, and local folder / wildcard ([example](https://github.com/allegroai/trains/blob/master/examples/reporting/artifacts.py)). +* Artifacts include, Pandas.DataFrame, Numpy, PIL Image, local files, and local folder / wildcard ([example](https://github.com/clearml/clearml/blob/master/examples/reporting/artifacts.py)). * Artifacts support for folder / wildcard, selected files will be zipped and uploaded. * Resource monitoring, remove sensor reading failure warnings. diff --git a/docs/release_notes/sdk/open_source/ver_0_12.md b/docs/release_notes/sdk/open_source/ver_0_12.md index 7dd0442b..4d8be1fa 100644 --- a/docs/release_notes/sdk/open_source/ver_0_12.md +++ b/docs/release_notes/sdk/open_source/ver_0_12.md @@ -10,11 +10,11 @@ title: Version 0.12 **Features and Bug Fixes** -* Add `default_output_uri` feature ([GitHub Issue #57](https://github.com/allegroai/trains/issues/57)). +* Add `default_output_uri` feature ([GitHub Issue #57](https://github.com/clearml/clearml/issues/57)). * Improve `trains-init` configuration wizard. * Improve argparser binding. -* Fix artifact support in data pipeline ([GitHub Issue #63](https://github.com/allegroai/trains/issues/63)). -* Fix threading issue while querying multiple experiments ([GitHub Issue #64](https://github.com/allegroai/trains/issues/64)). +* Fix artifact support in data pipeline ([GitHub Issue #63](https://github.com/clearml/clearml/issues/63)). +* Fix threading issue while querying multiple experiments ([GitHub Issue #64](https://github.com/clearml/clearml/issues/64)). * Fix uploading large files over slow HTTP connections. * Fix support for Git versions < 2. @@ -29,15 +29,15 @@ Do not reuse an experiment with artifacts. * Add `Task.connect_configuration`, connect configuration file (or configuration dictionary including nesting support). * Add `Task.connect_label_enumeration`, store model detection classes together with the model and experiment. -* Edit and override configuration file (or dictionary) content when executing with [trains-agent](https://github.com/allegroai/trains-agent). +* Edit and override configuration file (or dictionary) content when executing with [trains-agent](https://github.com/clearml/clearml-agent). * `Task.connect` automatically supports nested dictionaries (e.g. param / nested / key), including overriding values when - running with [trains-agent](https://github.com/allegroai/trains-agent). + running with [trains-agent](https://github.com/clearml/clearml-agent). * Add `Artifact.get`, download and load an artifact. * Add `Task.get_tasks`, retrieve previous experiments, get metrics, rename, and other uses. * Improve Windows support. * Improve Minio support. * Fix Python2 support. -* Fix Issue ([GitHub Issue #56](https://github.com/allegroai/trains/issues/56)). +* Fix Issue ([GitHub Issue #56](https://github.com/clearml/clearml/issues/56)). ### Trains 0.12.0 @@ -45,7 +45,7 @@ Do not reuse an experiment with artifacts. **Features and Bug Fixes** * TRAINS Agent support (Full DevOps support). -* Add automation and orchestration ([examples/automation](https://github.com/allegroai/trains/tree/master/examples/automation)). +* Add automation and orchestration ([examples/automation](https://github.com/clearml/clearml/tree/master/examples/automation)). Supports TRAINS-server v0.12 or above. * Add Logger x/y/z axis title for: report_surface / report_confusion_matrix / report_scatter3d / report_scatter2d / report_histogram. * Add support for TensorFlow 2.0. diff --git a/docs/release_notes/sdk/open_source/ver_0_13.md b/docs/release_notes/sdk/open_source/ver_0_13.md index e831be9d..88a00e35 100644 --- a/docs/release_notes/sdk/open_source/ver_0_13.md +++ b/docs/release_notes/sdk/open_source/ver_0_13.md @@ -47,7 +47,7 @@ title: Version 0.13 * Add support for SageMaker. * Improve configuration wizard. * Try to make sure TensorBoard is available when using torch. -* Do not store keras model network design if it cannot be serialized ([GitHub Issue #72](https://github.com/allegroai/trains/issues/72)). +* Do not store keras model network design if it cannot be serialized ([GitHub Issue #72](https://github.com/clearml/clearml/issues/72)). * Fix matplotlib binding support. @@ -55,21 +55,21 @@ title: Version 0.13 **Features and Bug Fixes** -* Add support for [trains-server](https://github.com/allegroai/trains-server) v0.13.0. +* Add support for [trains-server](https://github.com/clearml/clearml-server) v0.13.0. * Add support for nested (non-main) tasks. * Add warning when automatic argument parser binding cannot be turned off. * Add `Task.upload_artifact` support for external URLs (pre-uploaded). -* Add support for special characters in hyperparameter keys (white-spaces, `.` and `$`) ([GitHub Issue #69](https://github.com/allegroai/trains/issues/69)). +* Add support for special characters in hyperparameter keys (white-spaces, `.` and `$`) ([GitHub Issue #69](https://github.com/clearml/clearml/issues/69)). * Add support for PyTorch `.pt` model files. -* Calculate data-audit artifact uniqueness by user-criteria ([GitHub Issue #45](https://github.com/allegroai/trains/issues/45)). -* Use an environment variable for setting a default docker image ([GitHub Issue #58](https://github.com/allegroai/trains/issues/58)). +* Calculate data-audit artifact uniqueness by user-criteria ([GitHub Issue #45](https://github.com/clearml/clearml/issues/45)). +* Use an environment variable for setting a default docker image ([GitHub Issue #58](https://github.com/clearml/clearml/issues/58)). * Improve `trains-init` configuration wizard. * Update examples for new joblib versions. * Update jupyter example to TensorFlow 2. * Fix task clone to copy only input artifacts. * Fix matplotlib import binding when using `Agg` backend. -* Fix `ProxyDictPreWrite` and `ProxyDictPostWrite` so they can be pickled correctly ([GitHub Issue #72](https://github.com/allegroai/trains/issues/72)). +* Fix `ProxyDictPreWrite` and `ProxyDictPostWrite` so they can be pickled correctly ([GitHub Issue #72](https://github.com/clearml/clearml/issues/72)). * Fix requests issue in Python 2.7 that can cause a deadlock when importing netrc. -* Fix argparser binding sub-parser and type casting support ([GitHub Issue #74](https://github.com/allegroai/trains/issues/74)). +* Fix argparser binding sub-parser and type casting support ([GitHub Issue #74](https://github.com/clearml/clearml/issues/74)). * Fix argparser binding Python 2.7 unicode handling. * Fix unsynced connected hyperparameters overridden during remote execution. diff --git a/docs/release_notes/sdk/open_source/ver_0_14.md b/docs/release_notes/sdk/open_source/ver_0_14.md index 8ac6796c..e03f6916 100644 --- a/docs/release_notes/sdk/open_source/ver_0_14.md +++ b/docs/release_notes/sdk/open_source/ver_0_14.md @@ -15,7 +15,7 @@ title: Version 0.14 * Add the `Logger.report_histogram` method `mode` parameter. The values are `group` (the default), `stack`, and `relative`. * Add [PEP610](https://www.python.org/dev/peps/pep-0610/) support for the new `pip` version 20.1 and newer Git reference - feature (see [pypa/pip#609](https://github.com/pypa/pip/issues/609)) ([GitHub Issue #62](https://github.com/allegroai/trains/issues/62)). + feature (see [pypa/pip#609](https://github.com/pypa/pip/issues/609)) ([GitHub Issue #62](https://github.com/clearml/clearml/issues/62)). * Improve support for PyTorch Ignite integration. * Support `#` and `?` in uploaded image file names. @@ -29,8 +29,8 @@ title: Version 0.14 ### Trains 0.14.2 **Features** -* Add media (audio) support for both `Logger` and TensorBoard binding using `Logger.report_media` ([GitHub Issue #120](https://github.com/allegroai/trains/issues/120)). -* Add [autokeras example](https://github.com/allegroai/trains/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) ([GitHub Issue #125](https://github.com/allegroai/trains/issues/125)). +* Add media (audio) support for both `Logger` and TensorBoard binding using `Logger.report_media` ([GitHub Issue #120](https://github.com/clearml/clearml/issues/120)). +* Add [autokeras example](https://github.com/clearml/clearml/blob/master/examples/frameworks/autokeras/autokeras_imdb_example.py) ([GitHub Issue #125](https://github.com/clearml/clearml/issues/125)). * Add caching of extracted zip artifacts. * Add `Task.get_tasks` filtering support. * Add `trains.storage.StorageManager`. @@ -50,7 +50,7 @@ title: Version 0.14 **Bug Fixes** -* Fix renaming / deleting model file right after saving will break asynchronous upload ([GitHub Issue #123](https://github.com/allegroai/trains/issues/123)). +* Fix renaming / deleting model file right after saving will break asynchronous upload ([GitHub Issue #123](https://github.com/clearml/clearml/issues/123)). * Fix shutdown behavior: * Logger flush delay and termination. * Repository / package detection termination. @@ -69,10 +69,10 @@ title: Version 0.14 * Add OpenMPI / Slurm support, support `TRAINS_FORCE_MASTER_NODE` environment variable to allow forcefully specifying a node is master (not based on the MPI Rank). * Add `Task.get_models` and `Task.models` for retrieving stored models for previously executed tasks -* Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) ([GitHub Issue #26](https://github.com/allegroai/trains/issues/26)). +* Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) ([GitHub Issue #26](https://github.com/clearml/clearml/issues/26)). * Support reusing models. * Fix TensorFlow not a number (`NaN`) and infinity (`Inf`) -values support ([GitHub Issue #118](https://github.com/allegroai/trains/issues/118)). +values support ([GitHub Issue #118](https://github.com/clearml/clearml/issues/118)). * Fix joblib binding. * Fix metrics / files / models upload on shutdown. * Fix support for Task init / close multiple times. @@ -86,8 +86,8 @@ values support ([GitHub Issue #118](https://github.com/allegroai/trains/issues/1 * Add support for reporting tables (pandas.DataFrame, CSV files, URL to CSV file) using the `Logger.report_table` method. * Add support for setting a Task's initial iteration to allow continuing previous runs using the `Task.set_initial_iteration` and `Task.get_initial_iteration` methods. -* Add multiprocess task support ([GitHub Issue #96](https://github.com/allegroai/trains/issues/96)). -* Add `trains-init` support for config file override using the `TRAINS_CONFIG_FILE` environment variable or `--file` command line argument ([GitHub Issue #105](https://github.com/allegroai/trains/issues/105)). +* Add multiprocess task support ([GitHub Issue #96](https://github.com/clearml/clearml/issues/96)). +* Add `trains-init` support for config file override using the `TRAINS_CONFIG_FILE` environment variable or `--file` command line argument ([GitHub Issue #105](https://github.com/clearml/clearml/issues/105)). * Add initial slurm support. * Allow disabling repository detection when calling `Task.init`. * Add Jupyter Notebook server execution example. diff --git a/docs/release_notes/sdk/open_source/ver_0_15.md b/docs/release_notes/sdk/open_source/ver_0_15.md index ade4a8f5..8e825506 100644 --- a/docs/release_notes/sdk/open_source/ver_0_15.md +++ b/docs/release_notes/sdk/open_source/ver_0_15.md @@ -11,10 +11,10 @@ title: Version 0.15 **Features** -* Add direct plotly figure reporting ([GitHub Issue #136](https://github.com/allegroai/trains/issues/136)). -* Improve matplotlib integration ([GitHub Issue #140](https://github.com/allegroai/trains/issues/140)). -* Add `Task.set_parent` ([GitHub Issue #144](https://github.com/allegroai/trains/issues/144)). -* Add `Task.wait_for_status` ([GitHub Issue #145](https://github.com/allegroai/trains/issues/145)). +* Add direct plotly figure reporting ([GitHub Issue #136](https://github.com/clearml/clearml/issues/136)). +* Improve matplotlib integration ([GitHub Issue #140](https://github.com/clearml/clearml/issues/140)). +* Add `Task.set_parent` ([GitHub Issue #144](https://github.com/clearml/clearml/issues/144)). +* Add `Task.wait_for_status` ([GitHub Issue #145](https://github.com/clearml/clearml/issues/145)). * Enhance framework `save` / `load` binding callbacks ([pytorch/ignite Issue #1056](https://github.com/pytorch/ignite/issues/1056)). * Add `TrainsJob.is_failed`. * Add APIClient support using `trains.backend_api.session.APIClient`. @@ -23,10 +23,10 @@ title: Version 0.15 **Bug Fixes** -* Fix `pytorch_tensorboard.py` example ([GitHub Issue #148](https://github.com/allegroai/trains/issues/148)). -* Fix `ssh://user@domain@server.com/` in Git repository ([GitHub Issue #149](https://github.com/allegroai/trains/issues/149)). +* Fix `pytorch_tensorboard.py` example ([GitHub Issue #148](https://github.com/clearml/clearml/issues/148)). +* Fix `ssh://user@domain@server.com/` in Git repository ([GitHub Issue #149](https://github.com/clearml/clearml/issues/149)). * Fix do not call `make_deterministic` on import. -* Fix initialization wizard (allow at most two verification retries, then print error) ([GitHub trains-agent Issue #16](https://github.com/allegroai/trains-agent/issues/16)). +* Fix initialization wizard (allow at most two verification retries, then print error) ([GitHub trains-agent Issue #16](https://github.com/clearml/clearml-agent/issues/16)). * Fix Google CoLab code / package detection. * Improve pytorch-ignite integration. * Fix TensorFlow version 2 and later histogram binding. @@ -39,17 +39,17 @@ title: Version 0.15 **Features** -* Add automation support including hyperparameter optimization (see example [here](https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py)) +* Add automation support including hyperparameter optimization (see example [here](https://github.com/clearml/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py)) * `Task.init` `auto_connect_arg_parser` argument can accept a dictionary disabling specific keys from the argparser (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589355878206700)) * Allow `worker_id` override using `TRAINS_WORKER_NAME` environment variable (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589920581251600?thread_ts=1589920220.250100&cid=CTK20V944)) -* Support layout configuration for plotly objects using `extra_layout` argument in all `Logger` reporting methods ([GitHub Issue #136](https://github.com/allegroai/trains/issues/136)). -* Add `Task.execute_remotely` to allow cloning and enqueuing a locally executed task (or stopping and re-enqueuing a remotely executed task) ([GitHub Issue #128](https://github.com/allegroai/trains/issues/128)). +* Support layout configuration for plotly objects using `extra_layout` argument in all `Logger` reporting methods ([GitHub Issue #136](https://github.com/clearml/clearml/issues/136)). +* Add `Task.execute_remotely` to allow cloning and enqueuing a locally executed task (or stopping and re-enqueuing a remotely executed task) ([GitHub Issue #128](https://github.com/clearml/clearml/issues/128)). * Add Parquet framework and model type. * Support recursive model folder packaging. * Add `Task.get_reported_console_output` and `Task.get_reported_scalars` to allow retrieval of reported output and scalar metrics. * Add `Task.add_requirements` to force requirement package into "installed packages." * Improve task reuse responsiveness. -* Add `raise_on_error` (default False) argument to `Model.get_local_copy` and `Artifact.get_local_copy` ([GitHub trains-agent Issue #17](https://github.com/allegroai/trains-agent/issues/17)). +* Add `raise_on_error` (default False) argument to `Model.get_local_copy` and `Artifact.get_local_copy` ([GitHub trains-agent Issue #17](https://github.com/clearml/clearml-agent/issues/17)). * Support `Task.get_task` without project name (i.e. all projects) * Support using the file calling `Task.init` as the task's script in case sys.argv doesn't point to a git repository * Support detecting and remotely executing code running from a module (i.e. `-m module`) @@ -60,8 +60,8 @@ title: Version 0.15 **Bug Fixes** -* Fix `exact_match_regex` in case of empty pattern ([GitHub Issue #138](https://github.com/allegroai/trains/issues/138)). -* Address deprecation warning and newer `attrs` versions in MetricsEventAdapter ([GitHub Issue #134](https://github.com/allegroai/trains/issues/134)). +* Fix `exact_match_regex` in case of empty pattern ([GitHub Issue #138](https://github.com/clearml/clearml/issues/138)). +* Address deprecation warning and newer `attrs` versions in MetricsEventAdapter ([GitHub Issue #134](https://github.com/clearml/clearml/issues/134)). * Fix issues with plotly support (Trains Slack channel [thread](https://clearml.slack.com/archives/CTK20V944/p1589871253243600) and [thread](https://clearml.slack.com/archives/CTK20V944/p1589279340195000)). * Fix default argument behavior to match argparse behavior. * Fix `OutputModel` with `task=None` should use current task, if exists. @@ -70,5 +70,5 @@ title: Version 0.15 * Fix `NaN`, `Inf` and `-Inf` values display in reported table (not supported by JSON). * Limit max requirement size to 0.5 MB. * Fix issues with repository analysis. -* Fix `StorageManager` so it should only try to extract .zip files, `Model` should not auto extract package ([GitHub trains-agent Issue #17](https://github.com/allegroai/trains-agent/issues/17)). +* Fix `StorageManager` so it should only try to extract .zip files, `Model` should not auto extract package ([GitHub trains-agent Issue #17](https://github.com/clearml/clearml-agent/issues/17)). diff --git a/docs/release_notes/sdk/open_source/ver_0_16.md b/docs/release_notes/sdk/open_source/ver_0_16.md index 18c0c34f..20a8a7e0 100644 --- a/docs/release_notes/sdk/open_source/ver_0_16.md +++ b/docs/release_notes/sdk/open_source/ver_0_16.md @@ -10,13 +10,13 @@ title: Version 0.16 **Features** -- Add Hydra support (GitHub trains Issue 219). -- Add cifar ignite example (GitHub trains Issue 237). -- Add auto extraction of `tar.gz` files when using `StorageManager` (GitHub trains Issue 237). -- Add `Task.init()` argument `auto_connect_streams` controlling stdout / stderr / logging capture (GitHub trains Issue 181). -- Add carriage return flush support using the `sdk.development.worker.console_cr_flush_period` configuration setting (GitHub trains Issue 181). -- Add `Task.create_function_task()` to allow creating a new task, using a function and arguments, to be executed remotely (GitHub trains Issue 230). -- Allow disabling SSL certificates verification using `Task.setup_upload()` argument `verify` or AWS S3 bucket configuration `verify` property (GitHub trains Issue 256). +- Add Hydra support (GitHub trains Issue 219). +- Add cifar ignite example (GitHub trains Issue 237). +- Add auto extraction of `tar.gz` files when using `StorageManager` (GitHub trains Issue 237). +- Add `Task.init()` argument `auto_connect_streams` controlling stdout / stderr / logging capture (GitHub trains Issue 181). +- Add carriage return flush support using the `sdk.development.worker.console_cr_flush_period` configuration setting (GitHub trains Issue 181). +- Add `Task.create_function_task()` to allow creating a new task, using a function and arguments, to be executed remotely (GitHub trains Issue 230). +- Allow disabling SSL certificates verification using `Task.setup_upload()` argument `verify` or AWS S3 bucket configuration `verify` property (GitHub trains Issue 256). - Add `StorageManager.get_files_server()`. - Add `Task.get_project_id()` using project name. - Add `project_name` argument to `Task.set_project()`. @@ -28,13 +28,13 @@ title: Version 0.16 **Bug Fixes** -- Fix Optuna HPO parameter serializing (GitHub trains Issue 254). -- Fix connect dictionary `''` cast to `None` (GitHub trains Issue 258). -- Fix lightgbm binding keyword argument issue (GitHub trains Issue 251). -- Fix artifact preview if artifact body is remote URI (GitHub trains Issue 239). -- Fix infinite recursion in `StorageManager` upload (GitHub trains Issue 253). -- Fix keras reusing model object only if the filename is the same (GitHub trains Issue 252). -- Fix running remotely with no configuration should not crash but output a warning (GitHub trains Issue 243). +- Fix Optuna HPO parameter serializing (GitHub trains Issue 254). +- Fix connect dictionary `''` cast to `None` (GitHub trains Issue 258). +- Fix lightgbm binding keyword argument issue (GitHub trains Issue 251). +- Fix artifact preview if artifact body is remote URI (GitHub trains Issue 239). +- Fix infinite recursion in `StorageManager` upload (GitHub trains Issue 253). +- Fix keras reusing model object only if the filename is the same (GitHub trains Issue 252). +- Fix running remotely with no configuration should not crash but output a warning (GitHub trains Issue 243). - Fix matplotlib 3.3.3 support: - Fix global figure enumeration. - Fix binding without a title reported a single plot (`untitled 00`) instead of increasing the counter. @@ -59,9 +59,9 @@ title: Version 0.16 **Features** * Add LightGBM support. -* Add initial Hydra support (GitHub trains Issue 219). -* Add synchronous support for `Task.upload_artifact()` (GitHub trains Issue 231). -* Add `sdk.development.store_code_diff_from_remote` (default `false`) to store diff from remote HEAD instead of local HEAD (GitHub trains Issue 222). +* Add initial Hydra support (GitHub trains Issue 219). +* Add synchronous support for `Task.upload_artifact()` (GitHub trains Issue 231). +* Add `sdk.development.store_code_diff_from_remote` (default `false`) to store diff from remote HEAD instead of local HEAD (GitHub trains Issue 222). * Add `sdk.development.detect_with_conda_freeze` (default `true`) for full conda freeze (requires trains-agent >= 16.2). * Add user properties support in Task object. * Add `Logger.report_table()` support for table as list of lists. @@ -78,7 +78,7 @@ title: Version 0.16 **Bug Fixes** -* Fix `Task.update_output_model()` wrong argument order (GitHub trains Issue 220). +* Fix `Task.update_output_model()` wrong argument order (GitHub trains Issue 220). * Fix initializing task on argparse parse in remote mode. Do not call `Task.init()` to avoid auto connect, use `Task.get_task()` instead. * Fix detected task cwd outside of repository root folder. * Fix `Task.connect(dict)` to place non-existing entries on the section name instead of General. @@ -99,24 +99,24 @@ title: Version 0.16 **Features** -* Add `Task.set_resource_monitor_iteration_timeout()` to set ResourceMonitor iteration wait duration timeout (GitHub trains Issue 208). -* Add PyTorch Lightning save/restore model binding (GitHub trains Issue 212). +* Add `Task.set_resource_monitor_iteration_timeout()` to set ResourceMonitor iteration wait duration timeout (GitHub trains Issue 208). +* Add PyTorch Lightning save/restore model binding (GitHub trains Issue 212). * Add `git diff` for repository submodule (requires git 2.14 or above). * Add `TrainsJob.is_completed()` and `TrainsJob.is_aborted()`. * Add `Task.logger` property. -* Add Pipeline Controller automation and example (see [here](https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_tasks.py)). +* Add Pipeline Controller automation and example (see [here](https://github.com/clearml/clearml/blob/master/examples/pipeline/pipeline_from_tasks.py)). * Add improved trace filtering capabilities in `trains.debugging.trace.trace_trains()`. * Add default help per argument (if not provided) in ArgParser binding. * Deprecate `Task.reporter`. * Update PyTorch example. -* Remove warning on skipped auto-magic model logging (GitHub trains Issue 206). +* Remove warning on skipped auto-magic model logging (GitHub trains Issue 206). * Support Keras restructuring for Network, Model and Sequential. * Update autokeras requirements according to [https://github.com/keras-team/autokeras#installation](https://github.com/keras-team/autokeras#installation). **Bug Fixes** -* Fix joblib auto logging models failing on compressed streams (GitHub trains Issue 203). -* Fix sending empty reports (GitHub trains Issue 205). +* Fix joblib auto logging models failing on compressed streams (GitHub trains Issue 203). +* Fix sending empty reports (GitHub trains Issue 205). * Fix scatter2d sub-sampling and rounding. * Fix plots reporting: * `NaN` representation (matplotlib conversion). @@ -142,12 +142,12 @@ title: Version 0.16 **Bug Fixes** -* Fix typing dependency for `Python<3.5` ([GitHub trains Issue 184](https://github.com/allegroai/trains/issues/184)). +* Fix typing dependency for `Python<3.5` ([GitHub trains Issue 184](https://github.com/clearml/clearml/issues/184)). * Fix git+https requirements handling, resolve top_level.txt package name (kerastuner from git was not detected). * Fix `Task.get_reported_console_output()` for new Trains Server API v2.9. * Fix cache handling for different partitions / drives / devices. * Disable offline mode when running remotely (i.e. executed by Trains Agent). -* Fix artifact upload to only use file stream when not uploading a locally stored file (multipart upload is not supported on stream upload) (GitHub trains Issue 189). +* Fix artifact upload to only use file stream when not uploading a locally stored file (multipart upload is not supported on stream upload) (GitHub trains Issue 189). * Fix double-escaped model design text when connecting OutputModel. @@ -155,12 +155,12 @@ title: Version 0.16 **Features** -* Add continuing of previously executed experiments. Add `Task.init()` argument `continue_last_task` to continue a previously used Task ([GitHub Issue #160](https://github.com/allegroai/trains/issues/160)). -* Allow Task editing / creation from code. `Task.export_task/import_task/update_task()` ([GitHub Issue #128](https://github.com/allegroai/trains/issues/128)). +* Add continuing of previously executed experiments. Add `Task.init()` argument `continue_last_task` to continue a previously used Task ([GitHub Issue #160](https://github.com/clearml/clearml/issues/160)). +* Allow Task editing / creation from code. `Task.export_task/import_task/update_task()` ([GitHub Issue #128](https://github.com/clearml/clearml/issues/128)). * Add offline mode. Use `Task.set_offline()` and `Task.import_offline_session()`: * Support setting offline mode via `TRAINS_OFFLINE_MODE=1` environment variable. * Support setting offline API version via `TRAINS_OFFLINE_MODE=2.9` environment variable. -* Automatically pickle all objects uploaded as artifacts, `task.upload_artifact()` argument `auto_pickle=True` ([GitHub Issue #153](https://github.com/allegroai/trains/issues/153)). +* Automatically pickle all objects uploaded as artifacts, `task.upload_artifact()` argument `auto_pickle=True` ([GitHub Issue #153](https://github.com/clearml/clearml/issues/153)). * Add multiple sections / groups support for Task hyperparameters, using `Task.connect()`. * Add multiple configurations (files) using `Task.connect_configuration()`. * Allow enabling OS environment logging using the `sdk.development.log_os_environments` configuration parameter (complements the `TRAINS_LOG_ENVIRONMENT` environment variable). @@ -168,25 +168,25 @@ title: Version 0.16 * Add initial Keras-Tuner support ([GitHub Issue keras-team/keras-tuner #334](https://github.com/keras-team/keras-tuner/issues/334)). * Add automatic FastAI logging. It is disabled if Tensorboard is loaded (assuming TensorBoardLogger will be used). * Support Tensorboard text logging (`add_text()`) as debug samples (`.txt` files), instead of as console output. -* Allow for more standard confusion matrix reporting. `Logger.report_confusion_matrix()` argument `yaxis_reversed` (flips the confusion matrix if `True`, default `False`) ([GitHub Issue #165](https://github.com/allegroai/trains/issues/165)). +* Allow for more standard confusion matrix reporting. `Logger.report_confusion_matrix()` argument `yaxis_reversed` (flips the confusion matrix if `True`, default `False`) ([GitHub Issue #165](https://github.com/clearml/clearml/issues/165)). * Add support for Trains Server 0.16.0 (API v2.9 support). -* Allow disabling Trains update message from the log using the `TRAINS_SUPPRESS_UPDATE_MESSAGE` environment variable ([GitHub Issue #157](https://github.com/allegroai/trains/issues/157)). +* Allow disabling Trains update message from the log using the `TRAINS_SUPPRESS_UPDATE_MESSAGE` environment variable ([GitHub Issue #157](https://github.com/clearml/clearml/issues/157)). * Add AWS EC2 Auto-Scaler service wizard and Service. * Improved and updated examples: * Add Keras Tuner CIFAR10 example. * Add FastAI example. - * Update PyTorch Jupyter notebook examples ([GitHub Issue #150](https://github.com/allegroai/trains/issues/150)). + * Update PyTorch Jupyter notebook examples ([GitHub Issue #150](https://github.com/clearml/clearml/issues/150)). * Support global requirements detection using `pip freeze` (set `sdk.development.detect_with_pip_freeze` configuration in `trains.conf`). * Add `Task.get_projects()` to get all projects in the system, sorted by last update time. **Bug Fixes** -* Fix UTC to time stamp in comment ([GitHub Issue #152](https://github.com/allegroai/trains/issues/152)). +* Fix UTC to time stamp in comment ([GitHub Issue #152](https://github.com/clearml/clearml/issues/152)). * Fix and enhance GPU monitoring: - * Fix GPU stats on Windows machines ([GitHub Issue #177](https://github.com/allegroai/trains/issues/177)). - * More robust GPU monitoring ([GitHub Issue #170](https://github.com/allegroai/trains/issues/170)). -* Fix filename too long bug ([GitHub trains-server Issue #49](https://github.com/allegroai/trains-server/issues/49)). -* Fix TensorFlow image logging to allow images with no width / height / color metadata ([GitHub Issue #182](https://github.com/allegroai/trains/issues/182)). + * Fix GPU stats on Windows machines ([GitHub Issue #177](https://github.com/clearml/clearml/issues/177)). + * More robust GPU monitoring ([GitHub Issue #170](https://github.com/clearml/clearml/issues/170)). +* Fix filename too long bug ([GitHub trains-server Issue #49](https://github.com/clearml/clearml-server/issues/49)). +* Fix TensorFlow image logging to allow images with no width / height / color metadata ([GitHub Issue #182](https://github.com/clearml/clearml/issues/182)). * Fix multiprocessing Pool throw exception in pool hangs execution. Call original signal handler and re-flush `stdout`. * Fix `plotly` support for `matplotlib` 3.3. * Add Python 2.7 support for `get_current_thread_id()`. diff --git a/docs/release_notes/sdk/open_source/ver_0_17.md b/docs/release_notes/sdk/open_source/ver_0_17.md index aa23d87e..1f389bc2 100644 --- a/docs/release_notes/sdk/open_source/ver_0_17.md +++ b/docs/release_notes/sdk/open_source/ver_0_17.md @@ -6,7 +6,7 @@ title: Version 0.17 **Features** -- Add `force_download` argument to `Artifact.get()` and `Artifact.get_local_copy()` ([clearml GitHub Issue 319](https://github.com/allegroai/clearml/issues/319)). +- Add `force_download` argument to `Artifact.get()` and `Artifact.get_local_copy()` ([clearml GitHub Issue 319](https://github.com/clearml/clearml/issues/319)). - Support all reporting using subprocesses instead of threads (default `sdk.development.report_use_subprocess=True`). - Improve Datasets support: * Add `clearml-data publish` to allow publishing a dataset task. @@ -23,7 +23,7 @@ title: Version 0.17 - Add links to Tasks in optimization summary table (not just Task IDs). - Add support for datetime in scatter plots + matplotlib support. - Improve plotly value type conforming. -- Improve PyTorch `DataLoader` speed ([clearml GitHub Issue 207](https://github.com/allegroai/clearml/issues/207)). +- Improve PyTorch `DataLoader` speed ([clearml GitHub Issue 207](https://github.com/clearml/clearml/issues/207)). - Update Auto Scaler default values and configuration. - Examples: * Add Hydra example. @@ -32,24 +32,24 @@ title: Version 0.17 **Bug Fixes** -- Fix warning or error message if requirements parsing failed ([clearml GitHub Issue 291](https://github.com/allegroai/clearml/issues/291)). -- Fix pytorch-lighting multi node store ([clearml GitHub Issue 292](https://github.com/allegroai/clearml/issues/292)). -- Fix strip remote diff ([clearml GitHub Issue 295](https://github.com/allegroai/clearml/issues/295)). -- Fix Python package detection `sklearn` -> `scikit-learn` ([clearml GitHub Issue 296](https://github.com/allegroai/clearml/issues/296)). +- Fix warning or error message if requirements parsing failed ([clearml GitHub Issue 291](https://github.com/clearml/clearml/issues/291)). +- Fix pytorch-lighting multi node store ([clearml GitHub Issue 292](https://github.com/clearml/clearml/issues/292)). +- Fix strip remote diff ([clearml GitHub Issue 295](https://github.com/clearml/clearml/issues/295)). +- Fix Python package detection `sklearn` -> `scikit-learn` ([clearml GitHub Issue 296](https://github.com/clearml/clearml/issues/296)). - Fix argparse issues: - * Fix argparse with `[None]` in default parameter ([clearml GitHub Issue 297](https://github.com/allegroai/clearml/issues/297)). - * Fix parsing of arguments in scientific notation ([clearml GitHub Issue 313](https://github.com/allegroai/clearml/issues/313)). + * Fix argparse with `[None]` in default parameter ([clearml GitHub Issue 297](https://github.com/clearml/clearml/issues/297)). + * Fix parsing of arguments in scientific notation ([clearml GitHub Issue 313](https://github.com/clearml/clearml/issues/313)). * Fix argparser logging always captures defaults (Windows only, ignored cmd). * Fix argparse `nargs` passed in command line `--nargs 1 2` should be stored as `[1, 2]` not as `['1', '2']`. * Fix support for nonstandard argparse with default value that is not of defined type. * Fix server updated with the argparse in remote before Task.init() is called (respect skipped args). - Fix Dataset support: - * Fix `Dataset.remove_files()` can't find files when files are in dataset root ([clearml GitHub Issue 303](https://github.com/allegroai/clearml/issues/303)). + * Fix `Dataset.remove_files()` can't find files when files are in dataset root ([clearml GitHub Issue 303](https://github.com/clearml/clearml/issues/303)). * Fix closing a dataset with only files removed. * Fix Dataset generate removed / modified / added of summary table. -- Fix Hydra multi-run support ([clearml GitHub Issue 306](https://github.com/allegroai/clearml/issues/306)). +- Fix Hydra multi-run support ([clearml GitHub Issue 306](https://github.com/clearml/clearml/issues/306)). - Fix TF/TensorBoard support: - * Fix TensorBoard multiple `Task.init()` / `Task.close()` calls within the same process ([clearml GitHub Issue 312](https://github.com/allegroai/clearml/issues/312)). + * Fix TensorBoard multiple `Task.init()` / `Task.close()` calls within the same process ([clearml GitHub Issue 312](https://github.com/clearml/clearml/issues/312)). * Fix TensorBoard 2+ `pr_curve`. * Fix TF `pr_curve` should not be inverted. * Fix TF +2.3 mixed eager mode execution summary metrics not reported. @@ -85,7 +85,7 @@ title: Version 0.17 **Features** -- Add `HyperParameterOptimizer` parallel coordinates ([clearml GitHub Issue 279](https://github.com/allegroai/trains/issues/279)). +- Add `HyperParameterOptimizer` parallel coordinates ([clearml GitHub Issue 279](https://github.com/clearml/clearml/issues/279)). - Add `Task.init()` argument `tags`. - Change HPO parallel coordinates color scale, yellow is low. - Change `HyperParameterOptimizer` `spawn_task_project` to `spawn_project`. @@ -105,7 +105,7 @@ title: Version 0.17 * Add `Task.delete()` support. * Add `Task.debug_simulate_remote_task()` to simulate task execution by ClearML Agent. -* Add warning on archived Task in pipeline ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)). +* Add warning on archived Task in pipeline ([clearml GitHub Issue 274](https://github.com/clearml/clearml/issues/274)). * Add `Task.init(..., output_uri=True)`, which will use the default files_server as output uri. * Make `clearml-data` CLI stateful, so it remembers last dataset ID as default dataset. * Added `HyperParameterOptimizer.get_optimizer_top_experiments()` for querying post execution optimization pipeline. @@ -116,9 +116,9 @@ title: Version 0.17 **Bug Fixes** -* Fix PipelineController running remotely without configuration does not execute the default code pipeline ([clearml GitHub Issue 273](https://github.com/allegroai/trains/issues/273)). -* Fix reusing task after its project was deleted ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)). -* Fix `Task.archived_tag read-only` property does not work ([clearml GitHub Issue 274](https://github.com/allegroai/trains/issues/274)). +* Fix PipelineController running remotely without configuration does not execute the default code pipeline ([clearml GitHub Issue 273](https://github.com/clearml/clearml/issues/273)). +* Fix reusing task after its project was deleted ([clearml GitHub Issue 274](https://github.com/clearml/clearml/issues/274)). +* Fix `Task.archived_tag read-only` property does not work ([clearml GitHub Issue 274](https://github.com/clearml/clearml/issues/274)). * Fix argparse support to store consistent str representation of custom objects. Avoid changing default value if remote value matches. * Fix argparse type as function. * Fix Dataset add single and multiple file(s). @@ -151,9 +151,9 @@ title: Version 0.17 **Features** -* Renamed `trains` as `clearml` (see [clearml](https://github.com/allegroai/clearml) in GitHub). -* Add `clearml-data` CLI for Differential Dataset Management & Versioning (see details). -* Add `clearml-task` CLI for creating (importing) code into **ClearML** (see details). +* Renamed `trains` as `clearml` (see [clearml](https://github.com/clearml/clearml) in GitHub). +* Add `clearml-data` CLI for Differential Dataset Management & Versioning (see details). +* Add `clearml-task` CLI for creating (importing) code into **ClearML** (see details). * Add `Task.create`, which supports importing a code / repository into **ClearML**, including remote execution with `clearml-agent`. **Bug Fix** diff --git a/docs/release_notes/sdk/open_source/ver_1_0.md b/docs/release_notes/sdk/open_source/ver_1_0.md index 2ceee78e..e2f9ef68 100644 --- a/docs/release_notes/sdk/open_source/ver_1_0.md +++ b/docs/release_notes/sdk/open_source/ver_1_0.md @@ -5,8 +5,8 @@ title: Version 1.0 **Features** -- Add Click support ([ClearML GitHub issue #386](https://github.com/allegroai/clearml/issues/386)) -- Add progress bar to SHA2 generation ([ClearML GitHub issue #396](https://github.com/allegroai/clearml/issues/396)) +- Add Click support ([ClearML GitHub issue #386](https://github.com/clearml/clearml/issues/386)) +- Add progress bar to SHA2 generation ([ClearML GitHub issue #396](https://github.com/clearml/clearml/issues/396)) - Add prefix to Task reported runtime info: `cpu_cores`, `gpu_driver_version` and `gpu_driver_cuda_version` - Add support for `Logger.report_text()` explicit log-level reporting - Add `return_full_path` argument to `StorageManager.list()` @@ -21,8 +21,8 @@ title: Version 1.0 **Bug Fixes** -- Fix Pandas with multi-index ([ClearML GitHub issue #399](https://github.com/allegroai/clearml/issues/399)) -- Fix check permissions fail in `HTTPDriver` ([ClearML GitHub issue #394](https://github.com/allegroai/clearml/issues/394)) +- Fix Pandas with multi-index ([ClearML GitHub issue #399](https://github.com/clearml/clearml/issues/399)) +- Fix check permissions fail in `HTTPDriver` ([ClearML GitHub issue #394](https://github.com/clearml/clearml/issues/394)) - Fix Dataset not setting system tag on existing `data_processing` Tasks - Fix disable redundant resource monitoring in pipeline controller - Fix `ClearMLJob` when both `project` and `target_project` are specified @@ -37,17 +37,17 @@ title: Version 1.0 **Features** -- Add Google Colab notebook tutorial ([ClearML GitHub PR 368](https://github.com/allegroai/clearml/issues/368) and [ClearML GitHub PR 374](https://github.com/allegroai/clearml/issues/374)) -- Add support for GIF images in Tensorboard ([ClearML GitHub issue 372](https://github.com/allegroai/clearml/issues/372)) -- Add a tensorboardX example for `add_video` (creates GIFs in tensorboard) ([ClearML GitHub PR 372](https://github.com/allegroai/clearml/issues/372)) +- Add Google Colab notebook tutorial ([ClearML GitHub PR 368](https://github.com/clearml/clearml/issues/368) and [ClearML GitHub PR 374](https://github.com/clearml/clearml/issues/374)) +- Add support for GIF images in Tensorboard ([ClearML GitHub issue 372](https://github.com/clearml/clearml/issues/372)) +- Add a tensorboardX example for `add_video` (creates GIFs in tensorboard) ([ClearML GitHub PR 372](https://github.com/clearml/clearml/issues/372)) - Add auto scaler customizable boot bash script - Add `Task.ignore_requirements` - Deprecate `Logger.tensorboard_single_series_per_graph()` as it is now controlled from the UI 🙂 **Bug Fixes** -- Fix `default_output_uri` for Dataset creation ([ClearML GitHub issue 371](https://github.com/allegroai/clearml/issues/371)) -- Fix `clearml-task` failing without a docker script ([ClearML GitHub issue 378](https://github.com/allegroai/clearml/issues/378)) +- Fix `default_output_uri` for Dataset creation ([ClearML GitHub issue 371](https://github.com/clearml/clearml/issues/371)) +- Fix `clearml-task` failing without a docker script ([ClearML GitHub issue 378](https://github.com/clearml/clearml/issues/378)) - Fix PyTorch DDP sub-process spawn multi-process - Fix `Task.execute_remotely()` on created Task (not initialized Task) - Fix auto scaler custom bash script should be called last before starting agent @@ -58,7 +58,7 @@ title: Version 1.0 **Features** -- Use default `boto` credential chain if no keys are provided in the configuration file or environment variables ([ClearML GitHub PR 342](https://github.com/allegroai/clearml/issues/342)) +- Use default `boto` credential chain if no keys are provided in the configuration file or environment variables ([ClearML GitHub PR 342](https://github.com/clearml/clearml/issues/342)) - Support `DummyModel` configuration ([Slack Channel](https://clearml.slack.com/archives/CTK20V944/p1621469235085400)) - Add `report_matplotlib_figure(..., report_interactive=False)` allowing to upload a matplotlib as a non-interactive (high quality png) plot - Add `Logger.matplotlib_force_report_non_interactive()` @@ -75,10 +75,10 @@ title: Version 1.0 **Bug Fixes** -- Fix Hydra should not store the full resolved OmegaConf ([ClearML GitHub issue 327](https://github.com/allegroai/clearml/issues/327)) -- Fix direct import of keras save/load model functions ([ClearML GitHub issue 355](https://github.com/allegroai/clearml/issues/355)) -- Fix run as module ([ClearML GitHub issue 359](https://github.com/allegroai/clearml/issues/359)) -- Fix Python 2.7 support ([ClearML GitHub issue 366](https://github.com/allegroai/clearml/issues/366)) +- Fix Hydra should not store the full resolved OmegaConf ([ClearML GitHub issue 327](https://github.com/clearml/clearml/issues/327)) +- Fix direct import of keras save/load model functions ([ClearML GitHub issue 355](https://github.com/clearml/clearml/issues/355)) +- Fix run as module ([ClearML GitHub issue 359](https://github.com/clearml/clearml/issues/359)) +- Fix Python 2.7 support ([ClearML GitHub issue 366](https://github.com/clearml/clearml/issues/366)) - Fix `Task.add_requirements()` passing `package_version` starting with `@`, `;` or `#` - Fix import keras from TF - Fix support for Hydra's `run_job()` change in parameter order by passing `config` and `task_function` as keyword arguments @@ -113,10 +113,10 @@ Arguments order changed in `Logger.report_line_plot()`, `Logger.report_plotly()` **Features** -* Add OS environment variable `CLEARML_DEFAULT_OUTPUT_URI` to override default `output_uri` for automatic models upload ([ClearML GitHub issue 328](https://github.com/allegroai/clearml/issues/328)) -* Add `Task.force_requirements_env_freeze()` forcing `pip freeze` instead of package analysis ([ClearML GitHub issue 304](https://github.com/allegroai/clearml/issues/304)) +* Add OS environment variable `CLEARML_DEFAULT_OUTPUT_URI` to override default `output_uri` for automatic models upload ([ClearML GitHub issue 328](https://github.com/clearml/clearml/issues/328)) +* Add `Task.force_requirements_env_freeze()` forcing `pip freeze` instead of package analysis ([ClearML GitHub issue 304](https://github.com/clearml/clearml/issues/304)) * Improve pipeline support - * Add Task parameters override in `ClearmlJob` ([ClearML GitHub issue 214](https://github.com/allegroai/clearml/issues/214)) + * Add Task parameters override in `ClearmlJob` ([ClearML GitHub issue 214](https://github.com/clearml/clearml/issues/214)) * Add PipelineController node skip visualization and pre/post-execution callback * Allow enabling/disabling tensorboard auto-connect using `Task.init(auto_connect_frameworks={'tensorboard': True/False})` (default: `True`) * Allow to continue running from previous execution iteration @@ -146,9 +146,9 @@ Arguments order changed in `Logger.report_line_plot()`, `Logger.report_plotly()` **Bug Fixes** -* Fix `\r` and `\n` in debug samples title/series ([ClearML GitHub issue 323](https://github.com/allegroai/clearml/issues/323)) -* Fix Google API credentials without storage section raises exception on import ([ClearML GitHub issue 331](https://github.com/allegroai/clearml/issues/331)) -* Fix Matplotlib automagic export legend not showing series names ([ClearML GitHub issue 337](https://github.com/allegroai/clearml/issues/337)) +* Fix `\r` and `\n` in debug samples title/series ([ClearML GitHub issue 323](https://github.com/clearml/clearml/issues/323)) +* Fix Google API credentials without storage section raises exception on import ([ClearML GitHub issue 331](https://github.com/clearml/clearml/issues/331)) +* Fix Matplotlib automagic export legend not showing series names ([ClearML GitHub issue 337](https://github.com/clearml/clearml/issues/337)) * Fix `Task.set_comment/name()` passing `None` should be equivalent to `''` * Fix `Task.set_credentials()` uses incorrect config file when both `trains.conf` and `clearml.conf` are found * Fix `Task.set_credetnials()` should override settings from config file diff --git a/docs/release_notes/sdk/open_source/ver_1_1.md b/docs/release_notes/sdk/open_source/ver_1_1.md index d62ba639..6deecd44 100644 --- a/docs/release_notes/sdk/open_source/ver_1_1.md +++ b/docs/release_notes/sdk/open_source/ver_1_1.md @@ -6,7 +6,7 @@ title: Version 1.1 **Features** -- Add `Task.force_store_standalone_script()` to force storing standalone script instead of a Git repository reference ([ClearML GitHub issue #340](https://github.com/allegroai/clearml/issues/340)) +- Add `Task.force_store_standalone_script()` to force storing standalone script instead of a Git repository reference ([ClearML GitHub issue #340](https://github.com/clearml/clearml/issues/340)) - Add `Logger.set_default_debug_sample_history()` and `Logger.get_default_debug_sample_history()` to allow controlling maximum debug samples programmatically - Add populate now stores function arg types as part of the hyperparameters @@ -15,8 +15,8 @@ title: Version 1.1 **Bug Fixes** -- Fix and upgrade the SlackMonitor ([ClearML GitHub issue #533](https://github.com/allegroai/clearml/issues/533)) -- Fix network issues causing Task to stop on status change when no status change has occurred ([ClearML GitHub issue #535](https://github.com/allegroai/clearml/issues/535)) +- Fix and upgrade the SlackMonitor ([ClearML GitHub issue #533](https://github.com/clearml/clearml/issues/533)) +- Fix network issues causing Task to stop on status change when no status change has occurred ([ClearML GitHub issue #535](https://github.com/clearml/clearml/issues/535)) - Fix Pipeline controller function support for dict as input argument - Fix uploading the same metric/variant from multiple processes in threading mode should create a unique file per process (since global counter is not passed between the subprocesses) - Fix resource monitoring should only run in the main process when using threaded logging mode @@ -32,15 +32,15 @@ title: Version 1.1 ### ClearML 1.1.5 **Features** -* Add support for `jsonargpraser` ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403)) +* Add support for `jsonargpraser` ([ClearML GitHub issue #403](https://github.com/clearml/clearml/issues/403)) * Add `HyperParameterOptimizer.get_top_experiments_details()` returns the hparams and metrics of the top performing - experiments of an HPO ([ClearML GitHub issue #473](https://github.com/allegroai/clearml/issues/473)) + experiments of an HPO ([ClearML GitHub issue #473](https://github.com/clearml/clearml/issues/473)) * Allow overriding initial iteration offset using environment variable (`CLEARML_SET_ITERATION_OFFSET`) or `Task.init(continue_last_task==)` - ( [ClearML GitHub issue #496](https://github.com/allegroai/clearml/issues/496)) -* Add better input handling for `clearml-init` in colab ([ClearML GitHub issue #515](https://github.com/allegroai/clearml/issues/515)) -* Add environment variable for default request method ([ClearML GitHub issue #521](https://github.com/allegroai/clearml/issues/521)) -* Add `LocalClearmlJob` as possible option for HPO ([ClearML GitHub issue #525](https://github.com/allegroai/clearml/issues/525)) -* Add convenience functionality to `clearml-data` ([ClearML GitHub issue #526](https://github.com/allegroai/clearml/issues/526)) + ( [ClearML GitHub issue #496](https://github.com/clearml/clearml/issues/496)) +* Add better input handling for `clearml-init` in colab ([ClearML GitHub issue #515](https://github.com/clearml/clearml/issues/515)) +* Add environment variable for default request method ([ClearML GitHub issue #521](https://github.com/clearml/clearml/issues/521)) +* Add `LocalClearmlJob` as possible option for HPO ([ClearML GitHub issue #525](https://github.com/clearml/clearml/issues/525)) +* Add convenience functionality to `clearml-data` ([ClearML GitHub issue #526](https://github.com/clearml/clearml/issues/526)) * Add support for `vscode-jupyter` ([microsoft/vscode-jupyter#8531](https://github.com/microsoft/vscode-jupyter/pull/8531)) * Improve detection of running reporting subprocess (including zombie state) * Support controlling S3/Google Cloud Storage `_stream_download_pool_connections` using the `stream_connections` configuration @@ -59,21 +59,21 @@ title: Version 1.1 * Add `PipelineDecorator.multi_instance_support` * Add `PipelineDecorator` to `clearml` and `clearml.automation` namespaces * Documentation and examples - * Update docstrings ([ClearML GitHub issue #501](https://github.com/allegroai/clearml/issues/501)) - * Add Markdown in pipeline jupyter notebooks ([ClearML GitHub issue #502](https://github.com/allegroai/clearml/issues/502)) - * Update pipeline example ([ClearML GitHub issue #494](https://github.com/allegroai/clearml/issues/494)) - * Add abseil example ([ClearML GitHub issue #509](https://github.com/allegroai/clearml/issues/509)) - * Change README to dark theme ([ClearML GitHub issue #513](https://github.com/allegroai/clearml/issues/513)) - * Update XGBoost example ([ClearML GitHub issue #524](https://github.com/allegroai/clearml/issues/524)) - * Change example name ([ClearML GitHub issue #528](https://github.com/allegroai/clearml/issues/528)) + * Update docstrings ([ClearML GitHub issue #501](https://github.com/clearml/clearml/issues/501)) + * Add Markdown in pipeline jupyter notebooks ([ClearML GitHub issue #502](https://github.com/clearml/clearml/issues/502)) + * Update pipeline example ([ClearML GitHub issue #494](https://github.com/clearml/clearml/issues/494)) + * Add abseil example ([ClearML GitHub issue #509](https://github.com/clearml/clearml/issues/509)) + * Change README to dark theme ([ClearML GitHub issue #513](https://github.com/clearml/clearml/issues/513)) + * Update XGBoost example ([ClearML GitHub issue #524](https://github.com/clearml/clearml/issues/524)) + * Change example name ([ClearML GitHub issue #528](https://github.com/clearml/clearml/issues/528)) **Bug Fixes** -* Fix `TriggerScheduler` on Dataset change ([ClearML GitHub issue #491](https://github.com/allegroai/clearml/issues/491)) -* Fix links in Jupyter Notebooks ([ClearML GitHub issue #505](https://github.com/allegroai/clearml/issues/505)) -* Fix `pandas` delta datetime conversion ([ClearML GitHub issue #510](https://github.com/allegroai/clearml/issues/510)) -* Fix `matplotlib` auto-magic detect bar graph series name ([ClearML GitHub issue #518](https://github.com/allegroai/clearml/issues/518)) +* Fix `TriggerScheduler` on Dataset change ([ClearML GitHub issue #491](https://github.com/clearml/clearml/issues/491)) +* Fix links in Jupyter Notebooks ([ClearML GitHub issue #505](https://github.com/clearml/clearml/issues/505)) +* Fix `pandas` delta datetime conversion ([ClearML GitHub issue #510](https://github.com/clearml/clearml/issues/510)) +* Fix `matplotlib` auto-magic detect bar graph series name ([ClearML GitHub issue #518](https://github.com/clearml/clearml/issues/518)) * Fix path limitation on storage services (posix, object storage) when storing target artifacts by limiting length of - project name (full path) and task name used for object path ([ClearML GitHub issue #516](https://github.com/allegroai/clearml/issues/516)) + project name (full path) and task name used for object path ([ClearML GitHub issue #516](https://github.com/clearml/clearml/issues/516)) * Fix multi-processing context block catching exception * Fix Google Cloud Storage with no default project causes a crash * Fix main process's reporting subprocess lost, switch back to thread mode @@ -100,8 +100,8 @@ title: Version 1.1 **Bug Fixes** -- Fix duplicate keyword argument (affects `clearml-data`, `Dataset.get()`) ([ClearML GitHub issue #490](https://github.com/allegroai/clearml/issues/490)) -- Fix session raises missing host error when in offline mode ([ClearML GitHub issue #489](https://github.com/allegroai/clearml/issues/489)) +- Fix duplicate keyword argument (affects `clearml-data`, `Dataset.get()`) ([ClearML GitHub issue #490](https://github.com/clearml/clearml/issues/490)) +- Fix session raises missing host error when in offline mode ([ClearML GitHub issue #489](https://github.com/clearml/clearml/issues/489)) - Fix `Task.get_task()` does not load `output_uri` from stored Task - Fix `Task.get_models()['input']` returns string instead of `clearml.Model` - Fix `tf.saved_model.load()` binding for `TensorFlow>=2.0` @@ -111,11 +111,11 @@ title: Version 1.1 **Features** -- Add support for MegEngine with examples ([ClearML GitHub issue #455](https://github.com/allegroai/clearml/issues/455)) -- Add `TaskTypes` to main namespace ([ClearML GitHub issue #453](https://github.com/allegroai/clearml/issues/453)) -- Add `LogUnifomParameterRange` for hyperparameter optimization with Optuna ([ClearML GitHub issue #462](https://github.com/allegroai/clearml/issues/462)) +- Add support for MegEngine with examples ([ClearML GitHub issue #455](https://github.com/clearml/clearml/issues/455)) +- Add `TaskTypes` to main namespace ([ClearML GitHub issue #453](https://github.com/clearml/clearml/issues/453)) +- Add `LogUnifomParameterRange` for hyperparameter optimization with Optuna ([ClearML GitHub issue #462](https://github.com/clearml/clearml/issues/462)) - Add joblib (equivalent to scikit) to `Task.init(auto_connect_frameworks)` argument -- Log environment variables starting with `*` in `environ_bind.py` ([ClearML GitHub issue #459](https://github.com/allegroai/clearml/issues/459)) +- Log environment variables starting with `*` in `environ_bind.py` ([ClearML GitHub issue #459](https://github.com/clearml/clearml/issues/459)) - Pipeline - Add eager decorated pipeline execution - Support pipeline monitoring for scalars/models/artifacts @@ -142,13 +142,13 @@ title: Version 1.1 **Bug Fixes** - Fix PyJWT resiliency support -- Fix xgb train overload ([ClearML GitHub issue #456](https://github.com/allegroai/clearml/issues/456)) -- Fix `http://` throws `OSError` in Windows by using `pathlib2` instead of `os` ([ClearML GitHub issue #463](https://github.com/allegroai/clearml/issues/463)) -- Fix local diff should include staged commits, otherwise applying git diff fails ([ClearML GitHub issue #457](https://github.com/allegroai/clearml/issues/457)) -- Fix `task.upload_artifact` non-standard dictionary will now revert to `pickle` ([ClearML GitHub issue #452](https://github.com/allegroai/clearml/issues/452)) -- Fix `S3BucketConfig.is_valid()` for EC2 environments with `use_credentials_chain` ([ClearML GitHub issue #478](https://github.com/allegroai/clearml/issues/478)) -- Fix audio classifier example when training with a custom dataset ([ClearML GitHub issue #484](https://github.com/allegroai/clearml/issues/484)) -- Fix `clearml-task` diff was corrupted by Windows drive letter and separator ([ClearML GitHub issue #483](https://github.com/allegroai/clearml/issues/483)) +- Fix xgb train overload ([ClearML GitHub issue #456](https://github.com/clearml/clearml/issues/456)) +- Fix `http://` throws `OSError` in Windows by using `pathlib2` instead of `os` ([ClearML GitHub issue #463](https://github.com/clearml/clearml/issues/463)) +- Fix local diff should include staged commits, otherwise applying git diff fails ([ClearML GitHub issue #457](https://github.com/clearml/clearml/issues/457)) +- Fix `task.upload_artifact` non-standard dictionary will now revert to `pickle` ([ClearML GitHub issue #452](https://github.com/clearml/clearml/issues/452)) +- Fix `S3BucketConfig.is_valid()` for EC2 environments with `use_credentials_chain` ([ClearML GitHub issue #478](https://github.com/clearml/clearml/issues/478)) +- Fix audio classifier example when training with a custom dataset ([ClearML GitHub issue #484](https://github.com/clearml/clearml/issues/484)) +- Fix `clearml-task` diff was corrupted by Windows drive letter and separator ([ClearML GitHub issue #483](https://github.com/clearml/clearml/issues/483)) - Fix TQDM "line cleanup" not using `CR` but rather arrow-up escape sequence - Fix `task.connect(dict)` value casting - if `None` is the default value, use backend stored type - Fix Jupyter notebook should always set Task as completed/stopped, never failed (exceptions are caught in interactive session) @@ -196,9 +196,9 @@ title: Version 1.1 - Add Task Cron Scheduler - Add PipelineController from function - Add PipelineDecorator (`PipelineDecorator.pipeline` and `PipelineDecorator.component` decorators for full custom pipeline logic) -- Add xgboost auto metric logging ([ClearML GitHub issue #381](https://github.com/allegroai/clearml/issues/381)) -- Add `sdk.storage.log.report_upload_chunk_size_mb` and `sdk.storage.log.report_download_chunk_size_mb` configuration options to control upload/download log reporting ([ClearML GitHub issue #424](https://github.com/allegroai/clearml/issues/424)) -- Add new optional `auto_connect_frameworks` argument value to `Task.init()` (e.g. `auto_connect_frameworks={'tfdefines':False}`) to allow disabling TF defines ([ClearML GitHub issue #408](https://github.com/allegroai/clearml/issues/408)) +- Add xgboost auto metric logging ([ClearML GitHub issue #381](https://github.com/clearml/clearml/issues/381)) +- Add `sdk.storage.log.report_upload_chunk_size_mb` and `sdk.storage.log.report_download_chunk_size_mb` configuration options to control upload/download log reporting ([ClearML GitHub issue #424](https://github.com/clearml/clearml/issues/424)) +- Add new optional `auto_connect_frameworks` argument value to `Task.init()` (e.g. `auto_connect_frameworks={'tfdefines':False}`) to allow disabling TF defines ([ClearML GitHub issue #408](https://github.com/clearml/clearml/issues/408)) - Add support for `CLEARNL_CONFIG_VERBOSE` environment variable to allow external control over verbosity of the configuration loading process - Add support for uploading artifacts with a list of files using `Task.upload_artifcats(name, [Path(), Path()])` - Add missing *clearml-task* parameters `--docker_args`, `--docker_bash_setup_script` and `--output-uri` @@ -219,18 +219,18 @@ title: Version 1.1 **Bug Fixes** -- Fix plotly plots converting `NaN` to `nan` instead of `null` ([ClearML GitHub issue #373](https://github.com/allegroai/clearml/issues/373)) -- Fix deprecation warning ([ClearML GitHub issue #376](https://github.com/allegroai/clearml/issues/376)) -- Fix plotly multi-index without index names ([ClearML GitHub issue #399](https://github.com/allegroai/clearml/issues/399)) -- Fix click support ([ClearML GitHub issue #437](https://github.com/allegroai/clearml/issues/437)) -- Fix docstring ([ClearML GitHub issue #438](https://github.com/allegroai/clearml/issues/438)) -- Fix passing `task-type` to *clearml-task* ([ClearML GitHub issue #422](https://github.com/allegroai/clearml/issues/422)) -- Fix `clearml-task --version` throws an error ([ClearML GitHub issue #422](https://github.com/allegroai/clearml/issues/422)) -- Fix *clearml-task* ssh repository links are not detected as remote repositories ([ClearML GitHub issue #423](https://github.com/allegroai/clearml/issues/423)) -- Fix `getattr` throws an exception ([ClearML GitHub issue #426](https://github.com/allegroai/clearml/issues/426)) -- Fix encoding while saving notebook preview ([ClearML GitHub issue #443](https://github.com/allegroai/clearml/issues/443)) -- Fix poetry toml file without requirements.txt ([ClearML GitHub issue #444](https://github.com/allegroai/clearml/issues/444)) -- Fix `PY3.x` fails calling `SemLock._after_fork` with forkserver context, forking while lock is acquired ([ClearML Agent GitHub issue #73](https://github.com/allegroai/clearml-agent/issues/73)) +- Fix plotly plots converting `NaN` to `nan` instead of `null` ([ClearML GitHub issue #373](https://github.com/clearml/clearml/issues/373)) +- Fix deprecation warning ([ClearML GitHub issue #376](https://github.com/clearml/clearml/issues/376)) +- Fix plotly multi-index without index names ([ClearML GitHub issue #399](https://github.com/clearml/clearml/issues/399)) +- Fix click support ([ClearML GitHub issue #437](https://github.com/clearml/clearml/issues/437)) +- Fix docstring ([ClearML GitHub issue #438](https://github.com/clearml/clearml/issues/438)) +- Fix passing `task-type` to *clearml-task* ([ClearML GitHub issue #422](https://github.com/clearml/clearml/issues/422)) +- Fix `clearml-task --version` throws an error ([ClearML GitHub issue #422](https://github.com/clearml/clearml/issues/422)) +- Fix *clearml-task* ssh repository links are not detected as remote repositories ([ClearML GitHub issue #423](https://github.com/clearml/clearml/issues/423)) +- Fix `getattr` throws an exception ([ClearML GitHub issue #426](https://github.com/clearml/clearml/issues/426)) +- Fix encoding while saving notebook preview ([ClearML GitHub issue #443](https://github.com/clearml/clearml/issues/443)) +- Fix poetry toml file without requirements.txt ([ClearML GitHub issue #444](https://github.com/clearml/clearml/issues/444)) +- Fix `PY3.x` fails calling `SemLock._after_fork` with forkserver context, forking while lock is acquired ([ClearML Agent GitHub issue #73](https://github.com/clearml/clearml-agent/issues/73)) - Fix wrong download path in `StorageManager.download_folder()` - Fix jupyter notebook `display(...)` convert to `print(...)` - Fix TensorFlow `add_image()` with `description='text'` diff --git a/docs/release_notes/sdk/open_source/ver_1_10.md b/docs/release_notes/sdk/open_source/ver_1_10.md index 9a8cc480..6a7690f6 100644 --- a/docs/release_notes/sdk/open_source/ver_1_10.md +++ b/docs/release_notes/sdk/open_source/ver_1_10.md @@ -5,12 +5,12 @@ title: Version 1.10 ### ClearML 1.10.4 **New Features and Improvements** -* Update references to `GetAllRequest` in Task docstrings ([ClearML GitHub issue #981](https://github.com/allegroai/clearml/issues/981)) +* Update references to `GetAllRequest` in Task docstrings ([ClearML GitHub issue #981](https://github.com/clearml/clearml/issues/981)) **Bug Fixes** -* Fix issue with JSON conversion when using `jsonargparse` during remote execution ([ClearML GitHub issue #1010](https://github.com/allegroai/clearml/issues/1010) -and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent/issues/153)) -* Fix `ProxyDictPostWrite.update()` not triggering a write-back to the backend ([ClearML GitHub issue #985](https://github.com/allegroai/clearml/issues/985)) +* Fix issue with JSON conversion when using `jsonargparse` during remote execution ([ClearML GitHub issue #1010](https://github.com/clearml/clearml/issues/1010) +and [ClearML Agent GitHub issue #153](https://github.com/clearml/clearml-agent/issues/153)) +* Fix `ProxyDictPostWrite.update()` not triggering a write-back to the backend ([ClearML GitHub issue #985](https://github.com/clearml/clearml/issues/985)) * Fix standard streams being patched in subtasks, which caused the streams to be reported to the wrong task * Fix TypeError when using `Task.query_tasks()` without specifying the `task_filter` parameter * Fix Tensorboard negative iterations, they are now zeroed @@ -25,14 +25,14 @@ and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent ### ClearML 1.10.2 **New Features and Improvements** -* Allow task filtering with `Task.query_tasks()` using `TaskTypes` ([ClearML GitHub issue #938](https://github.com/allegroai/clearml/issues/938)) +* Allow task filtering with `Task.query_tasks()` using `TaskTypes` ([ClearML GitHub issue #938](https://github.com/clearml/clearml/issues/938)) * Allow adding tags to individual pipeline steps and to the `PipelineController` * Restrict `Dataset.create()` usage for old server API versions * Improve docstrings **Bug Fixes** -* Fix `Task.get_tasks` docstring ([ClearML GitHub issue #937](https://github.com/allegroai/clearml/issues/937)) -* Fix `Task.get_reported_scalars()` docstring for `x_axis` parameter's `"timestamp"` value ([ClearML GitHub issue #963](https://github.com/allegroai/clearml/issues/963)) +* Fix `Task.get_tasks` docstring ([ClearML GitHub issue #937](https://github.com/clearml/clearml/issues/937)) +* Fix `Task.get_reported_scalars()` docstring for `x_axis` parameter's `"timestamp"` value ([ClearML GitHub issue #963](https://github.com/clearml/clearml/issues/963)) * Fix missing plots on models in Windows * Fix Pipeline version auto-bumping * Fix `APIClient.events.debug_images()` for latest server API version @@ -40,7 +40,7 @@ and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent ### ClearML 1.10.1 **New Features and Improvements** -* Add cast parameter to Task.get_parameter method ([ClearML GitHub issue #951](https://github.com/allegroai/clearml/issues/951)) +* Add cast parameter to Task.get_parameter method ([ClearML GitHub issue #951](https://github.com/clearml/clearml/issues/951)) **Bug Fixes** * Fix backward compatibility issue when using datetime filtering with `APIClient().models.get_all` and `APIClient().tasks.get_all` requests @@ -50,13 +50,13 @@ and [ClearML Agent GitHub issue #153](https://github.com/allegroai/clearml-agent **Breaking Changes** * `StorageManager.download_file()` and `StorageManager.download_folder()` from MinIO and Azure no longer include bucket name -in target local path ([ClearML GitHub issue #709](https://github.com/allegroai/clearml/issues/709)) +in target local path ([ClearML GitHub issue #709](https://github.com/clearml/clearml/issues/709)) **New Features and Improvements** * Add support for attaching plots and scalars to models -* Add `Task.get_debug_sample()` to retrieve task's debug samples ([ClearML GitHub issue #761](https://github.com/allegroai/clearml/issues/761)) -* Add custom artifact serialization for pipelines ([ClearML GitHub issue #689](https://github.com/allegroai/clearml/issues/689)) -* Add table layout configuration to `Logger.report_table()` through `extra_data` parameter ([ClearML GitHub issue #796](https://github.com/allegroai/clearml/issues/796)) +* Add `Task.get_debug_sample()` to retrieve task's debug samples ([ClearML GitHub issue #761](https://github.com/clearml/clearml/issues/761)) +* Add custom artifact serialization for pipelines ([ClearML GitHub issue #689](https://github.com/clearml/clearml/issues/689)) +* Add table layout configuration to `Logger.report_table()` through `extra_data` parameter ([ClearML GitHub issue #796](https://github.com/clearml/clearml/issues/796)) * Add `PipelineController.connect_configuration()` to add configuration dictionaries to a pipeline * Add `Dataset.delete()` configurable hosted contents deletion support: * Delete ClearML file server hosted files @@ -64,18 +64,18 @@ in target local path ([ClearML GitHub issue #709](https://github.com/allegroai/c * Add `force_download` argument to `Model.get_local_copy()` to force download a new copy of a model even if already in cache * Add `PipelineController.is_successful()` criteria customization * Add support for API v2.23 -* Add `Getting Started` Notebooks ([ClearML PR #948](https://github.com/allegroai/clearml/pull/948)) -* Improve `hyper_parameter` example ([ClearML PR #927](https://github.com/allegroai/clearml/pull/927)) -* Correct `Task.connect()` docstring ([ClearML Docs issue #473](https://github.com/allegroai/clearml-DOCS/issues/473)) -* Improve the `Task.mark_completed()` and `Task.close()` docstrings ([ClearML Docs issue #486](https://github.com/allegroai/clearml-docs/issues/486)) +* Add `Getting Started` Notebooks ([ClearML PR #948](https://github.com/clearml/clearml/pull/948)) +* Improve `hyper_parameter` example ([ClearML PR #927](https://github.com/clearml/clearml/pull/927)) +* Correct `Task.connect()` docstring ([ClearML Docs issue #473](https://github.com/clearml/clearml-DOCS/issues/473)) +* Improve the `Task.mark_completed()` and `Task.close()` docstrings ([ClearML Docs issue #486](https://github.com/clearml/clearml-docs/issues/486)) **Bug Fixes** -* Fix logging dependencies that use the subdirectory argument when pip installing from a git repo ([ClearML GitHub issue #946](https://github.com/allegroai/clearml/issues/946)) -* Fix `Task.import_offline_session()` does not import offline models ([ClearML GitHub issue #653](https://github.com/allegroai/clearml/issues/653)) -* Fix `clearml-init` incorrectly sets Web and API server ports ([ClearML Server GitHub issue #181](https://github.com/allegroai/clearml-server/issues/181) and [ClearML GitHub issue #910](https://github.com/allegroai/clearml/issues/910)) -* Fix multiple models trained by the same framework are not all automatically logged ([ClearML GitHub issue #767](https://github.com/allegroai/clearml/issues/767)) -* Fix parallel coordinates plot displays categorical variables unclearly ([ClearML GitHub issue #907](https://github.com/allegroai/clearml/issues/907)) +* Fix logging dependencies that use the subdirectory argument when pip installing from a git repo ([ClearML GitHub issue #946](https://github.com/clearml/clearml/issues/946)) +* Fix `Task.import_offline_session()` does not import offline models ([ClearML GitHub issue #653](https://github.com/clearml/clearml/issues/653)) +* Fix `clearml-init` incorrectly sets Web and API server ports ([ClearML Server GitHub issue #181](https://github.com/clearml/clearml-server/issues/181) and [ClearML GitHub issue #910](https://github.com/clearml/clearml/issues/910)) +* Fix multiple models trained by the same framework are not all automatically logged ([ClearML GitHub issue #767](https://github.com/clearml/clearml/issues/767)) +* Fix parallel coordinates plot displays categorical variables unclearly ([ClearML GitHub issue #907](https://github.com/clearml/clearml/issues/907)) * Fix runtime toggling task offline mode in the context of an open task * Fix information not being captured from Jupyter Notebook in Sagemaker studio * Fix `python-fire` support diff --git a/docs/release_notes/sdk/open_source/ver_1_11.md b/docs/release_notes/sdk/open_source/ver_1_11.md index 6d8a7044..7e1664f0 100644 --- a/docs/release_notes/sdk/open_source/ver_1_11.md +++ b/docs/release_notes/sdk/open_source/ver_1_11.md @@ -5,9 +5,9 @@ title: Version 1.11 ### ClearML 1.11.1 **New Features and Improvements** -* Support `lightning>=2.0` ([ClearML GitHub issue #1032](https://github.com/allegroai/clearml/issues/1032)) +* Support `lightning>=2.0` ([ClearML GitHub issue #1032](https://github.com/clearml/clearml/issues/1032)) * Add `clearml-init` support for standard HTTP and HTTPS ports in webapp in conjunction with non-default api/files server ports - ([ClearML Server GitHub issue #198](https://github.com/allegroai/clearml-server/issues/198)) + ([ClearML Server GitHub issue #198](https://github.com/clearml/clearml-server/issues/198)) * Improve docstrings **Bug Fixes** @@ -20,8 +20,8 @@ title: Version 1.11 ### ClearML 1.11.0 **New Features and Improvements** -* Add offline mode for ClearML Datasets ([ClearML GitHub issue #854](https://github.com/allegroai/clearml/issues/854)) -* Improve performance of `Dataset.add_external_files()` ([ClearML GitHub PR #962](https://github.com/allegroai/clearml/pull/962)) +* Add offline mode for ClearML Datasets ([ClearML GitHub issue #854](https://github.com/clearml/clearml/issues/854)) +* Improve performance of `Dataset.add_external_files()` ([ClearML GitHub PR #962](https://github.com/clearml/clearml/pull/962)) * Add `Task.launch_multi_node()` for distributed experiment execution * Add `Task.get_all_reported_scalars()` to fetch all available scalar data * Add `Task.remove_input_models()` to disassociate input models from a task @@ -30,10 +30,10 @@ title: Version 1.11 * Add support for automatic Dataset content preview for JSON files **Bug Fixes** -* Fix multiple local parallel experiments conflicting on storage access validation ([ClearML GitHub issue #897](https://github.com/allegroai/clearml/issues/897)) -* Fix code running with `joblib` hangs ([ClearML GitHub issue #1009](https://github.com/allegroai/clearml/issues/1009)) -* Fix comment for storage scheme `gs://` ([ClearML GitHub issue #986](https://github.com/allegroai/clearml/issues/986)) -* Fix typo "Categories per column" ([ClearML GitHub PR #1021](https://github.com/allegroai/clearml/pull/1021)) +* Fix multiple local parallel experiments conflicting on storage access validation ([ClearML GitHub issue #897](https://github.com/clearml/clearml/issues/897)) +* Fix code running with `joblib` hangs ([ClearML GitHub issue #1009](https://github.com/clearml/clearml/issues/1009)) +* Fix comment for storage scheme `gs://` ([ClearML GitHub issue #986](https://github.com/clearml/clearml/issues/986)) +* Fix typo "Categories per column" ([ClearML GitHub PR #1021](https://github.com/clearml/clearml/pull/1021)) * Fix downloading artifacts from GCP when the download URI includes unsafe characters * Fix pipeline step cache is not cleared when the only change to the step is that the docker image changed * Fix `Task.report_text()` sometimes reporting to an incorrect task when multiple tasks run simultaneously diff --git a/docs/release_notes/sdk/open_source/ver_1_12.md b/docs/release_notes/sdk/open_source/ver_1_12.md index d9aa4224..0a425c91 100644 --- a/docs/release_notes/sdk/open_source/ver_1_12.md +++ b/docs/release_notes/sdk/open_source/ver_1_12.md @@ -9,7 +9,7 @@ title: Version 1.12 the Pipeline DAG at runtime (as opposed to adhering to the existing structure stored on the task when running locally) **Bug Fixes** -* Fix using `OutputModel` reporting and other methods fail if `OutputModel.update_weights()` is not called before that ([ClearML GitHub issue #1078](https://github.com/allegroai/clearml/issues/1078)) +* Fix using `OutputModel` reporting and other methods fail if `OutputModel.update_weights()` is not called before that ([ClearML GitHub issue #1078](https://github.com/clearml/clearml/issues/1078)) * Fix hash for Pandas objects artifacts * Fix Hydra/OmegaConf `--overrides` support * Fix Pipeline metrics not forwarded to pipeline controller at the end of a pipeline step @@ -17,32 +17,32 @@ the Pipeline DAG at runtime (as opposed to adhering to the existing structure st ### ClearML 1.12.1 **New Features and Improvements** -* Raise error if dataset name is empty ([ClearML GitHub PR #1083](https://github.com/allegroai/clearml/pull/1083)) +* Raise error if dataset name is empty ([ClearML GitHub PR #1083](https://github.com/clearml/clearml/pull/1083)) * Add support for `spawn` start method for Python `multiprocess`. This should help circumvent issues like [this](https://github.com/python/cpython/issues/84559). **Bug Fix** -* Fix S3 uploads fail with `LazyEvalWrapper` type error ([ClearML GitHub issue #1081](https://github.com/allegroai/clearml/issues/1081)) +* Fix S3 uploads fail with `LazyEvalWrapper` type error ([ClearML GitHub issue #1081](https://github.com/clearml/clearml/issues/1081)) ### ClearML 1.12.0 :::info BREAKING CHANGES -As a result of fixing [issue #1051](https://github.com/allegroai/clearml/issues/1051) and similar issues with retrieving +As a result of fixing [issue #1051](https://github.com/clearml/clearml/issues/1051) and similar issues with retrieving task data/models/artifacts/samples stored in GCS, we introduced some breaking changes. To access resources registered to GCS from ClearML SDK versions 1.11.0 to 1.11.1, see -the instructions [here](https://github.com/allegroai/clearml/tree/master/docs/errata_breaking_change_gcs_sdk_1_11_x.md). +the instructions [here](https://github.com/clearml/clearml/tree/master/docs/errata_breaking_change_gcs_sdk_1_11_x.md). ::: **New Features and Improvements** -* Add `include_archive` parameter to `Dataset.list_datasets()`: include archived datasets in list ([ClearML GitHub issue #1067](https://github.com/allegroai/clearml/issues/1067)) +* Add `include_archive` parameter to `Dataset.list_datasets()`: include archived datasets in list ([ClearML GitHub issue #1067](https://github.com/clearml/clearml/issues/1067)) * Add support to specify the multipart chunk size and threshold using the `aws.boto3.multipart_chunksize` and -`aws.boto3.multipart_threshold` configuration options in the clearml.conf ([ClearML GitHub issue #1058](https://github.com/allegroai/clearml/issues/1058)) +`aws.boto3.multipart_threshold` configuration options in the clearml.conf ([ClearML GitHub issue #1058](https://github.com/clearml/clearml/issues/1058)) * Add `PipelineController.get_pipeline()` for retrieving previously run pipelines. **Bug Fixes** -* Fix AWS driver issues ([ClearML GitHub PR #1000](https://github.com/allegroai/clearml/pull/1000)): +* Fix AWS driver issues ([ClearML GitHub PR #1000](https://github.com/clearml/clearml/pull/1000)): * Fix credential authentication failure when attempting to use token * Fix instantiation within VPC without AvailabilityZones -* Fix `continue_last_task=0` is ignored in pipelines run with `retry_on_failure` ([ClearML GitHub issue #1054](https://github.com/allegroai/clearml/issues/1054)) +* Fix `continue_last_task=0` is ignored in pipelines run with `retry_on_failure` ([ClearML GitHub issue #1054](https://github.com/clearml/clearml/issues/1054)) * Fix `Task.connect_configuration()` doesn't handle dictionaries with special characters * Fix pipeline steps created with `PipelineDecorator` aren't cached * Fix `Task.get_by_name()` doesn't return the most recent task when multiple tasks have same name diff --git a/docs/release_notes/sdk/open_source/ver_1_13.md b/docs/release_notes/sdk/open_source/ver_1_13.md index 954d2c0b..0994ff93 100644 --- a/docs/release_notes/sdk/open_source/ver_1_13.md +++ b/docs/release_notes/sdk/open_source/ver_1_13.md @@ -6,27 +6,27 @@ title: Version 1.13 **New Features and Improvements** -* Support parallel dataset verification ([ClearML GitHub issue #1130](https://github.com/allegroai/clearml/issues/1130)) -* Support `lightgbm` model loading from a string ([ClearML GitHub issue #1135](https://github.com/allegroai/clearml/issues/1135)) +* Support parallel dataset verification ([ClearML GitHub issue #1130](https://github.com/clearml/clearml/issues/1130)) +* Support `lightgbm` model loading from a string ([ClearML GitHub issue #1135](https://github.com/clearml/clearml/issues/1135)) * Add support for Hydra command-line syntax for modifying `omegaconf` * Add support for setting `output_uri` in pipeline decorators and functions * Add more visibility when overriding `jsonargparse` arguments * Use `StorageManager` cache when downloading models * Support parsing GS credentials as a JSON string if they do not point to a JSON file * Support ignoring `default_output_uri` on remote runs when `Task.init(output_uri=False)` was specified and the backend task data has no `output_uri` specified -* Add configuration files logging [example](https://github.com/allegroai/clearml/blob/e6f1cc8a5c65f7007cf6452745d09795e776a548/examples/reporting/config_files.py) +* Add configuration files logging [example](https://github.com/clearml/clearml/blob/e6f1cc8a5c65f7007cf6452745d09795e776a548/examples/reporting/config_files.py) * Support raising `LoginError` when initializing -* Add Hugging Face Transformers example ([ClearML GitHub PR #1146](https://github.com/allegroai/clearml/pull/1146)) +* Add Hugging Face Transformers example ([ClearML GitHub PR #1146](https://github.com/clearml/clearml/pull/1146)) * Improve docstrings for tag filters -* Add a security policy file to the repository ([ClearML GitHub issue #1148](https://github.com/allegroai/clearml/issues/1148)) +* Add a security policy file to the repository ([ClearML GitHub issue #1148](https://github.com/clearml/clearml/issues/1148)) **Bug Fixes** -* Fix Google Storage subdir path in bucket URL breaks validation ([ClearML GitHub PR #1117](https://github.com/allegroai/clearml/pull/1117)) -* Fix jsonschema `DeprecationWarning` in newer versions ([ClearML GitHub PR #1128](https://github.com/allegroai/clearml/pull/1128)) -* Fix `task.set_script(diff="")` does not clear the uncommitted changes ([ClearML GitHub issue #1106](https://github.com/allegroai/clearml/issues/1106)) -* Fix parallel dataset verification fails on older Python versions ([ClearML GitHub issue #1144](https://github.com/allegroai/clearml/issues/1144)) -* Fix `attrs.__version__` deprecation warning ([ClearML GitHub issue #1145](https://github.com/allegroai/clearml/issues/1145)) +* Fix Google Storage subdir path in bucket URL breaks validation ([ClearML GitHub PR #1117](https://github.com/clearml/clearml/pull/1117)) +* Fix jsonschema `DeprecationWarning` in newer versions ([ClearML GitHub PR #1128](https://github.com/clearml/clearml/pull/1128)) +* Fix `task.set_script(diff="")` does not clear the uncommitted changes ([ClearML GitHub issue #1106](https://github.com/clearml/clearml/issues/1106)) +* Fix parallel dataset verification fails on older Python versions ([ClearML GitHub issue #1144](https://github.com/clearml/clearml/issues/1144)) +* Fix `attrs.__version__` deprecation warning ([ClearML GitHub issue #1145](https://github.com/clearml/clearml/issues/1145)) * Fix `Task.get_parameters_as_dict(cast=True)` raises a YAML parser error * Fix potential threading issues when setting `self.lock` to `None` * Fix pipeline version is not displayed in the table view and version list @@ -41,21 +41,21 @@ title: Version 1.13 ### ClearML 1.13.1 **Bug Fixes** -* Fix Hydra overrides integration ([ClearML GitHub issue #1120](https://github.com/allegroai/clearml/issues/1120)) +* Fix Hydra overrides integration ([ClearML GitHub issue #1120](https://github.com/clearml/clearml/issues/1120)) * Fix GPU memory used reports 0 when memory can not be queried per process (edge case) ### ClearML 1.13.0 **New Features** -* Add support for recursive list, dict, and tuple ref parsing for `PipelineController.add step()` parameter overrides ([ClearML GitHub issue #1089](https://github.com/allegroai/clearml/issues/1089)) +* Add support for recursive list, dict, and tuple ref parsing for `PipelineController.add step()` parameter overrides ([ClearML GitHub issue #1089](https://github.com/clearml/clearml/issues/1089)) * Update PyNVML to the latest NVIDIA version for better GPU monitoring * Add `force_single_script_file` argument to `Task.create()` to avoid git repository auto detection * Use `os.register_at_fork` instead of monkey patching fork for `python>3.6` * Add support to programmatically archive and unarchive models from the model registry using the `Model.archive()` and -`Model.unarchive()` methods ([ClearML GitHub issue #1096](https://github.com/allegroai/clearml/issues/1096)) +`Model.unarchive()` methods ([ClearML GitHub issue #1096](https://github.com/clearml/clearml/issues/1096)) **Bug Fixes** -* Fix model reporting in TensorFlow 2.13 does not work properly ([ClearML GitHub issue #1112](https://github.com/allegroai/clearml/issues/1112)) +* Fix model reporting in TensorFlow 2.13 does not work properly ([ClearML GitHub issue #1112](https://github.com/clearml/clearml/issues/1112)) * Fix GPU memory used reports 0 when memory can not be queried per process * Fix FastAI binding not reporting scalars if TensorBoard is installed * Fix JSON serialization error does not raise exception diff --git a/docs/release_notes/sdk/open_source/ver_1_14.md b/docs/release_notes/sdk/open_source/ver_1_14.md index 046eb183..1d3d52be 100644 --- a/docs/release_notes/sdk/open_source/ver_1_14.md +++ b/docs/release_notes/sdk/open_source/ver_1_14.md @@ -5,7 +5,7 @@ title: Version 1.14 ### ClearML 1.14.4 **Bug Fixes** -* Fix AWS S3 configuration `secure` property not forwarded in `StorageHelper` ([ClearML GitHub PR #1205](https://github.com/allegroai/clearml/pull/1205)) +* Fix AWS S3 configuration `secure` property not forwarded in `StorageHelper` ([ClearML GitHub PR #1205](https://github.com/clearml/clearml/pull/1205)) * Fix `NVIDIA_VISIBLE_DEVICES=all` env var value will result in no GPU metrics being reported * Fix setting `output_uri` when an AWS S3 config section with `secure=false` exists raises an error * Fix `CacheManager` attempts to clear cache when file limit has not been reached resulting in slow unnecessary checks @@ -14,7 +14,7 @@ title: Version 1.14 **New Features and Bug Fixes** * Add hash check for pickle files (resolves [CVE-2024-24590](https://github.com/advisories/GHSA-cpcw-9h9m-wqw9)) -* Fix `clearml-task` relative paths calculation when `--cwd`, `--folder`, and `--script` are provided ([ClearML GitHub issue #1160](https://github.com/allegroai/clearml/issues/1160)) +* Fix `clearml-task` relative paths calculation when `--cwd`, `--folder`, and `--script` are provided ([ClearML GitHub issue #1160](https://github.com/clearml/clearml/issues/1160)) * Fix regression causing an error when `secure` is provided in AWS S3 bucket settings ### ClearML 1.14.2 @@ -27,10 +27,10 @@ title: Version 1.14 disable overriding values when running remotely * Add support for setting a custom certificate path using the `CLEARML_API_HOST_VERIFY_CERT` environment variable (instead of just allowing the user to disable verification) -* Add dataset reporting example ([ClearML GitHub PR #1180](https://github.com/allegroai/clearml/pull/1180)) +* Add dataset reporting example ([ClearML GitHub PR #1180](https://github.com/clearml/clearml/pull/1180)) **Bug Fixes** -* Fix missing component callbacks on multiple step calls ([ClearML GitHub PR #1195](https://github.com/allegroai/clearml/pull/1195)) +* Fix missing component callbacks on multiple step calls ([ClearML GitHub PR #1195](https://github.com/clearml/clearml/pull/1195)) * Fix unsafe usage of `tempfile.mktemp` * Fix potential path traversal on file download ([CVE-2024-24591](https://github.com/advisories/GHSA-m95h-p4gg-wfw3)) * Fix Python 3.12 support by not using `distutils` @@ -46,7 +46,7 @@ title: Version 1.14 ### ClearML 1.14.0 **New Features and Improvements** -* Add support for decorated pipeline steps ([ClearML GitHub issue #1154](https://github.com/allegroai/clearml/issues/1154)) +* Add support for decorated pipeline steps ([ClearML GitHub issue #1154](https://github.com/clearml/clearml/issues/1154)) * Improve `torch.distributed` behavior and auto-detect Huggingface accelerate distributed execution * Improve `jsonargparse` integration: `allow_config_file_override_from_ui` argument added for enabling/disabling overriding configuration file entries @@ -60,10 +60,10 @@ and can be imported globally instead * Add tabular data multi-stage pipeline process example **Bug Fixes** -* Fix `langchain` integration ([ClearML GitHub issues #1124](https://github.com/allegroai/clearml/issues/1124) and [ClearML GitHub issue #1126](https://github.com/allegroai/clearml/issues/1126)) -* Fix using `target_project` with `TaskScheduler.add_task()` corrupts project if used with pipelines ([ClearML GitHub issue #1137](https://github.com/allegroai/clearml/issues/1137)) -* Fix `PipelineController.add_function_step()` fails when adding custom decorated step ([ClearML GitHub issue #1154](https://github.com/allegroai/clearml/issues/1154)) -* Fix `jsonargparse` binding does not capture parameters before `Task.init()` is called ([ClearML GitHub issue #1164](https://github.com/allegroai/clearml/issues/1164)) +* Fix `langchain` integration ([ClearML GitHub issues #1124](https://github.com/clearml/clearml/issues/1124) and [ClearML GitHub issue #1126](https://github.com/clearml/clearml/issues/1126)) +* Fix using `target_project` with `TaskScheduler.add_task()` corrupts project if used with pipelines ([ClearML GitHub issue #1137](https://github.com/clearml/clearml/issues/1137)) +* Fix `PipelineController.add_function_step()` fails when adding custom decorated step ([ClearML GitHub issue #1154](https://github.com/clearml/clearml/issues/1154)) +* Fix `jsonargparse` binding does not capture parameters before `Task.init()` is called ([ClearML GitHub issue #1164](https://github.com/clearml/clearml/issues/1164)) * Fix invalid `jsonargparse` arguments should be deleted * Fix handling of appended parameters to Hydra defaults list * Fix `Task.set_repo()` and `Task.set_packages()` do not wait for automatic asynchronous repository detection diff --git a/docs/release_notes/sdk/open_source/ver_1_15.md b/docs/release_notes/sdk/open_source/ver_1_15.md index 5517cdd8..2527b5bf 100644 --- a/docs/release_notes/sdk/open_source/ver_1_15.md +++ b/docs/release_notes/sdk/open_source/ver_1_15.md @@ -5,26 +5,26 @@ title: Version 1.15 ### ClearML 1.15.1 **Bug Fixes** -* Fix autoscaler should recheck that the worker is still idle before shutting it down ([ClearML GitHub issue #1202](https://github.com/allegroai/clearml/issues/1202)) +* Fix autoscaler should recheck that the worker is still idle before shutting it down ([ClearML GitHub issue #1202](https://github.com/clearml/clearml/issues/1202)) * Fix resource monitor fails to get GPU stats in some edge cases ### ClearML 1.15.0 **New Features** -* Add draft option to pipeline steps ([ClearML GitHub PR #1226](https://github.com/allegroai/clearml/pull/1226)) -* Add support for custom working directory for pipelines ([ClearML GitHub issue #1194](https://github.com/allegroai/clearml/issues/1194)) +* Add draft option to pipeline steps ([ClearML GitHub PR #1226](https://github.com/clearml/clearml/pull/1226)) +* Add support for custom working directory for pipelines ([ClearML GitHub issue #1194](https://github.com/clearml/clearml/issues/1194)) * Add `Task.get_requirements()` method that returns the task's requirements * Allow controlling the number of threads used by `StorageManager.download_folder()` using the `max_workers` argument * Update examples dependencies * Improve auto populate in `Task.init()` **Bug Fixes** -* Fix UTF-8 script code encoding issue ([ClearML GitHub issue #1207](https://github.com/allegroai/clearml/issues/1207)) -* Fix Colab docs ([ClearML GitHub issue #1204](https://github.com/allegroai/clearml/issues/1204)) -* Fix metrics reporting with `OutputModel` while in offline mode results in an error ([ClearML GitHub issue #1172](https://github.com/allegroai/clearml/issues/1172)) -* Fix task running in Google Colab doesn't properly get the notebook diff ([ClearML GitHub issue #1204](https://github.com/allegroai/clearml/issues/1204)) -* Fix hydra binds break in offline mode ([ClearML GitHub issue #1215](https://github.com/allegroai/clearml/issues/1215)) -* Fix HPO crashes when optimizing for single value scalars ([ClearML GitHub issue #1221](https://github.com/allegroai/clearml/issues/1221)) +* Fix UTF-8 script code encoding issue ([ClearML GitHub issue #1207](https://github.com/clearml/clearml/issues/1207)) +* Fix Colab docs ([ClearML GitHub issue #1204](https://github.com/clearml/clearml/issues/1204)) +* Fix metrics reporting with `OutputModel` while in offline mode results in an error ([ClearML GitHub issue #1172](https://github.com/clearml/clearml/issues/1172)) +* Fix task running in Google Colab doesn't properly get the notebook diff ([ClearML GitHub issue #1204](https://github.com/clearml/clearml/issues/1204)) +* Fix hydra binds break in offline mode ([ClearML GitHub issue #1215](https://github.com/clearml/clearml/issues/1215)) +* Fix HPO crashes when optimizing for single value scalars ([ClearML GitHub issue #1221](https://github.com/clearml/clearml/issues/1221)) * Fix GPU info such as `gpu_memory` and `gpu_type` is not being collected in some cases * Fix `clearml-data` CLI tool moves non-dataset tasks to a `.dataset` project when the respective task is not a dataset * Fix pandas `DataFrame` artifacts with hierarchical indices get mangled by the CSV round-trip diff --git a/docs/release_notes/sdk/open_source/ver_1_16.md b/docs/release_notes/sdk/open_source/ver_1_16.md index 005d5c4d..577e23cc 100644 --- a/docs/release_notes/sdk/open_source/ver_1_16.md +++ b/docs/release_notes/sdk/open_source/ver_1_16.md @@ -12,7 +12,7 @@ uploading artifacts * Add `Task.request_external_endpoint()` to request external endpoints on supported backends **Bug Fixes** -* Fix `clearml-data search` CLI error if dataset version is `None` ([ClearML GitHub issue #1329](https://github.com/allegroai/clearml/issues/1321)) +* Fix `clearml-data search` CLI error if dataset version is `None` ([ClearML GitHub issue #1329](https://github.com/clearml/clearml/issues/1321)) * Fix `maxfile` attribute dropped in `psutil` v6.0.0 causing an error to be printed * Fix `api.auth.req_token_expiration_sec` configuration option to `api.auth.request_token_expiration_sec` (matches agent setting, keep backwards compatibility) * Bump six version due to Python 3.12 issue with `six.moves` @@ -32,8 +32,8 @@ configuration option) ### ClearML 1.16.3 **New Features and Improvements** -* Add `--tags` option to `clearml-task` ([ClearML GitHub issue #1284](https://github.com/allegroai/clearml/issues/1284)) -* Add `retries` parameter to `StorageManager.upload_folder()` ([ClearML GitHub issue #1305](https://github.com/allegroai/clearml/issues/1305)) +* Add `--tags` option to `clearml-task` ([ClearML GitHub issue #1284](https://github.com/clearml/clearml/issues/1284)) +* Add `retries` parameter to `StorageManager.upload_folder()` ([ClearML GitHub issue #1305](https://github.com/clearml/clearml/issues/1305)) * Add `clearml-task` and `CreateAndPopulate` support for bash scripts, ipynb, and python modules (requires `clearml-agent v1.9+`) * Add support for HTTP file upload progress reporting * Add `CLEARML_MULTI_NODE_SINGLE_TASK` (values -1, 0, 1, 2) for easier multi-node single Task workloads @@ -41,8 +41,8 @@ configuration option) * Change `Model.task` property to return connected task **Bug Fixes** -* Fix Kerastuner framework and examples ([ClearML GitHub issue #1279](https://github.com/allegroai/clearml/issues/1279)) -* Fix scalar logging bug with Fire ([ClearML GitHub issue #1300](https://github.com/allegroai/clearml/issues/1300)) +* Fix Kerastuner framework and examples ([ClearML GitHub issue #1279](https://github.com/clearml/clearml/issues/1279)) +* Fix scalar logging bug with Fire ([ClearML GitHub issue #1300](https://github.com/clearml/clearml/issues/1300)) * Fix support passing folder to `Task.get_script_info()` to get the git info * Fix `Task.launch_multi_node()` to enforce the parent of sub-tasks to be the master node 0 task * Fix tensorboard numpy 2.0 incompatibility breaks binding @@ -59,14 +59,14 @@ configuration option) ### ClearML 1.16.2 **New Features and Improvements** -* Make dataset preview optional ([ClearML GitHub issue #1227](https://github.com/allegroai/clearml/issues/1227)) +* Make dataset preview optional ([ClearML GitHub issue #1227](https://github.com/clearml/clearml/issues/1227)) * Add `api.public_ip_ping` (default: `8.8.8.8`) and `api.public_ip_service_urls` (default: `["api.ipify.org";, "ident.me";]`) configuration settings to detect public IP and network interface **Bug Fixes** -* Fix python-fire integration ([ClearML GitHub issue #1268](https://github.com/allegroai/clearml/issues/1268)) -* Fix path substitution for `file://` URIs (ClearML GitHub issues [#1217](https://github.com/allegroai/clearml/issues/1217) -and [#1233](https://github.com/allegroai/clearml/issues/1233)) +* Fix python-fire integration ([ClearML GitHub issue #1268](https://github.com/clearml/clearml/issues/1268)) +* Fix path substitution for `file://` URIs (ClearML GitHub issues [#1217](https://github.com/clearml/clearml/issues/1217) +and [#1233](https://github.com/clearml/clearml/issues/1233)) * Fix numpy 2.0 compatibility (`np.NINF` removed) * Fix no need to recreate reporter if forking and reporting in subprocess * Fix forked detection mechanism @@ -89,9 +89,9 @@ and `sdk.development.worker.max_wait_for_first_iteration_to_start_sec` configura * Add Python 3.12 support **Bug Fixes** -* Fix pytorch-lightning patches ([ClearML GitHub issue #1249](https://github.com/allegroai/clearml/issues/1249)) -* Fix parameter overrides are converted to string when using HPO ([ClearML GitHub issue #975](https://github.com/allegroai/clearml/issues/975)) -* Fix FastAI performance ([ClearML GitHub issue #1234](https://github.com/allegroai/clearml/issues/1234)) +* Fix pytorch-lightning patches ([ClearML GitHub issue #1249](https://github.com/clearml/clearml/issues/1249)) +* Fix parameter overrides are converted to string when using HPO ([ClearML GitHub issue #975](https://github.com/clearml/clearml/issues/975)) +* Fix FastAI performance ([ClearML GitHub issue #1234](https://github.com/clearml/clearml/issues/1234)) * Fix MIG GPU support * Fix AMD GPU metrics collection * Fix Jupyter password might not be used in some protected JupyterLab instances diff --git a/docs/release_notes/sdk/open_source/ver_1_17.md b/docs/release_notes/sdk/open_source/ver_1_17.md index f181a7fe..dd5a58df 100644 --- a/docs/release_notes/sdk/open_source/ver_1_17.md +++ b/docs/release_notes/sdk/open_source/ver_1_17.md @@ -15,7 +15,7 @@ title: Version 1.17 ### ClearML 1.17.0 **New Features** -* Add programmatic pipeline clone using `PipelineController.create()` and `PipelineController.clone()` ([ClearML GitHub issue #1353](https://github.com/allegroai/clearml/issues/1353)) +* Add programmatic pipeline clone using `PipelineController.create()` and `PipelineController.clone()` ([ClearML GitHub issue #1353](https://github.com/clearml/clearml/issues/1353)) * Add Python 3.13 support * Add support for local imports in pipeline steps * Add support for the ClearML HTTP router using `Task.get_http_router()` @@ -23,7 +23,7 @@ title: Version 1.17 * Update `pyjwt` version **Bug Fixes** -* Fix slow handling of cached files with large `cache_file_limit` ([ClearML GitHub issue #1352](https://github.com/allegroai/clearml/issues/1352)) +* Fix slow handling of cached files with large `cache_file_limit` ([ClearML GitHub issue #1352](https://github.com/clearml/clearml/issues/1352)) * Fix pipeline crash when repository is set to a directory * Fix token is not renewed when using an external token (`CLEARML_AUTH_TOKEN`) and no credentials * Fix external files don't download from parent datasets if they have been modified/removed in the child dataset \ No newline at end of file diff --git a/docs/release_notes/sdk/open_source/ver_1_2.md b/docs/release_notes/sdk/open_source/ver_1_2.md index 7553e757..3730182a 100644 --- a/docs/release_notes/sdk/open_source/ver_1_2.md +++ b/docs/release_notes/sdk/open_source/ver_1_2.md @@ -6,38 +6,38 @@ title: Version 1.2 **Bug Fix** -- Fix HTTP download fails constructing URL ([ClearML GitHub issue #593](https://github.com/allegroai/clearml/issues/593)) +- Fix HTTP download fails constructing URL ([ClearML GitHub issue #593](https://github.com/clearml/clearml/issues/593)) ### ClearML 1.2.0 **Features** -- Add fastai v2 support ([ClearML GitHub PR #571](https://github.com/allegroai/clearml/pull/571)) -- Add catboost support ([ClearML GitHub PR #542](https://github.com/allegroai/clearml/pull/542)) -- Add Python Fire support ([ClearML GitHub PR #550](https://github.com/allegroai/clearml/pull/550)) -- Add new Azure Storage driver support ([ClearML GitHub PR #548](https://github.com/allegroai/clearml/pull/548)) -- Add requirements file support in `Task.add_requirements` ([ClearML GitHub PR #575](https://github.com/allegroai/clearml/pull/575)) -- Allow overriding `auto_delete_file` in `Task.update_output_model()` ([ClearML GitHub issue #554](https://github.com/allegroai/clearml/issues/554)) +- Add fastai v2 support ([ClearML GitHub PR #571](https://github.com/clearml/clearml/pull/571)) +- Add catboost support ([ClearML GitHub PR #542](https://github.com/clearml/clearml/pull/542)) +- Add Python Fire support ([ClearML GitHub PR #550](https://github.com/clearml/clearml/pull/550)) +- Add new Azure Storage driver support ([ClearML GitHub PR #548](https://github.com/clearml/clearml/pull/548)) +- Add requirements file support in `Task.add_requirements` ([ClearML GitHub PR #575](https://github.com/clearml/clearml/pull/575)) +- Allow overriding `auto_delete_file` in `Task.update_output_model()` ([ClearML GitHub issue #554](https://github.com/clearml/clearml/issues/554)) - Support `artifact_object` empty string - Add `skip_zero_size_check` to `StorageManager.download_folder()` -- Add support for extra HTTP retry codes (see [here](https://github.com/allegroai/clearml/blob/2c916181b90c784fe0bd267cd67ea915e53e36e4/clearml/backend_api/config/default/api.conf#L29) or use `CLEARML_API_EXTRA_RETRY_CODES`) +- Add support for extra HTTP retry codes (see [here](https://github.com/clearml/clearml/blob/2c916181b90c784fe0bd267cd67ea915e53e36e4/clearml/backend_api/config/default/api.conf#L29) or use `CLEARML_API_EXTRA_RETRY_CODES`) - Add `Task.get_parameters()` cast back to original type - Add callback support to `Task.delete()` - Add autoscaler CPU-only support - Add AWS autoscaler IAM instance profile support - Update examples - - Edit HTML reporting examples ([ClearML GitHub PR #546](https://github.com/allegroai/clearml/pull/546)) - - Add model reporting examples ([ClearML GitHub PR #553](https://github.com/allegroai/clearml/pull/553)) + - Edit HTML reporting examples ([ClearML GitHub PR #546](https://github.com/clearml/clearml/pull/546)) + - Add model reporting examples ([ClearML GitHub PR #553](https://github.com/clearml/clearml/pull/553)) **Bug Fixes** -- Fix `nargs="?"` without type does not properly cast the default value ([ClearML GitHub issue #531](https://github.com/allegroai/clearml/issues/531)) -- Fix using invalid configurations ([ClearML GitHub issue #544](https://github.com/allegroai/clearml/issues/544)) -- Fix extra_layout not passed to report_matrix ([ClearML GitHub issue #559](https://github.com/allegroai/clearml/issues/559)) -- Fix group arguments in click ([ClearML GitHub PR #561](https://github.com/allegroai/clearml/pull/561)) -- Fix no warning when failing to patch argparse ([ClearML GitHub PR #576](https://github.com/allegroai/clearml/pull/576)) -- Fix crash in `Dataset.upload()` when there is nothing to upload ([ClearML GitHub PR #579](https://github.com/allegroai/clearml/pull/579)) -- Fix requirements, refactor and reformat examples ([ClearML GitHub PRs #567](https://github.com/allegroai/clearml/pull/567), [#573](https://github.com/allegroai/clearml/pull/573), [#582](https://github.com/allegroai/clearml/pull/582)) +- Fix `nargs="?"` without type does not properly cast the default value ([ClearML GitHub issue #531](https://github.com/clearml/clearml/issues/531)) +- Fix using invalid configurations ([ClearML GitHub issue #544](https://github.com/clearml/clearml/issues/544)) +- Fix extra_layout not passed to report_matrix ([ClearML GitHub issue #559](https://github.com/clearml/clearml/issues/559)) +- Fix group arguments in click ([ClearML GitHub PR #561](https://github.com/clearml/clearml/pull/561)) +- Fix no warning when failing to patch argparse ([ClearML GitHub PR #576](https://github.com/clearml/clearml/pull/576)) +- Fix crash in `Dataset.upload()` when there is nothing to upload ([ClearML GitHub PR #579](https://github.com/clearml/clearml/pull/579)) +- Fix requirements, refactor and reformat examples ([ClearML GitHub PRs #567](https://github.com/clearml/clearml/pull/567), [#573](https://github.com/clearml/clearml/pull/573), [#582](https://github.com/clearml/clearml/pull/582)) - Auto-scaler - Change confusing log message - Fix AWS tags support @@ -52,5 +52,5 @@ title: Version 1.2 - Fix `clearml-init` changing web host after pasting full credentials - Fix fileserver upload does not support path in URL - Fix crash on semaphore acquire error -- Fix docs and docstrings ([ClearML GitHub PRs #558](https://github.com/allegroai/clearml/pull/558), [#560](https://github.com/allegroai/clearml/pull/560)) +- Fix docs and docstrings ([ClearML GitHub PRs #558](https://github.com/clearml/clearml/pull/558), [#560](https://github.com/clearml/clearml/pull/560)) diff --git a/docs/release_notes/sdk/open_source/ver_1_3.md b/docs/release_notes/sdk/open_source/ver_1_3.md index 60377735..453a3eef 100644 --- a/docs/release_notes/sdk/open_source/ver_1_3.md +++ b/docs/release_notes/sdk/open_source/ver_1_3.md @@ -6,16 +6,16 @@ title: Version 1.3 **New Features and Improvements** -* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/allegroai/clearml/issues/604)) +* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/clearml/clearml/issues/604)) * Add reserved OS environments warning -* Add git credentials to colab example ([ClearML GitHub PR #621](https://github.com/allegroai/clearml/pull/621)) -* Add jsonargparse support ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403)) +* Add git credentials to colab example ([ClearML GitHub PR #621](https://github.com/clearml/clearml/pull/621)) +* Add jsonargparse support ([ClearML GitHub issue #403](https://github.com/clearml/clearml/issues/403)) * Update autokeras example **Bug Fixes** -* Fix sub-project separators are incorrectly quoted in generated URLs ([ClearML GitHub PR #584](https://github.com/allegroai/clearml/pull/584)) -* Revert Optuna deprecation fix ([ClearML GitHub PR #613](https://github.com/allegroai/clearml/pull/613)) +* Fix sub-project separators are incorrectly quoted in generated URLs ([ClearML GitHub PR #584](https://github.com/clearml/clearml/pull/584)) +* Revert Optuna deprecation fix ([ClearML GitHub PR #613](https://github.com/clearml/clearml/pull/613)) * Fix HPO randomly aborts running tasks before the time limit * Fix cloud driver overwrites `agent.extra_docker_arguments` * Fix Pipeline Controller auto-magic framework connect @@ -29,9 +29,9 @@ title: Version 1.3 **Bug Fixes** -* Update Slack SDK requirement ([ClearML GitHub issue #597](https://github.com/allegroai/clearml/issues/597)) -* Fix fork after `task.close()` is called ([ClearML GitHub issue #605](https://github.com/allegroai/clearml/issues/605)) -* Fix Azure storage upload ([ClearML GitHub issue #598](https://github.com/allegroai/clearml/issues/598)) +* Update Slack SDK requirement ([ClearML GitHub issue #597](https://github.com/clearml/clearml/issues/597)) +* Fix fork after `task.close()` is called ([ClearML GitHub issue #605](https://github.com/clearml/clearml/issues/605)) +* Fix Azure storage upload ([ClearML GitHub issue #598](https://github.com/clearml/clearml/issues/598)) * Fix offline mode crash * Fix task delete response not checked * Fix pipeline controller kwargs with list @@ -55,6 +55,6 @@ title: Version 1.3 **Bug Fixes** -* Fix failed catboost bind on GPU ([ClearML GitHub issue #592](https://github.com/allegroai/clearml/issues/592)) +* Fix failed catboost bind on GPU ([ClearML GitHub issue #592](https://github.com/clearml/clearml/issues/592)) * Fix Optuna `n_jobs` deprecation warning * Fix invalid method called on `delete()` error diff --git a/docs/release_notes/sdk/open_source/ver_1_4.md b/docs/release_notes/sdk/open_source/ver_1_4.md index 2d3d95e3..0df23ad0 100644 --- a/docs/release_notes/sdk/open_source/ver_1_4.md +++ b/docs/release_notes/sdk/open_source/ver_1_4.md @@ -5,16 +5,16 @@ title: Version 1.4 ### ClearML 1.4.1 **Bug Fix** -* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674)) +* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/clearml/clearml/issues/674)) ### ClearML 1.4.0 **New Features** -* Add OpenMMLab example ([ClearML GitHub issue #654](https://github.com/allegroai/clearml/issues/654)) -* Add support for saving artifacts with different formats ([ClearML GitHub issue #634](https://github.com/allegroai/clearml/issues/634)) -* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/allegroai/clearml/issues/604)) -* Support more than 500 results in `Task.get_tasks()` using the `fetch_only_first_page` argument ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612)) -* Support links in `clearml-data` ([ClearML GitHub issue #585](https://github.com/allegroai/clearml/issues/585)) +* Add OpenMMLab example ([ClearML GitHub issue #654](https://github.com/clearml/clearml/issues/654)) +* Add support for saving artifacts with different formats ([ClearML GitHub issue #634](https://github.com/clearml/clearml/issues/634)) +* Add support for setting reported values for `NaN` and `Inf` ([ClearML GitHub issue #604](https://github.com/clearml/clearml/issues/604)) +* Support more than 500 results in `Task.get_tasks()` using the `fetch_only_first_page` argument ([ClearML GitHub issue #612](https://github.com/clearml/clearml/issues/612)) +* Support links in `clearml-data` ([ClearML GitHub issue #585](https://github.com/clearml/clearml/issues/585)) * Support deferred task initialization using `Task.init()` argument `deferred_init` (beta feature) * Support resuming experiments when importing an Offline session * Add `--import-offline-session` command line option to `clearml-task` @@ -28,15 +28,15 @@ title: Version 1.4 * Improve Dataset version table * Add warning to Dataset creation on current Task * Examples and documentation - * Add manual seaborn logging example ([ClearML GitHub PR #628](https://github.com/allegroai/clearml/pull/628)) + * Add manual seaborn logging example ([ClearML GitHub PR #628](https://github.com/clearml/clearml/pull/628)) * Change package author - * Change pipeline example to run locally ([ClearML GitHub PR #642](https://github.com/allegroai/clearml/pull/642)) - * Update PyTorch Lightning example for `pytorch-lightning>=v1.6.0` ([ClearML GitHub PR #650](https://github.com/allegroai/clearml/pull/650)) + * Change pipeline example to run locally ([ClearML GitHub PR #642](https://github.com/clearml/clearml/pull/642)) + * Update PyTorch Lightning example for `pytorch-lightning>=v1.6.0` ([ClearML GitHub PR #650](https://github.com/clearml/clearml/pull/650)) **Bug Fixes** -* Fix Keras model config serialization in `PatchKerasModelIO` ([ClearML GitHub issue #614](https://github.com/allegroai/clearml/issues/614)) -* Fix `task.get_parameters_as_dict(cast=True)` casts `False` to `True` ([ClearML GitHub PR #622](https://github.com/allegroai/clearml/pull/622)) -* Fix Fire integration is not compatible with typing library ([ClearML GitHub issue #610](https://github.com/allegroai/clearml/issues/610)) +* Fix Keras model config serialization in `PatchKerasModelIO` ([ClearML GitHub issue #614](https://github.com/clearml/clearml/issues/614)) +* Fix `task.get_parameters_as_dict(cast=True)` casts `False` to `True` ([ClearML GitHub PR #622](https://github.com/clearml/clearml/pull/622)) +* Fix Fire integration is not compatible with typing library ([ClearML GitHub issue #610](https://github.com/clearml/clearml/issues/610)) * Fix remote execution with `argparse` mutually exclusive groups raises "required" error even when no argument is required * Fix Hydra tasks never fail and are only set to completed (fix handling return code) * Fix `clearml-data` wildcard support diff --git a/docs/release_notes/sdk/open_source/ver_1_5.md b/docs/release_notes/sdk/open_source/ver_1_5.md index e336be8c..771589ad 100644 --- a/docs/release_notes/sdk/open_source/ver_1_5.md +++ b/docs/release_notes/sdk/open_source/ver_1_5.md @@ -5,19 +5,19 @@ title: Version 1.5 ### ClearML 1.5.0 **New Features and Improvements** -* Add support for single value metric reporting ClearML GitHub issue ([ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400)) -* Add support for specifying parameter sections in `PipelineDecorator` ([ClearML GitHub issue #629](https://github.com/allegroai/clearml/issues/629)) +* Add support for single value metric reporting ClearML GitHub issue ([ClearML GitHub issue #400](https://github.com/clearml/clearml/issues/400)) +* Add support for specifying parameter sections in `PipelineDecorator` ([ClearML GitHub issue #629](https://github.com/clearml/clearml/issues/629)) * Add support for parallel uploads and downloads (upload / download and zip / unzip of artifacts) * Add support for specifying execution details (repository, branch, commit, packages, image) in `PipelineDecorator` * Bump PyJWT version due to "*Key confusion through non-blocklisted public key formats*" vulnerability * Add support for AWS Session Token (using boto3's `aws_session_token` argument) **Bug Fixes** -* Fix `Task.get_projects()` retrieves only the first 500 results ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612)) -* Fix failure to delete artifacts stored in Azure ([ClearML GitHub issue #660](https://github.com/allegroai/clearml/issues/660)) -* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674)) -* Fix number of unpacked values when syncing a dataset ([ClearML GitHub issue #682](https://github.com/allegroai/clearml/issues/682)) -* Fix FastAI DeprecationWarning ([ClearML GitHub PR #683](https://github.com/allegroai/clearml/issues/683)) +* Fix `Task.get_projects()` retrieves only the first 500 results ([ClearML GitHub issue #612](https://github.com/clearml/clearml/issues/612)) +* Fix failure to delete artifacts stored in Azure ([ClearML GitHub issue #660](https://github.com/clearml/clearml/issues/660)) +* Fix Process Pool hangs at exit ([ClearML GitHub issue #674](https://github.com/clearml/clearml/issues/674)) +* Fix number of unpacked values when syncing a dataset ([ClearML GitHub issue #682](https://github.com/clearml/clearml/issues/682)) +* Fix FastAI DeprecationWarning ([ClearML GitHub PR #683](https://github.com/clearml/clearml/issues/683)) * Fix `StorageManager.download_folder()` crash * Fix pipelines can't handle `None` return value * Fix pre-existing pipeline raises an exception diff --git a/docs/release_notes/sdk/open_source/ver_1_6.md b/docs/release_notes/sdk/open_source/ver_1_6.md index c7662ad7..5c34baaf 100644 --- a/docs/release_notes/sdk/open_source/ver_1_6.md +++ b/docs/release_notes/sdk/open_source/ver_1_6.md @@ -10,7 +10,7 @@ title: Version 1.6 ### ClearML 1.6.3 **New Features and Improvements** -* Add option to specify an endpoint URL when creating S3 resource service ([ClearML GitHub issue #673](https://github.com/allegroai/clearml/issues/673)) +* Add option to specify an endpoint URL when creating S3 resource service ([ClearML GitHub issue #673](https://github.com/clearml/clearml/issues/673)) * Add support for providing `ExtraArgs` to boto3 when uploading files using the `sdk.aws.s3.extra_args` configuration option * Add support for Server API 2.20 * Add `Task.get_num_enqueued_tasks()` to get the number of tasks enqueued in a specific queue @@ -25,12 +25,12 @@ title: Version 1.6 * Add wild-card support in `clearml-data` **Bug Fixes** -* Fix dataset download ([ClearML GitHub issue #713](https://github.com/allegroai/clearml/issues/713)) -* Fix lock is not released after dataset cache is downloaded ([ClearML GitHub issue #671](https://github.com/allegroai/clearml/issues/671)) -* Fix deadlock might occur when using process pool large number processes ([ClearML GitHub issue #674](https://github.com/allegroai/clearml/issues/674)) -* Fix 'series' not appearing on UI when using `logger.report_table()` ([ClearML GitHub issue #684](https://github.com/allegroai/clearml/issues/684)) -* Fix `Task.init()` docstring to include behavior when executing remotely ([ClearML GitHub PR #737](https://github.com/allegroai/clearml/pull/737)) -* Fix `KeyError` when running remotely and no params were passed to click ([ClearML Agent GitHub issue #111](https://github.com/allegroai/clearml-agent/issues/111)) +* Fix dataset download ([ClearML GitHub issue #713](https://github.com/clearml/clearml/issues/713)) +* Fix lock is not released after dataset cache is downloaded ([ClearML GitHub issue #671](https://github.com/clearml/clearml/issues/671)) +* Fix deadlock might occur when using process pool large number processes ([ClearML GitHub issue #674](https://github.com/clearml/clearml/issues/674)) +* Fix 'series' not appearing on UI when using `logger.report_table()` ([ClearML GitHub issue #684](https://github.com/clearml/clearml/issues/684)) +* Fix `Task.init()` docstring to include behavior when executing remotely ([ClearML GitHub PR #737](https://github.com/clearml/clearml/pull/737)) +* Fix `KeyError` when running remotely and no params were passed to click ([ClearML Agent GitHub issue #111](https://github.com/clearml/clearml-agent/issues/111)) * Fix full path is stored when uploading a single artifact file * Fix passing non-alphanumeric filename in `sdk.development.detect_with_pip_freeze` * Fix Python 3.6 and 3.10 support @@ -61,7 +61,7 @@ title: Version 1.6 **Bug Fix** -* Fix format string construction sometimes causing delayed evaluation errors ([ClearML GitHub issue #706](https://github.com/allegroai/clearml/issues/706)) +* Fix format string construction sometimes causing delayed evaluation errors ([ClearML GitHub issue #706](https://github.com/clearml/clearml/issues/706)) ### ClearML 1.6.1 @@ -83,7 +83,7 @@ title: Version 1.6 **Bug Fixes** * Fix axis range settings when logging plots -* Fix `Task.get_project()` to return more than 500 entries ([ClearML GitHub issue #612](https://github.com/allegroai/clearml/issues/612)) +* Fix `Task.get_project()` to return more than 500 entries ([ClearML GitHub issue #612](https://github.com/clearml/clearml/issues/612)) * Fix pipeline progress calculation * Fix `StorageManager.upload_folder()` returns `None` for both successful and unsuccessful uploads * Fix script path capturing stores a relative path and not an absolute path diff --git a/docs/release_notes/sdk/open_source/ver_1_7.md b/docs/release_notes/sdk/open_source/ver_1_7.md index d1eb260f..07cbe986 100644 --- a/docs/release_notes/sdk/open_source/ver_1_7.md +++ b/docs/release_notes/sdk/open_source/ver_1_7.md @@ -7,14 +7,14 @@ title: Version 1.7 * Support running Jupyter Notebook inside a git repository (repository will be referenced without uncommitted changes and Jupyter Notebook will be stored on plain code as uncommitted changes) * Add Jupyter Notebook fail warning -* Allow pipeline steps to return string paths without them being treated as a folder artifact and zipped ([ClearML GitHub issue #780](https://github.com/allegroai/clearml/issues/780)) +* Allow pipeline steps to return string paths without them being treated as a folder artifact and zipped ([ClearML GitHub issue #780](https://github.com/clearml/clearml/issues/780)) * Remove `future` from Python 3 requirements **Bug Fixes** -* Fix exception raised when using `ThreadPool` ([ClearML GitHub issue #790](https://github.com/allegroai/clearml/issues/790)) -* Fix Pyplot/Matplotlib binding reports incorrect line labels and colors ([ClearML GitHub issue #791](https://github.com/allegroai/clearml/issues/791)) +* Fix exception raised when using `ThreadPool` ([ClearML GitHub issue #790](https://github.com/clearml/clearml/issues/790)) +* Fix Pyplot/Matplotlib binding reports incorrect line labels and colors ([ClearML GitHub issue #791](https://github.com/clearml/clearml/issues/791)) * Pipelines - * Fix crash when running cloned pipeline that invokes a step twice ([ClearML GitHub issue #769](https://github.com/allegroai/clearml/issues/769)) + * Fix crash when running cloned pipeline that invokes a step twice ([ClearML GitHub issue #769](https://github.com/clearml/clearml/issues/769)) * Fix pipeline argument becomes `None` if default value is not set * Fix `retry_on_failure` callback does nothing when specified on `PipelineController.add_step()` * Fix pipeline clone logic @@ -23,7 +23,7 @@ title: Version 1.7 * Fix issue with old/new notebook packages installed * Fix local cache with access rules disabling partial local access * Fix `Task.upload_artifact()` fails uploading pandas `DataFrame` -* Fix relative paths in examples ([ClearML GitHub PR #787](https://github.com/allegroai/clearml/issues/787) +* Fix relative paths in examples ([ClearML GitHub PR #787](https://github.com/clearml/clearml/issues/787) ### ClearML 1.7.1 @@ -43,7 +43,7 @@ title: Version 1.7 **New Features and Improvements** * ClearML Data: Support providing list of links -* Upload artifacts with a custom serializer ([ClearML GitHub issue #689](https://github.com/allegroai/clearml/issues/689)) +* Upload artifacts with a custom serializer ([ClearML GitHub issue #689](https://github.com/clearml/clearml/issues/689)) * Allow user to specify extension when using custom serializer functions (for artifacts) * Skip server URL verification in `clearml-init` wizard process * When calling `Dataset.get()` without "alias" field, tell user that he can use alias to log it in the UI @@ -54,18 +54,18 @@ title: Version 1.7 * Add `Task.register_abort_callback` **Bug Fixes** -* Allow getting datasets with non-semantic versioning ([ClearML GitHub issue #776](https://github.com/allegroai/clearml/issues/776)) +* Allow getting datasets with non-semantic versioning ([ClearML GitHub issue #776](https://github.com/clearml/clearml/issues/776)) * Fix interactive plots (instead of a generated png) * Fix Python 2.7 support * Fix clearml datasets `list` functionality * Fix `Dataset.init` to not modify task (to `Dataset.create`) * Fix failure with large files upload on HTTPS * Fix 3d plots with plt shows 2d plot on task results page -* Fix uploading files with project's `default_upload_destination` ([ClearML GitHub issue #734](https://github.com/allegroai/clearml/issues/734)) +* Fix uploading files with project's `default_upload_destination` ([ClearML GitHub issue #734](https://github.com/clearml/clearml/issues/734)) * Fix broken Matplotlib reporting - using logarithmic scale breaks reporting * Fix wildcard support in `clearml-data` CLI -* Fix `report_histogram` - does not show "horizontal" orientation ([ClearML GitHub issue 699](https://github.com/allegroai/clearml/issues/699)) -* Fix table reporting - when using `logger.report_table(title, series, iteration, etc)`, the `series` arg does not appear in UI ([ClearML GitHub issue 684](https://github.com/allegroai/clearml/issues/684)) +* Fix `report_histogram` - does not show "horizontal" orientation ([ClearML GitHub issue 699](https://github.com/clearml/clearml/issues/699)) +* Fix table reporting - when using `logger.report_table(title, series, iteration, etc)`, the `series` arg does not appear in UI ([ClearML GitHub issue 684](https://github.com/clearml/clearml/issues/684)) * Fix artifacts (and models) use task's original name and not new name * Fix very long filenames from S3 can't be downloaded (with `get_local_copy()`) -* Fix overwrite of existing output models on pipeline task with `monitor_models` ([ClearML GitHub issue #757](https://github.com/allegroai/clearml/issues/757)) \ No newline at end of file +* Fix overwrite of existing output models on pipeline task with `monitor_models` ([ClearML GitHub issue #757](https://github.com/clearml/clearml/issues/757)) \ No newline at end of file diff --git a/docs/release_notes/sdk/open_source/ver_1_8.md b/docs/release_notes/sdk/open_source/ver_1_8.md index 6fc5443a..d9320e62 100644 --- a/docs/release_notes/sdk/open_source/ver_1_8.md +++ b/docs/release_notes/sdk/open_source/ver_1_8.md @@ -5,7 +5,7 @@ title: Version 1.8 ### ClearML 1.8.3 **Bug Fixes** -* Set GCS credentials to `None` if invalid service account credentials are provided ([ClearML GitHub issue #840](https://github.com/allegroai/clearml/issues/840)) +* Set GCS credentials to `None` if invalid service account credentials are provided ([ClearML GitHub issue #840](https://github.com/clearml/clearml/issues/840)) * Fix a sync issue when loading deferred configuration ### ClearML 1.8.2 @@ -20,8 +20,8 @@ title: Version 1.8 ### ClearML 1.8.1 **New Features and Improvements** -* Raise error on failed uploads ([ClearML GitHub issue #810](https://github.com/allegroai/clearml/issues/819)) -* Add hyperdataset examples ([ClearML GitHub PR #823](https://github.com/allegroai/clearml/commit/f6b9efe54e1246adba4036c56bc6e8a0bdb99948)) +* Raise error on failed uploads ([ClearML GitHub issue #810](https://github.com/clearml/clearml/issues/819)) +* Add hyperdataset examples ([ClearML GitHub PR #823](https://github.com/clearml/clearml/commit/f6b9efe54e1246adba4036c56bc6e8a0bdb99948)) * Change `report_event_flush_threshold` default to 100 * Add `ModelInfo.weights_object()` to store callback access to the actual model object being stored (valid for both pre/post save calls, otherwise `None`) @@ -29,8 +29,8 @@ pre/post save calls, otherwise `None`) * Support max connections setting for Azure storage using the `sdk.azure.storage.max_connection` configuration option **Bug Fixes** -* Fix clearml logger default level cannot be changed ([ClearML GitHub issue #741](https://github.com/allegroai/clearml/issues/741)) -* Fix Hydra doesn't get overridden information from ClearML ([ClearML GitHub issue #751](https://github.com/allegroai/clearml/issues/751)) +* Fix clearml logger default level cannot be changed ([ClearML GitHub issue #741](https://github.com/clearml/clearml/issues/741)) +* Fix Hydra doesn't get overridden information from ClearML ([ClearML GitHub issue #751](https://github.com/clearml/clearml/issues/751)) * Fix `StorageManager.list(“s3://..”, with_metadata=True)` doesn't work * Fix `ModelsList.keys()` is missing * Fix `CLEARML_DEFERRED_TASK_INIT=1` doesn't work @@ -39,10 +39,10 @@ pre/post save calls, otherwise `None`) ### ClearML 1.8.0 **New Features and Improvements** -* Add tarfile member sanitization to `extractall()`([ClearML GitHub PR #803](https://github.com/allegroai/clearml/pull/803)) -* Add `Task.delete_artifacts()` with `raise_on_errors` argument ([ClearML GitHub issue #805](https://github.com/allegroai/clearml/issues/805)) -* Add CI/CD example ([ClearML GitHub PR #815](https://github.com/allegroai/clearml/pull/815)) -* Limit number of `_serialize` requests when adding list of links with `add_external_files()` ([ClearML GitHub issue #813](https://github.com/allegroai/clearml/issues/813)) +* Add tarfile member sanitization to `extractall()`([ClearML GitHub PR #803](https://github.com/clearml/clearml/pull/803)) +* Add `Task.delete_artifacts()` with `raise_on_errors` argument ([ClearML GitHub issue #805](https://github.com/clearml/clearml/issues/805)) +* Add CI/CD example ([ClearML GitHub PR #815](https://github.com/clearml/clearml/pull/815)) +* Limit number of `_serialize` requests when adding list of links with `add_external_files()` ([ClearML GitHub issue #813](https://github.com/clearml/clearml/issues/813)) * Add support for connecting Enum values as parameters * Improve Colab integration (store entire colab, not history) * Add `clearml.browser_login` to authenticate browser online sessions such as Colab, Jupyter Notebooks etc. @@ -54,7 +54,7 @@ pre/post save calls, otherwise `None`) **Bug Fixes** * Fix passing `compression=ZIP_STORED` (or 0) to `Dataset.upload()` uses `ZIP_DEFLATED` and overrides the user-supplied -argument ([ClearML GitHub PR #812](https://github.com/allegroai/clearml/pull/812)) +argument ([ClearML GitHub PR #812](https://github.com/clearml/clearml/pull/812)) * Fix `unique_selector` is not applied properly on batches after the first batch. Remove default selector value since it does not work for all event types (and we always specify it anyway) * Fix `clearml-init` colab detection diff --git a/docs/release_notes/sdk/open_source/ver_1_9.md b/docs/release_notes/sdk/open_source/ver_1_9.md index a16b4ac9..9683af6b 100644 --- a/docs/release_notes/sdk/open_source/ver_1_9.md +++ b/docs/release_notes/sdk/open_source/ver_1_9.md @@ -6,22 +6,22 @@ title: Version 1.9 **Bug Fixes** * Fix broken `Task._get_status()`, which was breaking `clearml-session` in the latest version -* Fix path substitution, making it possible to store unsubstituted URL for models ([ClearML GitHub PR #935](https://github.com/allegroai/clearml/pull/935)) +* Fix path substitution, making it possible to store unsubstituted URL for models ([ClearML GitHub PR #935](https://github.com/clearml/clearml/pull/935)) ### ClearML 1.9.2 **New Features and Improvements** -* Support parsing queue name when providing execution queue in pipelines code ([ClearML GitHub PR #857](https://github.com/allegroai/clearml/pull/857)) -* Ignore `None` values for keys in the `click` argument parser ([ClearML GitHub issue #902](https://github.com/allegroai/clearml/issues/902)) -* Improve docstrings for `Task.mark_completed()` and `Task.close()` (ClearML GitHub PRs [#920](https://github.com/allegroai/clearml/pull/920) and [#921](https://github.com/allegroai/clearml/pull/921)) +* Support parsing queue name when providing execution queue in pipelines code ([ClearML GitHub PR #857](https://github.com/clearml/clearml/pull/857)) +* Ignore `None` values for keys in the `click` argument parser ([ClearML GitHub issue #902](https://github.com/clearml/clearml/issues/902)) +* Improve docstrings for `Task.mark_completed()` and `Task.close()` (ClearML GitHub PRs [#920](https://github.com/clearml/clearml/pull/920) and [#921](https://github.com/clearml/clearml/pull/921)) * Add pre/post execution callbacks to pipeline steps through `@PipelineDecorator.component` * Add status-change callback to pipeline steps through `PipelineController.add_step()`, `PipelineController.add_function_step()`, and `@PipelineDecorator.component` **Bug Fixes** -* Fix missing debug samples when reporting using TensorBoard ([ClearML GitHub issue #923](https://github.com/allegroai/clearml/issues/923)) -* Fix wrong Jupyter API token during repository detection ([ClearML GitHub PR #904](https://github.com/allegroai/clearml/pull/904)) -* Fix typo in the warning for very large git diffs ([ClearML GitHub PR #932](https://github.com/allegroai/clearml/pull/932)) +* Fix missing debug samples when reporting using TensorBoard ([ClearML GitHub issue #923](https://github.com/clearml/clearml/issues/923)) +* Fix wrong Jupyter API token during repository detection ([ClearML GitHub PR #904](https://github.com/clearml/clearml/pull/904)) +* Fix typo in the warning for very large git diffs ([ClearML GitHub PR #932](https://github.com/clearml/clearml/pull/932)) * Fix pipelines from tasks don't propagate `parameter_override` values in `PipelineController.add_step()` * Fix folders and files uploaded to S3 and Azure with `StorageManager.upload_file()` receive wrong MIME types * Fix the CSV file preview in Datasets @@ -33,17 +33,17 @@ and `@PipelineDecorator.component` ### ClearML 1.9.1 **New Features and Improvements** -* Add signature version to `boto3` configuration ([ClearML GitHub issue #883](https://github.com/allegroai/clearml/issues/883)) +* Add signature version to `boto3` configuration ([ClearML GitHub issue #883](https://github.com/clearml/clearml/issues/883)) * Allow requesting custom token expiration using the `api.auth.req_token_expiration_sec` configuration setting * Add Python 3.11 support **Bug Fixes** -* Fix `UniformParameterRange.to_list` throws error when step size is not defined ([ClearML GitHub issue #858](https://github.com/allegroai/clearml/issues/858)) -* Fix `StorageManager.list()` does not return size metadata ([ClearML GitHub issue #865](https://github.com/allegroai/clearml/issues/865)) -* Fix storage with path substitutions ([ClearML GitHub issue #825](https://github.com/allegroai/clearml/issues/825)) -* Fix extras in ClearML installation prevents clearml from being included in requirements ([ClearML GitHub issue #867](https://github.com/allegroai/clearml/issues/867)) -* Fix metadata set on an uploaded model object is not accessible ([ClearML GitHub issue #890](https://github.com/allegroai/clearml/issues/890)) -* Fix Azure storage upload not working ([ClearML GitHub issue #868](https://github.com/allegroai/clearml/issues/868)) +* Fix `UniformParameterRange.to_list` throws error when step size is not defined ([ClearML GitHub issue #858](https://github.com/clearml/clearml/issues/858)) +* Fix `StorageManager.list()` does not return size metadata ([ClearML GitHub issue #865](https://github.com/clearml/clearml/issues/865)) +* Fix storage with path substitutions ([ClearML GitHub issue #825](https://github.com/clearml/clearml/issues/825)) +* Fix extras in ClearML installation prevents clearml from being included in requirements ([ClearML GitHub issue #867](https://github.com/clearml/clearml/issues/867)) +* Fix metadata set on an uploaded model object is not accessible ([ClearML GitHub issue #890](https://github.com/clearml/clearml/issues/890)) +* Fix Azure storage upload not working ([ClearML GitHub issue #868](https://github.com/clearml/clearml/issues/868)) * Fix `task.connect` list of dicts parsed incorrectly in remote * Fix casting `None` to `int` fails uploads and permission checks * Fix numpy 1.24 support @@ -51,15 +51,15 @@ and `@PipelineDecorator.component` * Fix connecting a dictionary to task sometimes raises an exception * Fix authentication headers are not set on substituted fileserver URLs * Fix `Task.get_project_id()` cannot find hidden projects -* Fix TriggerScheduler docstrings ([ClearML GitHub issue #881](https://github.com/allegroai/clearml/issues/881)) +* Fix TriggerScheduler docstrings ([ClearML GitHub issue #881](https://github.com/clearml/clearml/issues/881)) ### ClearML 1.9.0 **New Features and Improvements** -* Add `r` prefix to `re.match()` strings ([ClearML GitHub issue #834](https://github.com/allegroai/clearml/issues/834)) -* Add `path_substitution` to `clearml.conf` example file ([ClearML GitHub PR #842](https://github.com/allegroai/clearml/pull/842)) -* Clarify `deferred_init` usage in `Task.init()` ([ClearML GitHub issue #855](https://github.com/allegroai/clearml/issues/855)) -* Add pipeline decorator argument to control docker image ([ClearML GitHub issue #856](https://github.com/allegroai/clearml/issues/856)) +* Add `r` prefix to `re.match()` strings ([ClearML GitHub issue #834](https://github.com/clearml/clearml/issues/834)) +* Add `path_substitution` to `clearml.conf` example file ([ClearML GitHub PR #842](https://github.com/clearml/clearml/pull/842)) +* Clarify `deferred_init` usage in `Task.init()` ([ClearML GitHub issue #855](https://github.com/clearml/clearml/issues/855)) +* Add pipeline decorator argument to control docker image ([ClearML GitHub issue #856](https://github.com/clearml/clearml/issues/856)) * Add `StorageManager.set_report_upload_chunk_size()` and `StorageManager.set_report_download_chunk_size()` to set chunk size for upload and download * Add `allow_archived` argument in `Task.get_tasks()` @@ -68,10 +68,10 @@ size for upload and download * Add `delete_from_storage` (default `True`) to `Task.delete_artifacts()` **Bug Fixes** -* Fix jsonargparse and pytorch lightning integration broken for remote execution ([ClearML GitHub issue #403](https://github.com/allegroai/clearml/issues/403)) -* Fix error when using `TaskScheduler` with `limit_execution_time` ([ClearML GitHub issue #648](https://github.com/allegroai/clearml/issues/648)) -* Fix dataset not synced if the changes are only modified files ([ClearML GitHub issue #822](https://github.com/allegroai/clearml/issues/822)) -* Fix `StorageHelper.delete()` does not respect path substitutions ([ClearML GitHub issue #825](https://github.com/allegroai/clearml/issues/825)) +* Fix jsonargparse and pytorch lightning integration broken for remote execution ([ClearML GitHub issue #403](https://github.com/clearml/clearml/issues/403)) +* Fix error when using `TaskScheduler` with `limit_execution_time` ([ClearML GitHub issue #648](https://github.com/clearml/clearml/issues/648)) +* Fix dataset not synced if the changes are only modified files ([ClearML GitHub issue #822](https://github.com/clearml/clearml/issues/822)) +* Fix `StorageHelper.delete()` does not respect path substitutions ([ClearML GitHub issue #825](https://github.com/clearml/clearml/issues/825)) * Fix can't write more than 2 GB to a file * Fix `StorageManager.get_file_size_bytes()` returns `ClientError` instead of `None` for invalid S3 links * Fix Dataset lineage view is broken with multiple dataset dependencies
Step 1 - Experiment ManagementStep 1 - Experiment Management Open In Colab
Step 2 - Remote Execution Agent SetupStep 2 - Remote Execution Agent Setup Open In Colab
Step 3 - Remotely Execute TasksStep 3 - Remotely Execute Tasks Open In Colab