diff --git a/docs/img/gif/integrations_yolov5.gif b/docs/img/gif/integrations_yolov5.gif new file mode 100644 index 00000000..f332940c Binary files /dev/null and b/docs/img/gif/integrations_yolov5.gif differ diff --git a/docs/integrations/yolov5.md b/docs/integrations/yolov5.md new file mode 100644 index 00000000..c32a0564 --- /dev/null +++ b/docs/integrations/yolov5.md @@ -0,0 +1,208 @@ +--- +title: YOLOv5 +--- + +ClearML helps you get the most out of ultralytics' [YOLOv5](https://github.com/ultralytics/yolov5) through its native +built in logger: +* Track every YOLOv5 training run in the ClearML experiment manager +* Version and easily access your custom training data with [ClearML Data](../clearml_data/clearml_data.md) +* Remotely train and monitor your YOLOv5 training runs using [ClearML Agent](../clearml_agent.md) +* Get the very best mAP using ClearML [Hyperparameter Optimization](../fundamentals/hpo.md) +* Turn your newly trained YOLOv5 model into an API with just a few commands using [ClearML Serving](../clearml_serving/clearml_serving.md) + +## Setup +1. Install the clearml python package: + + ```commandline + pip install clearml + ``` + +1. To keep track of your experiments and/or data, ClearML needs to communicate to a server. You have 2 server options: + * Sign up for free to the [ClearML Hosted Service](https://app.clear.ml/) + * Set up your own server, see [here](../deploying_clearml/clearml_server.md). +1. Connect the ClearML SDK to the server by creating credentials (go to the top right in to UI to **Settings > Workspace > Create new credentials**), + then execute the command below and follow the instructions: + + ```commandline + clearml-init + ``` + +That’s it! Now, whenever you train a model using YOLOv5, the run will be captured and tracked by ClearML – no additional +code necessary. + +## Training YOLOv5 with ClearML + +To enable ClearML experiment tracking, simply install the `clearml` pip package in your execution environment. + +```commandline +pip install clearml>=1.2.0 +``` + +This will enable integration with the YOLOv5 training script. In every training run from now on, the ClearML experiment +manager will capture: +* Source code and uncommitted changes +* Installed packages +* [Hyperparameters](../fundamentals/hyperparameters.md) +* Model files (use `--save-period n` to save a checkpoint every n epochs) +* Console output +* Scalars (e.g. mAP_0.5, mAP_0.5:0.95, precision, recall, losses) +* General information such as machine details, runtime, creation date etc. +* All produced plots such as label correlogram and confusion matrix +* Images with bounding boxes per epoch +* Mosaic per epoch +* Validation images per epoch +* And more + +All of this is captured into a [ClearML Task](../fundamentals/task.md). By default, a task called `Training` is created +in the `YOLOv5` project. To change the task’s name or project , use the `--project` and `--name` arguments when running +the `train.py` script. + +```commandline +python train.py --project my_project --name my_training --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache +``` + +:::tip project names +ClearML uses `/` as a delimiter for subprojects: using `example/sample` as a name will create the task in a `sample` +task within the `example` project. +::: + +You can visualize all the captured data in the task’s page in the [WebApp](../webapp/webapp_exp_track_visual.md). +Additionally, you can view all of your YOLOv5 runs tracked by ClearML in the [Experiments Table](../webapp/webapp_model_table.md). +Add custom columns to the table, such as mAP values, so you can easily sort and see what is the best performing model. +You can also select multiple experiments and directly [compare](../webapp/webapp_exp_comparing.md) them. + +## Dataset Version Management +Versioning your data separately from your code makes it easier to access the dataset version you need for your +experiments. [ClearML Data](../clearml_data/clearml_data.md) makes data easily accessible from every machine, and links +data and experiments for better traceability. + +### Prepare Your Dataset +The [YOLOv5 repository](https://github.com/ultralytics/yolov5) supports a number of different datasets by using yaml +files containing their information. By default, datasets are downloaded to the `../datasets` folder relative to the +repository root folder. For example, if you download the [coco128](https://github.com/ultralytics/yolov5/blob/master/data/coco128.yaml) +dataset using the link in the yaml file or the scripts provided by YOLOv5, you get this folder structure: + +``` +.. +|_ yolov5 +|_ datasets + |_ coco128 + |_ images + |_ labels + |_ LICENSE + |_ README.txt +``` + +You can use any dataset, as long as you maintain this folder structure. +Copy the dataset’s corresponding yaml file to the root of the dataset folder. + +``` +.. +|_ yolov5 +|_ datasets + |_ coco128 + |_ images + |_ labels + |_ coco128.yaml # <---- HERE! + |_ LICENSE + |_ README.txt +``` + +The yaml file needs to be stored along with your dataset so that ClearML can set up the dataset in a way that it's +accessible to train on the data with YOLO. + +### Upload Your Dataset +To create a ClearML dataset with this data, go to the dataset root folder and run the `sync` command: + +```commandline +cd coco128 +clearml-data sync --project YOLOv5 --name coco128 --folder . +``` + +This command syncs the folder's content with ClearML, packaging all of the folder's contents into a ClearML dataset. + +Alternatively, you could run these commands one after the other to create a dataset: + +```commandline +# Optionally add --parent if you want to base +# this version on another dataset version, so no duplicate files are uploaded! +clearml-data create --name coco128 --project YOLOv5 +clearml-data add --files . +clearml-data close +``` + +Both of these methods print to console a dataset ID, which you can later use to access your dataset: + +```console +clearml-data - Dataset Management & Versioning CLI +Creating a new dataset: +ClearML results page: https://app.clear.ml/projects//experiments//output/log +ClearML dataset page: https://app.clear.mli/datasets/simple//experiments/ +New dataset created id= +``` + +### Run Training Using A ClearML Dataset +Now that you have a ClearML dataset, you can very simply use it to train custom YOLOv5 models: + +```commandline +python train.py --img 640 --batch 16 --epochs 3 --data clearml:// --weights yolov5s.pt --cache +``` + + +## Remote Execution +ClearML logs all the information required to reproduce an experiment on a different machine (installed packages, +uncommitted changes etc.). The [ClearML Agent](../clearml_agent.md) listens to designated queues and when a task is +enqueued, the agent pulls it, recreates its environment, and runs it, reporting its scalars, plots, etc. to the +experiment manager. + +You can turn any machine (e.g. a cloud VM, a local GPU machine, your own laptop) into a ClearML Agent by simply running +the following command on it: + +```commandline +clearml-agent daemon --queue [--docker] +``` + +Use the ClearML [Autoscalers](../cloud_autoscaling/autoscaling_overview.md), to help you automatically deploy ClearML +agents on new remote machines in the cloud of your choice (AWS, GCP, Azure): The autoscaler automatically spins up and +shuts down instances as needed, according to the budget that you set. + + +### Cloning, Editing, And Enqueuing + +![Cloning, editing, enqueuing gif](../img/gif/integrations_yolov5.gif) + +Use ClearML’s web interface to edit task details, like configuration parameters or input models, then execute the task +with the new details on a remote machine: +* Clone the experiment +* Edit the hyperparameters and/or other details +* Enqueue the task + +The ClearML Agent executing the task will use the new values to [override any hard coded values](../clearml_agent.md). + +### Executing A Task Remotely +You can set a task to be executed remotely programmatically by adding `Task.execute_remotely()` to your script. This +method stops the current local execution of the task, and then enqueues it to a specified queue to re-run it on a remote machine. + +To run the YOLOv5 training script remotely, all you have to do is add `loggers.clearml.task.execute_remotely(queue="")` +to the `training.py script` after the ClearML logger has been instantiated: + +```python +# ... +# Loggers +data_dict = None +if RANK in {-1, 0}: + loggers = Loggers(save_dir, weights, opt, hyp, LOGGER) # loggers instance + if loggers.clearml: + loggers.clearml.task.execute_remotely(queue="my_queue") # <------ ADD THIS LINE + # Data_dict is either None is user did not choose for ClearML dataset or is filled in by ClearML + data_dict = loggers.clearml.data_dict +# … +``` + +## Hyperparameter Optimization +Use ClearML’s [`HyperParameterOptimizer`](../references/sdk/hpo_optimization_hyperparameteroptimizer.md) class to find +the hyperparameter values that yield the best performing models. + +To run hyperparameter optimization locally, you can use the [template script](https://github.com/ultralytics/yolov5/blob/master/utils/loggers/clearml/hpo.py) +provided with YOLOv5. Notice you need to fill in a baseline task ID, meaning a training task needs to have been run at +least once. This experiment will be cloned multiple times, and each clone's parameter will be overridden with a new value. diff --git a/sidebars.js b/sidebars.js index 03e53519..10ec717b 100644 --- a/sidebars.js +++ b/sidebars.js @@ -69,7 +69,7 @@ module.exports = { {'PyTorch Ignite':['guides/frameworks/pytorch_ignite/integration_pytorch_ignite', 'guides/frameworks/pytorch_ignite/pytorch_ignite_mnist']}, 'guides/frameworks/pytorch_lightning/pytorch_lightning_example', 'guides/frameworks/scikit-learn/sklearn_joblib_example', 'guides/frameworks/pytorch/pytorch_tensorboard', 'guides/frameworks/tensorboardx/tensorboardx', 'guides/frameworks/tensorflow/tensorflow_mnist', - 'integrations/seaborn', 'guides/frameworks/xgboost/xgboost_metrics' + 'integrations/seaborn', 'guides/frameworks/xgboost/xgboost_metrics', 'integrations/yolov5' ] }, 'integrations/storage',