Small edits (#828)

This commit is contained in:
pollfly 2024-04-24 10:48:14 +03:00 committed by GitHub
parent 4dc758ea3f
commit 0ca581c27b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
24 changed files with 87 additions and 87 deletions

View File

@ -69,7 +69,7 @@ errors in identifying the correct default branch.
| `--requirements` | Specify `requirements.txt` file to install when setting the session. By default, the` requirements.txt` from the repository will be used | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--packages` | Manually specify a list of required packages. Example: `--packages "tqdm>=2.1" "scikit-learn"` | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--docker` | Select the Docker image to use in the remote task | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--docker_args` | Add Docker arguments. Pass a single string in the following format: `--docker-args "<argument_string>"`. For example: `--docker-args "-v some_dir_1:other_dir_1 -v some_dir_2:other_dir_2"` | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--docker_args` | Add Docker arguments. Pass a single string in the following format: `--docker_args "<argument_string>"`. For example: `--docker_args "-v some_dir_1:other_dir_1 -v some_dir_2:other_dir_2"` | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--docker_bash_setup_script` | Add a bash script to be executed inside the Docker before setting up the task's environment | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--output-uri` | Set the task `output_uri`, upload destination for task models and artifacts | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |
| `--task-type` | Set the task type. Optional values: training, testing, inference, data_processing, application, monitor, controller, optimizer, service, qc, custom | <img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" /> |

View File

@ -217,7 +217,7 @@ clearml-data list [-h] [--id ID] [--project PROJECT] [--name NAME] [--version VE
|`--project`|Specify dataset project name (if used instead of ID, dataset name is also required)|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--name`|Specify dataset name (if used instead of ID, dataset project is also required)|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--version`|Specify dataset version. Default: most recent version |<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--filter`|Filter files based on folder / wildcard. Multiple filters are supported. Example: `folder/date_*.json folder/sub-folder`|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--filter`|Filter files based on folder / wildcard. Multiple filters are supported. Example: `folder/date_*.json folder/subfolder`|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
|`--modified`|Only list file changes (add / remove / modify) introduced in this version|<img src="/docs/latest/icons/ico-optional-yes.svg" alt="Yes" className="icon size-md center-md" />|
</div>

View File

@ -30,7 +30,7 @@ the needed files.
```
1. Add a folder. File addition is recursive, so it's enough to point at the folder
to captures all files and sub-folders:
to captures all files and subfolders:
```bash
clearml-data add --files data_samples

View File

@ -48,7 +48,7 @@ make sure to close a task, before initializing a new one. To close a task simply
(see example [here](../guides/advanced/multiple_tasks_single_process.md)).
When initializing a task, its project needs to be specified. If the project entered does not exist, it will be created on-the-fly.
Projects can be divided into subprojects, just like folders are broken into sub-folders.
Projects can be divided into subprojects, just like folders are broken into subfolders.
For example:
```python

View File

@ -83,7 +83,7 @@ executed tasks will have access to your storage service.
#### Additional Configuration
Go to a specific apps documentation page to view all configuration options
Go to a specific apps documentation page to view all configuration options:
* [AWS Autoscaler](../webapp/applications/apps_aws_autoscaler.md)
* [GCP Autoscaler](../webapp/applications/apps_gcp_autoscaler.md)

View File

@ -28,7 +28,7 @@ models, and dataviews, can be viewed in the project's [experiments table](../web
### Creating Subprojects
When [initializing a task](../clearml_sdk/task_sdk.md#task-creation), its project needs to be specified. If the project entered does not exist, it will be created.
Projects can contain subprojects, just like folders can contain sub-folders. Input into the `project_name`
Projects can contain subprojects, just like folders can contain subfolders. Input into the `project_name`
parameter a target project path. The project path should follow the project tree hierarchy, in which the project and
subprojects are slash (`/`) delimited.

View File

@ -125,7 +125,7 @@ ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72a
2021-07-25 13:59:16
```
**Thats it!** You are done integrating ClearML with your code :)
**That's it!** You are done integrating ClearML with your code :)
Now, [command-line arguments](../../fundamentals/hyperparameters.md#tracking-hyperparameters), [console output](../../fundamentals/logger.md#types-of-logged-results) as well as Tensorboard and Matplotlib will automatically be logged in the UI under the created Task.

View File

@ -131,7 +131,7 @@ Like before, you have to get the instance of the task training the original weig
Using TensorFlow, the snapshots are stored in a folder, meaning the `local_weights_path` will point to a folder containing your requested snapshot.
:::
As with artifacts, all models are cached, meaning the next time you run this code, no model needs to be downloaded.
Once one of the frameworks will load the weights file, the running task will be automatically updated with “Input Model” pointing directly to the original training Tasks Model.
Once one of the frameworks will load the weights file, the running task will be automatically updated with "Input Model" pointing directly to the original training Task's Model.
This feature lets you easily get a full genealogy of every trained and used model by your system!
## Log Metrics

View File

@ -20,7 +20,7 @@ This can create overhead that derails you from your core work!
ClearML Agent was designed to deal with such issues and more! It is a tool responsible for executing experiments on remote machines: on-premises or in the cloud! ClearML Agent provides the means to reproduce and track experiments in your
machine of choice through the ClearML WebApp with no need for additional code.
The agent will set up the environment for a specific Tasks execution (inside a Docker, or bare-metal), install the
The agent will set up the environment for a specific Task's execution (inside a Docker, or bare-metal), install the
required python packages, and execute and monitor the process.
@ -60,7 +60,7 @@ the agent executes an experiment inside a Docker container. For more information
## Clone an Experiment
Experiments already in the system can be reproduced for validation, or used as a baseline for further experimentation.
Cloning a task duplicates the tasks configuration, but not its outputs.
Cloning a task duplicates the task's configuration, but not its outputs.
**To clone an experiment in the ClearML WebApp:**
1. Click on any project card to open its [experiments table](../../webapp/webapp_exp_table.md)
@ -77,13 +77,13 @@ Once you have set up an experiment, it is now time to execute it.
**To execute an experiment through the ClearML WebApp:**
1. Right-click your draft experiment (the context menu is also available through the <img src="/docs/latest/icons/ico-bars-menu.svg" alt="Menu" className="icon size-md space-sm" />
button on the top right of the experiments info panel)
button on the top right of the experiment's info panel)
1. Click **ENQUEUE,** which will open the **ENQUEUE EXPERIMENT** window
1. In the window, select `default` in the queue menu
1. Click **ENQUEUE**
This action pushes the experiment into the `default` queue. The experiment's status becomes *Pending* until an agent
assigned to the queue fetches it, at which time the experiments status becomes *Running*. The agent executes the
assigned to the queue fetches it, at which time the experiment's status becomes *Running*. The agent executes the
experiment, and the experiment can be [tracked and its results visualized](../../webapp/webapp_exp_track_visual.md).
@ -106,7 +106,7 @@ Once a specific Task object has been obtained, it can be cloned, modified, and m
#### Clone an Experiment
To duplicate an experiment, use the [`Task.clone`](../../references/sdk/task.md#taskclone) method, and input either a
Task object or the Tasks ID as the `source_task` argument.
Task object or the Task's ID as the `source_task` argument.
```python
cloned_task = Task.clone(source_task=executed_task)
```
@ -173,7 +173,7 @@ ClearML also supports methods to explicitly log models. Models can be automatica
#### Log Metrics
Log as many metrics as you want from your processes using the [Logger](../../fundamentals/logger.md) module. This
improves the visibility of your processes progress.
improves the visibility of your processes' progress.
```python
from clearml import Logger
@ -212,7 +212,7 @@ tasks = Task.get_tasks(
```
#### Manage Your Data
Data is probably one of the biggest factors that determines the success of a project. Associating a models data with
Data is probably one of the biggest factors that determines the success of a project. Associating a model's data with
the model's configuration, code, and results (such as accuracy) is key to deducing meaningful insights into model behavior.
[ClearML Data](../../clearml_data/clearml_data.md) lets you version your data, so it's never lost, fetch it from every

View File

@ -20,27 +20,27 @@ keywords: [mlops, components, ClearML data]
<br/>
<Collapsible type="info" title="Video Transcript">
Hello and welcome to ClearML. In this video well take a look at both the command line and python interfaces of our data versioning tool called `clearml-data`.
Hello and welcome to ClearML. In this video we'll take a look at both the command line and python interfaces of our data versioning tool called `clearml-data`.
In the world of machine learning, you are very likely dealing with large amounts of data that you need to put into a dataset. ClearML Data solves 2 important challenges that occur in this situation:
One is accessibility, making sure the data can be accessed from every machine you use. And two is versioning, linking which dataset version was used in which task. This helps to make experiments more reproducible. Moreover, versioning systems like git were never really designed for the size and number of files in machine learning datasets. Were going to need something else.
One is accessibility, making sure the data can be accessed from every machine you use. And two is versioning, linking which dataset version was used in which task. This helps to make experiments more reproducible. Moreover, versioning systems like git were never really designed for the size and number of files in machine learning datasets. We're going to need something else.
ClearML Data comes built-in with the `clearml` python package and has both a command line interface for easy and quick operations and a python interface if you want more flexibility. Both interfaces are quite similar, so well address both of them in the video.
ClearML Data comes built-in with the `clearml` python package and has both a command line interface for easy and quick operations and a python interface if you want more flexibility. Both interfaces are quite similar, so we'll address both of them in the video.
Lets start with an example. Say I have some files here that I want to put into a dataset and start to keep track of.
Let's start with an example. Say I have some files here that I want to put into a dataset and start to keep track of.
First, we need to actually create an initial dataset version. The easiest way to do this is with the command line interface. Use the command `clearml-data create` and then give it a name and a project, just like with a ClearML task. It will return the dataset ID, which we will copy for later. The dataset is now initialized, but is still empty because we havent added any files yet.
First, we need to actually create an initial dataset version. The easiest way to do this is with the command line interface. Use the command `clearml-data create` and then give it a name and a project, just like with a ClearML task. It will return the dataset ID, which we will copy for later. The dataset is now initialized, but is still empty because we haven't added any files yet.
We can do that by using the `clearml-data add` command and providing the path to the files we want to add. This will recursively add all files in that path to the Dataset.
Now we need to tell the server that were done here. We can call `clearml-data close` to upload the files and change the dataset status to done, which finalizes this version of the dataset.
Now we need to tell the server that we're done here. We can call `clearml-data close` to upload the files and change the dataset status to done, which finalizes this version of the dataset.
The process of doing this with the python interface is very similar.
You can create a new Dataset by importing the Dataset object from the `clearml` pip package and calling its `create` method. Now we have to give the dataset a name and a project just like with the command line tool. The create method returns a dataset instance which we will use to do all of our operations on.
To add some files to this newly created dataset version, call the `add_files` method on the dataset object and provide a path to a local file or folder. Bear in mind that nothing is uploaded just yet, were simply instructing the dataset object what it should do when we eventually *do* want to upload.
To add some files to this newly created dataset version, call the `add_files` method on the dataset object and provide a path to a local file or folder. Bear in mind that nothing is uploaded just yet, we're simply instructing the dataset object what it should do when we eventually *do* want to upload.
A really useful thing we can do with the python interface is adding some interesting statistics about the dataset itself, such as a plot for example. Here we simply report a histogram on the amount of files in the train and test folders. You can add anything to a dataset that you can add to a ClearML task, so go nuts!
@ -48,37 +48,37 @@ Finally, upload the dataset and then finalize it, or just set `auto_upload` to `
In the web UI, we can now see the details of our dataset version by clicking on the Dataset button on the left. When we click on our newly created dataset here, we get an overview of our latest version, of course we have only one for now.
At a glance you can see things like the dataset ID, its size, and which files have been changed in this particular version. If you click on details, youll get a list of those files in the **Content** tab. Lets make the view a little larger with this button, so its easier to see. When we switch to the preview tab, we can see the histogram we made before as well as an automatically generated preview of some of the files in our dataset version. Feel free to add anything you want in here! Finally, you can check out the original console logs that can be handy for debugging.
At a glance you can see things like the dataset ID, its size, and which files have been changed in this particular version. If you click on details, you'll get a list of those files in the **Content** tab. Let's make the view a little larger with this button, so it's easier to see. When we switch to the preview tab, we can see the histogram we made before as well as an automatically generated preview of some of the files in our dataset version. Feel free to add anything you want in here! Finally, you can check out the original console logs that can be handy for debugging.
Now imagine were on a different machine. Maybe one from a team member, a classmate, or just one of your remote agents, and you want to get the dataset to do something cool with it.
Now imagine we're on a different machine. Maybe one from a team member, a classmate, or just one of your remote agents, and you want to get the dataset to do something cool with it.
Using the command line tool, you can download a dataset version locally by using the `clearml-data get` command and providing its unique ID. You can find a datasets ID in the UI here, or alternatively, you can search for a specific dataset by providing the dataset name, its project, some tags attached to the dataset or any combination of the three. Running the command will give you the system path where the data was downloaded.
Using the command line tool, you can download a dataset version locally by using the `clearml-data get` command and providing its unique ID. You can find a dataset's ID in the UI here, or alternatively, you can search for a specific dataset by providing the dataset name, its project, some tags attached to the dataset or any combination of the three. Running the command will give you the system path where the data was downloaded.
That path will be a local cached folder, which means that if you try to get the same dataset again, or any other dataset thats based on this one, it will check which files are already on your system, and it will not download these again.
That path will be a local cached folder, which means that if you try to get the same dataset again, or any other dataset that's based on this one, it will check which files are already on your system, and it will not download these again.
The python interface is similar, with one major difference. You can also get a dataset using any combination of name, project, ID or tags, but _getting_ the dataset does not mean it is downloaded, we simply got all of the metadata, which we can now access from the dataset object. This is important, as it means you dont have to download the dataset to make changes to it, or to add files. More on that in just a moment.
The python interface is similar, with one major difference. You can also get a dataset using any combination of name, project, ID or tags, but _getting_ the dataset does not mean it is downloaded, we simply got all of the metadata, which we can now access from the dataset object. This is important, as it means you don't have to download the dataset to make changes to it, or to add files. More on that in just a moment.
If you do want to download a local copy of the dataset, it has to be done explicitly, by calling `get_local_copy` which will return the path to which the data was downloaded for you.
This is a good approach for when you want to just download and use the data. But it *is* a read-only copy, so if we want to add or remove some data to create a new version, well have to get a mutable copy instead, which we can do by using `get_local_mutable_copy` instead. We can give it a local path, and it will download the dataset into that path, but this time, we have full control over the contents.
This is a good approach for when you want to just download and use the data. But it *is* a read-only copy, so if we want to add or remove some data to create a new version, we'll have to get a mutable copy instead, which we can do by using `get_local_mutable_copy` instead. We can give it a local path, and it will download the dataset into that path, but this time, we have full control over the contents.
We can do this with the command line tool too, by simply adding a `--copy` flag to the command
Now that we have this mutable copy, lets try to change our dataset and create a new version.
Now that we have this mutable copy, let's try to change our dataset and create a new version.
Lets say we found an issue with the hamburgers here, so we remove them from the folder. Then we add new pictures of chocolate cake. Essentially, we have now removed 3 files and added 4 new ones.
Let's say we found an issue with the hamburgers here, so we remove them from the folder. Then we add new pictures of chocolate cake. Essentially, we have now removed 3 files and added 4 new ones.
Now we can tell ClearML that the changes we made to this folder should become a new version of the previous dataset. We start by creating a new dataset just like we saw before, but now, we add the previous dataset ID as a parent. This tells ClearML that this new dataset version were creating is based on the previous one and so our dataset object here will already contain all the files that the parent contained.
Now we can tell ClearML that the changes we made to this folder should become a new version of the previous dataset. We start by creating a new dataset just like we saw before, but now, we add the previous dataset ID as a parent. This tells ClearML that this new dataset version we're creating is based on the previous one and so our dataset object here will already contain all the files that the parent contained.
Now we can manually remove and add the files that we want, even without actually downloading the dataset. It will just change the metadata inside the python object and sync everything when its finalized.
Now we can manually remove and add the files that we want, even without actually downloading the dataset. It will just change the metadata inside the python object and sync everything when it's finalized.
That said, we do have a local copy of the dataset in this case, so we have a better option.
Using the python SDK, we can call the `sync_folder` method. This method will essentially compare the dataset object metadata with the content of a `local_path` that you supply. So when we now call `finalize` and upload, it will only upload or remove the files that changed.
The command line interface doesnt have the python object for metadata, so it can only work with local data using the sync command. But it bunches this whole process together in one single command. Call `clearml-data sync`, provide it with the dataset name and project for the new version and maybe add some parent datasets too if applicable. This single call will create a new dataset version, sync it and then upload the changes all in 1 go. Neat, right?
The command line interface doesn't have the python object for metadata, so it can only work with local data using the sync command. But it bunches this whole process together in one single command. Call `clearml-data sync`, provide it with the dataset name and project for the new version and maybe add some parent datasets too if applicable. This single call will create a new dataset version, sync it and then upload the changes all in 1 go. Neat, right?
Now we can take a look again at the dataset UI. Well see our original dataset as well as the new version we made just now thats based on it.
Now we can take a look again at the dataset UI. We'll see our original dataset as well as the new version we made just now thats based on it.
When we click on our newest version in the lineage view, we can see that we indeed added 4 files and removed 3.

View File

@ -19,27 +19,27 @@ keywords: [mlops, components]
<br/>
<Collapsible type="info" title="Video Transcript">
Welcome to ClearML! This video will serve as an overview of the complete ClearML stack. Well introduce you to the most important concepts and show you how everything fits together, so you can deep dive into the next videos, which will cover the ClearML functionality in more detail.
Welcome to ClearML! This video will serve as an overview of the complete ClearML stack. We'll introduce you to the most important concepts and show you how everything fits together, so you can deep dive into the next videos, which will cover the ClearML functionality in more detail.
ClearML is designed to get you up and running in less than 10 minutes and 2 magic lines of code. But if you start digging, youll quickly find out that it has a lot of functionality to offer. So lets break it down, shall we?
ClearML is designed to get you up and running in less than 10 minutes and 2 magic lines of code. But if you start digging, you'll quickly find out that it has a lot of functionality to offer. So let's break it down, shall we?
At the heart of ClearML lies the experiment manager. It consists of the `clearml` pip package and the ClearML Server.
After running `pip install clearml` we can add 2 simple lines of python code to your existing codebase. These 2 lines will capture all the output that your code produces: logs, source code, hyperparameters, plots, images, you name it.
The pip package also includes `clearml-data`. It can help you keep track of your ever-changing datasets and provides an easy way to store, track and version control your data. Its also an easy way to share your dataset with colleagues over multiple machines while keeping track of who has which version. ClearML Data can even keep track of your datas ancestry, making sure you can always figure out where specific parts of your data came from.
The pip package also includes `clearml-data`. It can help you keep track of your ever-changing datasets and provides an easy way to store, track and version control your data. It's also an easy way to share your dataset with colleagues over multiple machines while keeping track of who has which version. ClearML Data can even keep track of your data's ancestry, making sure you can always figure out where specific parts of your data came from.
Both the 2 magic lines and the data tool will send all of their information to a ClearML server. This server then keeps an overview of your experiment runs and data sets over time, so you can always go back to a previous experiment, see how it was created and even recreate it exactly. Keep track of your best models by creating leaderboards based on your own metrics, and you can even directly compare multiple experiment runs, helping you to figure out the best way forward for your models.
To get started with a server right away, you can make use of the free tier. And when your needs grow, weve got you covered too! Just check out our website to find a tier that fits your organisation best. But, because were open source, you can also host your own completely for free. We have AWS images, Google Cloud images, you can run it on docker-compose locally or even, if you really hate yourself, run it on a self-hosted kubernetes cluster using our helm charts.
To get started with a server right away, you can make use of the free tier. And when your needs grow, we've got you covered too! Just check out our website to find a tier that fits your organisation best. But, because we're open source, you can also host your own completely for free. We have AWS images, Google Cloud images, you can run it on docker-compose locally or even, if you really hate yourself, run it on a self-hosted kubernetes cluster using our helm charts.
So, to recap: to get started, all you need is a pip package and a server to store everything. Easy right? But MLOps is much more than experiment and data management. Its also about automation and orchestration, which is exactly where the `clearml-agent` comes into play.
So, to recap: to get started, all you need is a pip package and a server to store everything. Easy right? But MLOps is much more than experiment and data management. It's also about automation and orchestration, which is exactly where the `clearml-agent` comes into play.
The `clearml-agent` is a daemon that you can run on 1 or multiple machines and turns them into workers. An agent executes an experiment or other workflow by reproducing the state of the code from the original machine to a remote machine.
Now that we have this remote execution capability, the possibilities are near endless.
For example, Its easy to set up an agent on either a CPU or a GPU machine, so you can easily run all of your experiments on any compute resource you have available. And if you spin up your agents in the cloud, theyll even support autoscaling out of the box.
For example, It's easy to set up an agent on either a CPU or a GPU machine, so you can easily run all of your experiments on any compute resource you have available. And if you spin up your agents in the cloud, they'll even support autoscaling out of the box.
You can set up multiple machines as agents to support large teams with their complex projects and easily configure a queuing system to get the most out of your available hardware.

View File

@ -69,7 +69,7 @@ keep track of installed packages and stuff like that. In this case, of course, w
it's only the `Task.init` and then just reporting some scalars.
What we do have is some scalars, so this is what it would look like, and we'll be using this one later down the line.
Right, so if I go back here to my code you can also see we have a GitHub folder with the workflow sub-folder in there.
Right, so if I go back here to my code you can also see we have a GitHub folder with the workflow subfolder in there.
This basically tells GitHub that whatever you do--a push or commit or whatever--it will check this `yaml` file to see
if it has to do any kind of checks. In this case, we'll call it ClearML checks, and we'll set it on to pull requests.
Now, most of the time that you're using ClearML, it's going to be interesting to do checks on a pull request because it

View File

@ -20,17 +20,17 @@ keywords: [mlops, components, hyperparameter optimization, hyperparameter]
<br/>
<Collapsible type="info" title="Video Transcript">
Hello and welcome to ClearML. In this video well take a look at one cool way of using the agent other than rerunning a task remotely: hyperparameter optimization (HPO).
Hello and welcome to ClearML. In this video we'll take a look at one cool way of using the agent other than rerunning a task remotely: hyperparameter optimization (HPO).
By now, we know that ClearML can easily capture our hyperparameters and scalars as part of the experiment tracking. We also know we can clone any task and change its hyperparameters, so theyll be injected into the original code at runtime. In the last video, we learnt how to make a remote machine execute this task automatically by using the agent.
By now, we know that ClearML can easily capture our hyperparameters and scalars as part of the experiment tracking. We also know we can clone any task and change its hyperparameters, so they'll be injected into the original code at runtime. In the last video, we learnt how to make a remote machine execute this task automatically by using the agent.
Soooo… Can we just clone a task like 100 times, inject different hyperparameters in every clone, run the clones on 10 agents and then sort the results based on a specific scalar?
Yeah, yeah we can, it's called hyperparameter optimization. And we can do all of this automatically too! No way you were going to clone and edit those 100 tasks yourself, right?
If you dont know what Hyperparameter Optimization is yet, you can find a link to our blog post on the topic in the description below. But in its most basic form, hyperparameter optimization tries to optimize a certain output by changing a set of inputs.
If you don't know what Hyperparameter Optimization is yet, you can find a link to our blog post on the topic in the description below. But in its most basic form, hyperparameter optimization tries to optimize a certain output by changing a set of inputs.
Lets say weve been working on this model here, and we were tracking our experiments with it anyway. We can see we have some hyperparameters to work with in the **Hyperparameters** tab of the web UI. They are logged by using the `Task.connect` function in our code. These are our inputs. We also have a scaler called `validation/epoch_accuracy`, that we want to get as high as possible. This is our output. We could also select to minimize the `epoch_loss` for example, that is something you can decide yourself.
Let's say we've been working on this model here, and we were tracking our experiments with it anyway. We can see we have some hyperparameters to work with in the **Hyperparameters** tab of the web UI. They are logged by using the `Task.connect` function in our code. These are our inputs. We also have a scaler called `validation/epoch_accuracy`, that we want to get as high as possible. This is our output. We could also select to minimize the `epoch_loss` for example, that is something you can decide yourself.
We can see that no code was used to log the scalar. It's done automatically because we are using TensorBoard.

View File

@ -20,19 +20,19 @@ keywords: [mlops, components, automation, orchestration, pipeline]
<br/>
<Collapsible type="info" title="Video Transcript">
Hello and welcome to ClearML. In this video well take a look at how pipelines can be used as a way to easily automate and orchestrate multiple tasks.
Hello and welcome to ClearML. In this video we'll take a look at how pipelines can be used as a way to easily automate and orchestrate multiple tasks.
Essentially, pipelines are a way to automate and orchestrate the execution of multiple tasks in a scalable way. Each task in the context of a ClearML pipeline is called a step or component, and it doesnt necessarily have to be an existing ClearML *task*, it can be any code.
Essentially, pipelines are a way to automate and orchestrate the execution of multiple tasks in a scalable way. Each task in the context of a ClearML pipeline is called a step or component, and it doesn't necessarily have to be an existing ClearML *task*, it can be any code.
A pipeline can be orchestrated using your own control logic. So you could say run task 2 only if task 1 was successful. But you can do more complex control logic too, like if the accuracy of the final model is not high enough, run the pipeline again with different parameters.
Pipelines are highly scalable too. Just like any object in the ClearML ecosystem, a pipeline is a task with inputs and outputs that you can clone just like any other. If you saw our video on HPO, this should ring a bell. Its completely doable to use hyperparameter optimization to optimize a complete pipeline and have all of the steps be run distributed on an auto-scaling cluster of agents. How is that not awesome?
Pipelines are highly scalable too. Just like any object in the ClearML ecosystem, a pipeline is a task with inputs and outputs that you can clone just like any other. If you saw our video on HPO, this should ring a bell. It's completely doable to use hyperparameter optimization to optimize a complete pipeline and have all of the steps be run distributed on an auto-scaling cluster of agents. How is that not awesome?
Ok, but how do we make one? In ClearML there are 2 main ways.
One is you can easily chain existing ClearML tasks together to create a single pipeline. This means each step in the pipeline is a task that you tracked before using the experiment manager. On the other hand, you could go a little deeper and create pipelines straight from your codebase, which is what well focus on in this video. But dont worry, the end result is the same in both cases: a ClearML pipeline.
One is you can easily chain existing ClearML tasks together to create a single pipeline. This means each step in the pipeline is a task that you tracked before using the experiment manager. On the other hand, you could go a little deeper and create pipelines straight from your codebase, which is what we'll focus on in this video. But don't worry, the end result is the same in both cases: a ClearML pipeline.
Lets say we have some functions that we already use to run ETL and another function that trains a model on the preprocessed data. We already have a main function too, that orchestrates when and how these other components should be run.
Let's say we have some functions that we already use to run ETL and another function that trains a model on the preprocessed data. We already have a main function too, that orchestrates when and how these other components should be run.
If we want to make this code into a pipeline, the first thing we have to do is to tell ClearML that these functions are supposed to become steps in our pipeline. We can do that by using a python decorator! For each function we want as a step, we can decorate it with `PipelineDecorator.component`.
@ -40,9 +40,9 @@ The component call will fully automatically transform this function into a Clear
We can specify what values the function will return and these will become artifacts in the new task. This will allow the following tasks in the pipeline to easily access them.
We can also cache the function, which means that if the pipeline is rerun, but this function didnt change, we will not execute the function again, which is super handy when loading lots of data that takes a long time for example.
We can also cache the function, which means that if the pipeline is rerun, but this function didn't change, we will not execute the function again, which is super handy when loading lots of data that takes a long time for example.
You can go quite far with configuring this component, one can even specify in which docker image this particular step should be executed when its run by the agent. Check our documentation in the links below for a detailed overview of all the arguments.
You can go quite far with configuring this component, one can even specify in which docker image this particular step should be executed when it's run by the agent. Check our documentation in the links below for a detailed overview of all the arguments.
The next thing we need is our control logic, the code that binds all other code together. In ClearML this is called a controller. We already have our control logic as code in our main function, so we can add a different decorator on here which is called: `pipeline`. The only arguments you need for the pipeline decorator is a name and a project just like any other task. Easy as pie.
@ -52,7 +52,7 @@ An important note here is that only if a step uses the output of a previous step
At last, we can now run our pipeline! We can choose to run it locally which means both the controller and all the steps will be run as subprocesses on your local machine. This is great for debugging, but if we want the real scaling powers of our pipeline, we can execute it normally and the pipeline and tasks will be queued instead, so they can be executed by our remote agents. The pipeline task itself will be enqueued in a special `services` queue, so when setting up your agents for pipeline execution, take a look at the documentation first.
After running the pipeline, you can see both the controller task and the first step popping up in the experiment view. But its easier to use the dedicated pipeline UI, which we can find on the left here.
After running the pipeline, you can see both the controller task and the first step popping up in the experiment view. But it's easier to use the dedicated pipeline UI, which we can find on the left here.
Here, we can find our pipeline project which automatically keeps track of every run we do. If we click on our pipeline here, we can see a nice visual representation of our pipeline steps.
@ -62,5 +62,5 @@ If we select a step from our pipeline, we can see much of the same details, but
But now comes the most powerful feature of all. Again, a pipeline controller is a task like any other, so… we can clone it like any other. Pressing the **+ New Run** button will allow us to do that from the UI! We can even change our global pipeline parameters here and, just like normal tasks, these will be injected into the original task and overwrite the original parameters. In this way, you can very quickly run many pipelines each with different parameters.
In the next video of this Getting Started series, well get a long-overdue look at ClearML Data, our data versioning tool. In the meantime, slap some pipeline decorators on your own functions for free at [app.clear.ml](https://app.clear.ml), and dont forget to join our [Slack channel](https://joinslack.clear.ml), if you need any help.
In the next video of this Getting Started series, we'll get a long-overdue look at ClearML Data, our data versioning tool. In the meantime, slap some pipeline decorators on your own functions for free at [app.clear.ml](https://app.clear.ml), and don't forget to join our [Slack channel](https://joinslack.clear.ml), if you need any help.
</Collapsible>

View File

@ -20,23 +20,23 @@ keywords: [mlops, components, automation, orchestration, pipeline]
<br/>
<Collapsible type="info" title="Video Transcript">
Hello and welcome to ClearML. In this video well take a look at how pipelines can be created from tasks instead of from code like we saw in the last video.
Hello and welcome to ClearML. In this video we'll take a look at how pipelines can be created from tasks instead of from code like we saw in the last video.
The tasks themselves are already in the system by using the experiment manager. Whats important to note here though is that hyperparameters, scalars, and artifacts should be reported correctly because the pipeline will consider them to be the inputs and outputs of each step. In that way, a step can easily access for example the artifacts from a previous step.
The tasks themselves are already in the system by using the experiment manager. What's important to note here though is that hyperparameters, scalars, and artifacts should be reported correctly because the pipeline will consider them to be the inputs and outputs of each step. In that way, a step can easily access for example the artifacts from a previous step.
So with the tasks as our steps this time, we really only need to add our control logic. And since we dont have the main function as we had in the last video, well put our control logic code in a dedicated `PipelineController` script instead. Lets start with a small example.
So with the tasks as our steps this time, we really only need to add our control logic. And since we don't have the main function as we had in the last video, we'll put our control logic code in a dedicated `PipelineController` script instead. Let's start with a small example.
Our example pipeline will consist of three distinct tasks. The first task downloads some data and then uploads it to ClearML as an artifact.
In a future video, Ill introduce you to ClearML Data which is actually our preferred way to handle data instead of uploading it as an artifact. So keep watching this getting started playlist if you want to know more.
In a future video, I'll introduce you to ClearML Data which is actually our preferred way to handle data instead of uploading it as an artifact. So keep watching this getting started playlist if you want to know more.
The next task will preprocess that data. It has some hyperparameters here that configure the way the preprocessing is done. As you can see, the dataset `url` parameter is still empty. When the pipeline is run, these hyperparameters can be overwritten by the output of the previous step. Well see how thats done a little later in the video. After the preprocessing, well upload the resulting training and test data as an artifact again.
The next task will preprocess that data. It has some hyperparameters here that configure the way the preprocessing is done. As you can see, the dataset `url` parameter is still empty. When the pipeline is run, these hyperparameters can be overwritten by the output of the previous step. We'll see how that's done a little later in the video. After the preprocessing, we'll upload the resulting training and test data as an artifact again.
The final task will train a model on the preprocessed data by downloading the train and test artifacts from the previous step. Again, the actual parameter, preprocessing task ID in this case, will be overwritten by the real ID when the pipeline is run. You can see here in my experiment list, that I already have these 3 tasks already logged.
Now comes our control logic. Lets start by making a simple python script. We can create a `PipelineController` object and give it a name and a project, it will become visible in the experiment list under that name because just like anything in ClearML, the controller is just a task, albeit a special type of task in this case.
Now comes our control logic. Let's start by making a simple python script. We can create a `PipelineController` object and give it a name and a project, it will become visible in the experiment list under that name because just like anything in ClearML, the controller is just a task, albeit a special type of task in this case.
Next, we can add some pipeline level parameters. These can be easily accessed from within every step of the pipeline. Theyre basically global variables. In this case well add a parameter that will tell the first step where to get the raw data from. This is very useful because well see later that we can easily rerun our pipeline with a different URL.
Next, we can add some pipeline level parameters. These can be easily accessed from within every step of the pipeline. They're basically global variables. In this case we'll add a parameter that will tell the first step where to get the raw data from. This is very useful because we'll see later that we can easily rerun our pipeline with a different URL.
Now we can define our steps. Each step needs a name and some link to the original task. We can either give it the original task's ID or provide the task name and project, in which case the controller will use the most recent task with that name in that project.
@ -48,7 +48,7 @@ Now we do the same for the final step. However, remember the empty hyperparamete
For example, we can tell the first step to use the global pipeline parameter raw data url like so. But we can also reference output artifacts from a previous step by using its name, and we can of course also just overwrite a parameter with a normal value. Finally, we can even pass along the unique task ID of a previous step, so you can get the task object based on that ID and access anything and everything within that task.
And thats it! We now have our first pipeline!
And that's it! We now have our first pipeline!
Just like in the previous video, we can run the whole pipeline locally first, to debug our flow and make sure everything is working. If everything works as planned, we can then start it normally and everything will be enqueued instead. Your agents listening to the services queue will pick up the pipeline controller, clone the tasks that form your steps, override the necessary parameters and enqueue them into the `default` queue, for your other agents to start working on.
@ -60,5 +60,5 @@ When we select a specific step, we can see its inputs and outputs as well as its
Finally, we can also clone the whole pipeline and change its parameters by clicking on the **+ New Run** button. This is the most powerful feature of all, as it allows us to really quickly rerun the whole pipeline with different parameters from the UI. The agents will take care of the rest!
In the next video of this Getting Started series, well take a look at ClearML Data, for real this time. In the meantime, spin up some pipeline controllers yourself for free at [app.clear.ml](https://app.clear.ml) and dont forget to join our [Slack channel](https://joinslack.clear.ml), if you need any help.
In the next video of this Getting Started series, we'll take a look at ClearML Data, for real this time. In the meantime, spin up some pipeline controllers yourself for free at [app.clear.ml](https://app.clear.ml) and don't forget to join our [Slack channel](https://joinslack.clear.ml), if you need any help.
</Collapsible>

View File

@ -20,29 +20,29 @@ keywords: [mlops, components, Autoscaler]
<br/>
<Collapsible type="info" title="Video Transcript">
Hello and welcome to ClearML. In this video well go a little more advanced and introduce autoscalers, the easiest way to build your very own flock of ClearML Agents.
Hello and welcome to ClearML. In this video we'll go a little more advanced and introduce autoscalers, the easiest way to build your very own flock of ClearML Agents.
Data science is inherently very inconsistent in its demand for compute resources. One moment youre just researching papers and need no compute at all, another moment youre making 16 GPUs scream and wishing you had more. Especially when running Hyperparameter Optimization or Pipelines, it can be very handy to have some extra hardware for a short time.
Data science is inherently very inconsistent in its demand for compute resources. One moment you're just researching papers and need no compute at all, another moment you're making 16 GPUs scream and wishing you had more. Especially when running Hyperparameter Optimization or Pipelines, it can be very handy to have some extra hardware for a short time.
Remote machines are easy to get from any cloud provider, and you only pay for the time you use them… as long as you dont forget to shut them down after youre done. Seriously, Im pretty sure at least 30% of GPU usage is people forgetting to shut down their remote machines.
Remote machines are easy to get from any cloud provider, and you only pay for the time you use them… as long as you don't forget to shut them down after you're done. Seriously, I'm pretty sure at least 30% of GPU usage is people forgetting to shut down their remote machines.
Anyway, thats what an autoscaler takes care of for you: spinning up as many machines as you need, when you need them and automatically shutting them down again when you dont.
Anyway, that's what an autoscaler takes care of for you: spinning up as many machines as you need, when you need them and automatically shutting them down again when you don't.
Once the autoscaler is deployed, you can just add experiments to a queue as we saw in the previous videos. Once there are experiments detected in the queue, the autoscaler will automatically spin up new remote machines and turn them into ClearML agents that will run them for you. No fiddling with remote SSH, and no docker containers, and no need to worry about shutting down--the Autoscaler does it for you.
You can also get fancy with queues. Create as many of them as you want, and you can specify which type of remote machine should serve which queues. So imagine you have a CPU queue and a GPU queue, all you have to do is put your experiment in the right queue, and you know exactly what type of machine will be running it.
Obviously, you also configure a maximum budget by limiting the number of machines that can be spun up at one time, so you dont incur unexpected expenses.
Obviously, you also configure a maximum budget by limiting the number of machines that can be spun up at one time, so you don't incur unexpected expenses.
Now that the theory is taken care of, lets take a look at how to set up an autoscaler on ClearML.
Now that the theory is taken care of, let's take a look at how to set up an autoscaler on ClearML.
To launch the autoscaler, go to [app.clear.ml](https://app.clear.ml) and open the Applications page. There youll find the autoscalers for each of the large cloud providers. To launch the autoscaler this way requires ClearML Pro, but its cheap enough that forgetting to shut down a remote GPU machine for 3 days costs more than a year of ClearML Pro, so…
To launch the autoscaler, go to [app.clear.ml](https://app.clear.ml) and open the Applications page. There you'll find the autoscalers for each of the large cloud providers. To launch the autoscaler this way requires ClearML Pro, but it's cheap enough that forgetting to shut down a remote GPU machine for 3 days costs more than a year of ClearML Pro, so…
Well go into the AWS wizard in this video, but the other autoscalers have a very similar setup. First are the credentials for your cloud provider of choice, make sure you assign the correct access rights because the autoscaler will use these credentials to launch the machines and shut them down again when they are idle.
We'll go into the AWS wizard in this video, but the other autoscalers have a very similar setup. First are the credentials for your cloud provider of choice, make sure you assign the correct access rights because the autoscaler will use these credentials to launch the machines and shut them down again when they are idle.
Naturally, you want the agent to be able to run your original code, so we need to supply our git credentials as well. This works by using a git application token as password, you can find how to generate such a token in the description below.
If youre running from a notebook, dont worry! Even notebooks that were tracked using the experiment manager can be reproduced on the remote machine!
If you're running from a notebook, dont worry! Even notebooks that were tracked using the experiment manager can be reproduced on the remote machine!
The last big, important setting is of course which kind of machines we want to spin up.

View File

@ -50,7 +50,7 @@ def main(pickle_url, mock_parameter='mock'):
(this is useful to create better visibility in projects with multiple pipelines, and for easy selection) (default:
`False`).
* `target_project` - If provided, all pipeline steps are cloned into the target project. If not provided, pipeline steps
are stored on the same project as the pipeline itself. Target sub-folder allows for easier organization of pipeline
are stored on the same project as the pipeline itself. Target subfolder allows for easier organization of pipeline
execution logic (Pipeline Task) and step execution Tasks. Example: "pipeline/component_execution".
* `abort_on_failure` - If `False` (default), failed pipeline steps will not cause the pipeline to stop immediately.
Instead, any step that is not connected (or indirectly connected) to the failed step, will still be executed.
@ -60,8 +60,8 @@ def main(pickle_url, mock_parameter='mock'):
* `pipeline_execution_queue` - The queue in which to enqueue the pipeline controller task. The default value is the
`services` queue. To run the pipeline logic locally while the components are executed remotely, pass
`pipeline_execution_queue=None`
* `skip_global_imports` If `True`, global imports will not be included in the steps execution. If `False` (default),
all global imports will be automatically imported at the beginning of each steps execution.
* `skip_global_imports` If `True`, global imports will not be included in the steps' execution. If `False` (default),
all global imports will be automatically imported at the beginning of each step's execution.
When the function is called, a corresponding ClearML Controller Task is created: its arguments are logged as the task's

View File

@ -73,8 +73,8 @@ title: Version 1.11
* Fix UI tag color list is empty in report page
* Fix clicking on UI project card navigates to experiments table instead of project overview
* Fix filters do not persist in UI resource card pages (datasets, pipelines, Hyper-Datasets, Reports)
* Fix UI experiment tables "Parent" filter not working
* Fix UI experiment scalar plot type cant be changed from linear to logarithmic in fullscreen mode
* Fix UI experiment table's "Parent" filter not working
* Fix UI experiment scalar plot type can't be changed from linear to logarithmic in fullscreen mode
* Fix UI tables sort numeric parameters alphabetically instead of numerically
* Fix model that has been unarchived remains in UI model archive
* Fix failed task configuration is editable

View File

@ -60,7 +60,7 @@ title: Version 1.15
* Fix incorrect plot name is displayed when expanding UI scalar plots
* Fix task IDs not displayed in UI experiment comparison graph titles and legends
* Fix single-value scalar plot not displaying "Download CSV" button when embedded in UI report
* Fix UI dashboard search returns results from teams work when "My Work" filter is enabled.
* Fix UI dashboard search returns results from team's work when "My Work" filter is enabled.
* Fix UI experiment table compare view does not persist
* Fix UI Workers and Queues displaying incorrect statistics
* Fix UI experiment scalar comparison missing active filter indicator

View File

@ -100,7 +100,7 @@ configuration option, allowing users to use `$ENV` in the task docker arguments
* Add history navigation to experiments plots UI page [ClearML GitHub issues #81](https://github.com/allegroai/clearml/issues/81) and [#255](https://github.com/allegroai/clearml/issues/255):
* Plots page shows last reported plot for each metric/variation combination
* Single plot view provides history navigation slider
* Add single value scalar reporting: Single value scalars are aggregated into a summary table in the experiments scalars
* Add single value scalar reporting: Single value scalars are aggregated into a summary table in the experiment's scalars
UI page [ClearML GitHub issue #400](https://github.com/allegroai/clearml/issues/400)
* Add "show hidden projects" control [ClearML GitHub issue #694](https://github.com/allegroai/clearml/issues/694)
* Improve UI projects page setting persistence - User's last chosen settings for recent/name sorting and team/personal

View File

@ -65,16 +65,16 @@ an empty string
* Add UI tables switch to detail mode through double-click [ClearML Server GitHub issue #134](https://github.com/allegroai/clearml-server/issues/134)
* Add customizable user activity timeout for UI logout
* Add UI navigation from experiment comparison back to originating experiment table
* Improve UI scalar comparison graph configuration - Persist users choice for viewing graph data.
* Improve UI scalar comparison graph configuration - Persist user's choice for viewing graph data.
* Add model IDs display in UI experiment artifact tab
* Add dataset description to UI dataset information display
**Bug Fixes**
* Fix UI experiment Hyperparameter tabs inefficient use of screen real estate [ClearML GitHub issue #705](https://github.com/allegroai/clearml/issues/705)
* Fix UI experiment Hyperparameter tab's inefficient use of screen real estate [ClearML GitHub issue #705](https://github.com/allegroai/clearml/issues/705)
* Fix navigating to an archived experiment's configuration causes UI to return to non-archived view [ClearML Server GitHub issue #148](https://github.com/allegroai/clearml-server/issues/148)
* Fix metric storage failure when large amount of metrics logged
* Fix UI plots downloaded as PNGs don't contain legends
* Fix UI plot colors cant be changed when colors are specified in code
* Fix UI plot colors can't be changed when colors are specified in code
* Fix experiment table hyperparameter column sorting fails when hyperparameter includes "."
* Fix artifacts aren't deleted when experiment is reset
* Fix UI project cards displaying incorrect task counts when "Show Hidden Projects" is enabled

View File

@ -120,5 +120,5 @@ size for upload and download
* Fix UI model link doesn't preserve double spaces, breaking the URI
* Fix tasks/models/datasets can't be moved to UI root project
* Fix UI tag color list doesn't display tags
* Fix UI dashboard search returns results from teams work when "My Work" filter is enabled
* Fix UI dashboard search returns results from team's work when "My Work" filter is enabled
* Fix UI experiment console does not display complete text when large text is reported

View File

@ -20,9 +20,9 @@ To clone an experiment:
1. Click **Clone**
:::note
By default, the new experiments parent task is set to the original tasks parent, unless the original task does not
By default, the new experiment's parent task is set to the original task's parent, unless the original task does not
have a parent, in which case the original task is set as the parent. Select `Set <cloned_experiment> as parent` to force
the original experiment to become the clones parent.
the original experiment to become the clone's parent.
:::
![Clone modal](../img/webapp_clone.png)
@ -43,7 +43,7 @@ To reset an experiment:
At the end of the process you are left with an experiment with draft status, meaning that it is editable.
Re-execute the new experiment:
1. If desired, modify the experiments configuration (see [Tuning Experiments](webapp_exp_tuning.md)).
1. If desired, modify the experiment's configuration (see [Tuning Experiments](webapp_exp_tuning.md)).
1. Enqueue the experiment for execution. Right-click the experiment > Enqueue > Select a queue > **ENQUEUE**.
:::note

View File

@ -198,7 +198,7 @@ The comparison view compares experiment scalar and plot results (for a more in d
When selected, the view presents a comparison of all [selected experiments](#selecting-multiple-experiments). If no
experiments are selected, all currently visible experiments in the table are displayed in the comparison.
In the dropdown menu, select to view **Scalars** or **Plots**. **Scalars** shows experiments' scalar results as time
In the dropdown menu, select to view **Scalars** or **Plots**. **Scalars** shows experiment scalar results as time
series line graphs. **Plots** shows the last reported iteration sample of each metric/variant combination per compared
experiment.