diff --git a/docs/apps/clearml_session.md b/docs/apps/clearml_session.md index 91f7d276..c9d1a43d 100644 --- a/docs/apps/clearml_session.md +++ b/docs/apps/clearml_session.md @@ -23,7 +23,7 @@ VS Code remote sessions use ports 8878 and 8898 respectively. ## Prerequisites -* `clearml` installed and configured. See [Getting Started](../getting_started/ds/ds_first_steps.md) for details. +* `clearml` installed and configured. See [ClearML Setup](../clearml_sdk/clearml_sdk_setup) for details. * At least one `clearml-agent` running on a remote host. See [installation](../clearml_agent/clearml_agent_setup.md#installation) for details. * An SSH client installed on your machine. To verify, open your terminal and execute `ssh`. If you did not receive an error, you are good to go. diff --git a/docs/clearml_data/clearml_data.md b/docs/clearml_data/clearml_data.md index de127ce7..259bf205 100644 --- a/docs/clearml_data/clearml_data.md +++ b/docs/clearml_data/clearml_data.md @@ -37,7 +37,7 @@ lineage and content information. See [dataset UI](../webapp/datasets/webapp_data ## Setup -`clearml-data` comes built-in with the `clearml` Python package! Check out the [Getting Started](../getting_started/ds/ds_first_steps.md) +`clearml-data` comes built-in with the `clearml` Python package! Check out the [ClearML Setup](../clearml_sdk/clearml_sdk_setup) guide for more info! ## Using ClearML Data diff --git a/docs/clearml_sdk/clearml_sdk.md b/docs/clearml_sdk/clearml_sdk.md index d5802c82..bf0e93c6 100644 --- a/docs/clearml_sdk/clearml_sdk.md +++ b/docs/clearml_sdk/clearml_sdk.md @@ -7,7 +7,7 @@ tasks for you, and an extensive set of powerful features and functionality you c and other workflows. :::tip Installation -For installation instructions, see [Getting Started](../getting_started/ds/ds_first_steps.md#install-clearml). +For installation instructions, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup#install-clearml). ::: The ClearML Python Package collects the scripts' entire execution information, including: diff --git a/docs/getting_started/ds/ds_first_steps.md b/docs/clearml_sdk/clearml_sdk_setup.md similarity index 57% rename from docs/getting_started/ds/ds_first_steps.md rename to docs/clearml_sdk/clearml_sdk_setup.md index 8e1aa996..cd374a06 100644 --- a/docs/getting_started/ds/ds_first_steps.md +++ b/docs/clearml_sdk/clearml_sdk_setup.md @@ -1,7 +1,9 @@ --- -title: First Steps +title: ClearML Python Package --- +This is step-by-step guide for installing the `clearml` Python package and connecting it to the ClearML Server. Once done, +you can integrate `clearml` into your code. ## Install ClearML @@ -44,8 +46,8 @@ pip install clearml CLEARML_CONFIG_FILE = MyOtherClearML.conf ``` - For more information about running tasks inside Docker containers, see [ClearML Agent Deployment](../../clearml_agent/clearml_agent_deployment.md) - and [ClearML Agent Reference](../../clearml_agent/clearml_agent_ref.md). + For more information about running tasks inside Docker containers, see [ClearML Agent Deployment](../clearml_agent/clearml_agent_deployment.md) + and [ClearML Agent Reference](../clearml_agent/clearml_agent_ref.md). @@ -83,7 +85,7 @@ pip install clearml CLEARML setup completed successfully. ``` -Now you can integrate ClearML into your code! Continue [here](#auto-log-experiment). +Now you can integrate ClearML into your code! Continue [here](../clearml_sdk/clearml_sdk_setup#auto-log-experiment). ### Jupyter Notebook To use ClearML with Jupyter Notebook, you need to configure ClearML Server access credentials for your notebook. @@ -94,49 +96,3 @@ To use ClearML with Jupyter Notebook, you need to configure ClearML Server acces 1. Add these commands to your notebook Now you can use ClearML in your notebook! - -## Auto-log Experiment - -In ClearML, experiments are organized as [Tasks](../../fundamentals/task.md). - -ClearML automatically logs your task and code, including outputs and parameters from popular ML frameworks, -once you integrate the ClearML [SDK](../../clearml_sdk/clearml_sdk.md) with your code. To control what ClearML automatically logs, see this [FAQ](../../faq.md#controlling_logging). - -At the beginning of your code, import the `clearml` package: - -```python -from clearml import Task -``` - -:::tip Full Automatic Logging -To ensure full automatic logging, it is recommended to import the `clearml` package at the top of your entry script. -::: - -Then initialize the Task object in your `main()` function, or the beginning of the script. - -```python -task = Task.init(project_name='great project', task_name='best task') -``` - -If the project does not already exist, a new one is created automatically. - -The console should display the following output: - -``` -ClearML Task: created new task id=1ca59ef1f86d44bd81cb517d529d9e5a -2021-07-25 13:59:09 -ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72ad233/experiments/1ca59ef1f86d44bd81cb517d529d9e5a/output/log -2021-07-25 13:59:16 -``` - -**That's it!** You are done integrating ClearML with your code :) - -Now, [command-line arguments](../../fundamentals/hyperparameters.md#tracking-hyperparameters), [console output](../../fundamentals/logger.md#types-of-logged-results) as well as Tensorboard and Matplotlib will automatically be logged in the UI under the created Task. - -Sit back, relax, and watch your models converge :) or continue to see what else can be done with ClearML [here](ds_second_steps.md). - -## YouTube Playlist - -Or watch the **Getting Started** playlist on ClearML's YouTube Channel! - -[![Watch the video](https://img.youtube.com/vi/bjWwZAzDxTY/hqdefault.jpg)](https://www.youtube.com/watch?v=bjWwZAzDxTY&list=PLMdIlCuMqSTnoC45ME5_JnsJX0zWqDdlO&index=2) diff --git a/docs/clearml_serving/clearml_serving_setup.md b/docs/clearml_serving/clearml_serving_setup.md index 1d7da9d4..05219921 100644 --- a/docs/clearml_serving/clearml_serving_setup.md +++ b/docs/clearml_serving/clearml_serving_setup.md @@ -13,7 +13,7 @@ The following page goes over how to set up and upgrade `clearml-serving`. ## Initial Setup 1. Set up your [ClearML Server](../deploying_clearml/clearml_server.md) or use the [free hosted service](https://app.clear.ml) -1. Connect `clearml` SDK to the server, see instructions [here](../getting_started/ds/ds_first_steps.md#install-clearml) +1. Connect `clearml` SDK to the server, see instructions [here](../clearml_sdk/clearml_sdk_setup#install-clearml) 1. Install clearml-serving CLI: diff --git a/docs/deploying_clearml/clearml_server.md b/docs/deploying_clearml/clearml_server.md index e9480307..dc137d76 100644 --- a/docs/deploying_clearml/clearml_server.md +++ b/docs/deploying_clearml/clearml_server.md @@ -49,7 +49,7 @@ authentication, subdomains, and load balancers, and use any of its many configur 1. Optionally, [configure ClearML Server](clearml_server_config.md) for additional features, including subdomains and load balancers, web login authentication, and the non-responsive task watchdog. -1. [Connect the ClearML SDK to the ClearML Server](../getting_started/ds/ds_first_steps.md) +1. [Connect the ClearML SDK to the ClearML Server](../clearml_sdk/clearml_sdk_setup) ## Updating diff --git a/docs/deploying_clearml/clearml_server_aws_ec2_ami.md b/docs/deploying_clearml/clearml_server_aws_ec2_ami.md index d44096e9..e3c3a8e0 100644 --- a/docs/deploying_clearml/clearml_server_aws_ec2_ami.md +++ b/docs/deploying_clearml/clearml_server_aws_ec2_ami.md @@ -150,4 +150,4 @@ The following section contains a list of AMI Image IDs per-region for the latest ## Next Step To keep track of your experiments and/or data, the `clearml` package needs to communicate with your server. -For instruction to connect the ClearML SDK to the server, see [Getting Started: First Steps](../getting_started/ds/ds_first_steps.md). +For instruction to connect the ClearML SDK to the server, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup). diff --git a/docs/deploying_clearml/clearml_server_gcp.md b/docs/deploying_clearml/clearml_server_gcp.md index 7b37f5e6..db5996a8 100644 --- a/docs/deploying_clearml/clearml_server_gcp.md +++ b/docs/deploying_clearml/clearml_server_gcp.md @@ -7,7 +7,7 @@ provides custom images for each released version of ClearML Server. For a list o [ClearML Server GCP Custom Image](#clearml-server-gcp-custom-image). To keep track of your experiments and/or data, the `clearml` package needs to communicate with the server you have deployed. -For instruction to connect the ClearML SDK to the server, see [Getting Started: First Steps](../getting_started/ds/ds_first_steps.md). +For instruction to connect the ClearML SDK to the server, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup). :::info In order for `clearml` to work with a ClearML Server on GCP, set `CLEARML_API_DEFAULT_REQ_METHOD=PUT` or @@ -155,4 +155,4 @@ The following section contains a list of Custom Image URLs (exported in differen ## Next Step To keep track of your experiments and/or data, the `clearml` package needs to communicate with your server. -For instruction to connect the ClearML SDK to the server, see [Getting Started: First Steps](../getting_started/ds/ds_first_steps.md). +For instruction to connect the ClearML SDK to the server, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup). diff --git a/docs/deploying_clearml/clearml_server_kubernetes_helm.md b/docs/deploying_clearml/clearml_server_kubernetes_helm.md index 5325b336..3023e738 100644 --- a/docs/deploying_clearml/clearml_server_kubernetes_helm.md +++ b/docs/deploying_clearml/clearml_server_kubernetes_helm.md @@ -32,4 +32,4 @@ instructions in the [Security](clearml_server_security.md) page. ## Next Step To keep track of your experiments and/or data, the `clearml` package needs to communicate with your server. -For instruction to connect the ClearML SDK to the server, see [Getting Started: First Steps](../getting_started/ds/ds_first_steps.md). \ No newline at end of file +For instruction to connect the ClearML SDK to the server, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup). \ No newline at end of file diff --git a/docs/deploying_clearml/clearml_server_linux_mac.md b/docs/deploying_clearml/clearml_server_linux_mac.md index a22eacc4..9509c748 100644 --- a/docs/deploying_clearml/clearml_server_linux_mac.md +++ b/docs/deploying_clearml/clearml_server_linux_mac.md @@ -227,4 +227,4 @@ If needed, restore data and configuration by doing the following: ## Next Step To keep track of your experiments and/or data, the `clearml` package needs to communicate with your server. -For instruction to connect the ClearML SDK to the server, see [Getting Started: First Steps](../getting_started/ds/ds_first_steps.md). \ No newline at end of file +For instruction to connect the ClearML SDK to the server, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup). \ No newline at end of file diff --git a/docs/deploying_clearml/clearml_server_win.md b/docs/deploying_clearml/clearml_server_win.md index ef4e849f..f3a54e20 100644 --- a/docs/deploying_clearml/clearml_server_win.md +++ b/docs/deploying_clearml/clearml_server_win.md @@ -89,4 +89,4 @@ After deploying ClearML Server, the services expose the following node ports: ## Next Step To keep track of your experiments and/or data, the `clearml` package needs to communicate with your server. -For instruction to connect the ClearML SDK to the server, see [Getting Started: First Steps](../getting_started/ds/ds_first_steps.md). \ No newline at end of file +For instruction to connect the ClearML SDK to the server, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup). \ No newline at end of file diff --git a/docs/getting_started/architecture.md b/docs/getting_started/architecture.md index 7f7a7e35..42d2fc2d 100644 --- a/docs/getting_started/architecture.md +++ b/docs/getting_started/architecture.md @@ -2,7 +2,7 @@ title: ClearML Modules --- -- [**ClearML Python Package**](../getting_started/ds/ds_first_steps.md#install-clearml) (`clearml`) for integrating ClearML into your existing code-base. +- [**ClearML Python Package**](auto_log_exp#install-clearml) (`clearml`) for integrating ClearML into your existing code-base. - [**ClearML Server**](../deploying_clearml/clearml_server.md) (`clearml-server`) for storing task, model, and workflow data, and supporting the Web UI experiment manager. It is also the control plane for the MLOps. - [**ClearML Agent**](../clearml_agent.md) (`clearml-agent`), the MLOps orchestration agent. Enabling task and workflow reproducibility, and scalability. - [**ClearML Data**](../clearml_data/clearml_data.md) (`clearml-data`) data management and versioning on top of file-systems/object-storage. diff --git a/docs/getting_started/auto_log_exp.md b/docs/getting_started/auto_log_exp.md new file mode 100644 index 00000000..e554bebf --- /dev/null +++ b/docs/getting_started/auto_log_exp.md @@ -0,0 +1,59 @@ +--- +title: Auto-log Experiments +--- + +In ClearML, experiments are organized as [Tasks](../fundamentals/task.md). + +When you integrate the ClearML SDK with your code, the ClearML task manager automatically captures: +* Source code and uncommitted changes +* Installed packages +* General information such as machine details, runtime, creation date etc. +* Model files, parameters, scalars, and plots from popular ML frameworks such as TensorFlow and PyTorch (see list of [supported frameworks](../clearml_sdk/task_sdk.md#automatic-logging)) +* Console output + +:::tip Automatic logging control +To control what ClearML automatically logs, see this [FAQ](../faq.md#controlling_logging). +::: + +## To Auto-log Your Experiments + +1. Install `clearml` and connect it to the ClearML Server (see [instructions](../clearml_sdk/clearml_sdk.md)) +1. At the beginning of your code, import the `clearml` package: + + ```python + from clearml import Task + ``` + + :::tip Full Automatic Logging + To ensure full automatic logging, it is recommended to import the `clearml` package at the top of your entry script. + ::: + +1. Initialize the Task object in your `main()` function, or the beginning of the script. + + ```python + task = Task.init(project_name='great project', task_name='best task') + ``` + + If the project does not already exist, a new one is created automatically. + + The console should display the following output: + + ``` + ClearML Task: created new task id=1ca59ef1f86d44bd81cb517d529d9e5a + 2021-07-25 13:59:09 + ClearML results page: https://app.clear.ml/projects/4043a1657f374e9298649c6ba72ad233/experiments/1ca59ef1f86d44bd81cb517d529d9e5a/output/log + 2025-01-25 13:59:16 + ``` + +1. Click the results page link to go to the [task's detail page in the ClearML WebApp](../webapp/webapp_exp_track_visual.md), + where you can monitor the task's status, view all its logged data, visualize its results, and more! + + ![Info panel](../img/webapp_tracking_40.png#light-mode-only) + ![Info panel](../img/webapp_tracking_40_dark.png#dark-mode-only) + +**That's it!** You are done integrating ClearML with your code :) + +Now, [command-line arguments](../fundamentals/hyperparameters.md#tracking-hyperparameters), [console output](../fundamentals/logger.md#types-of-logged-results), TensorBoard and Matplotlib, and much more will automatically be +logged in the UI under the created Task. + +Sit back, relax, and watch your models converge :) \ No newline at end of file diff --git a/docs/getting_started/ds/best_practices.md b/docs/getting_started/ds/best_practices.md index 1952e3c4..04deaa54 100644 --- a/docs/getting_started/ds/best_practices.md +++ b/docs/getting_started/ds/best_practices.md @@ -24,7 +24,7 @@ During early stages of model development, while code is still being modified hea These setups can be folded into each other and that's great! If you have a GPU machine for each researcher, that's awesome! The goal of this phase is to get a code, dataset, and environment set up, so you can start digging to find the best model! -- [ClearML SDK](../../clearml_sdk/clearml_sdk.md) should be integrated into your code (check out [Getting Started](ds_first_steps.md)). +- [ClearML SDK](../../clearml_sdk/clearml_sdk.md) should be integrated into your code (check out [ClearML Setup](../../clearml_sdk/clearml_sdk_setup.md)). This helps visualizing the results and tracking progress. - [ClearML Agent](../../clearml_agent.md) helps moving your work to other machines without the hassle of rebuilding the environment every time, while also creating an easy queue interface that easily lets you drop your tasks to be executed one by one diff --git a/docs/getting_started/ds/ds_second_steps.md b/docs/getting_started/ds/ds_second_steps.md deleted file mode 100644 index b1a98408..00000000 --- a/docs/getting_started/ds/ds_second_steps.md +++ /dev/null @@ -1,193 +0,0 @@ ---- -title: Next Steps ---- - -So, you've already [installed ClearML's Python package](ds_first_steps.md) and run your first task! - -Now, you'll learn how to track Hyperparameters, Artifacts, and Metrics! - -## Accessing Tasks - -Every previously executed experiment is stored as a Task. -A Task's project and name can be changed after it has been executed. -A Task is also automatically assigned an auto-generated unique identifier (UUID string) that cannot be changed and always locates the same Task in the system. - -Retrieve a Task object programmatically by querying the system based on either the Task ID, -or project and name combination. You can also query tasks based on their properties, like tags (see [Querying Tasks](../../clearml_sdk/task_sdk.md#querying--searching-tasks)). - -```python -prev_task = Task.get_task(task_id='123456deadbeef') -``` - -Once you have a Task object you can query the state of the Task, get its model(s), scalars, parameters, etc. - -## Log Hyperparameters - -For full reproducibility, it's paramount to save each task's hyperparameters. Since hyperparameters can have substantial impact -on model performance, saving and comparing them between tasks is sometimes the key to understanding model behavior. - -ClearML supports logging `argparse` module arguments out of the box, so once ClearML is integrated into the code, it automatically logs all parameters provided to the argument parser. - -You can also log parameter dictionaries (very useful when parsing an external configuration file and storing as a dict object), -whole configuration files, or even custom objects or [Hydra](https://hydra.cc/docs/intro/) configurations! - -```python -params_dictionary = {'epochs': 3, 'lr': 0.4} -task.connect(params_dictionary) -``` - -See [Configuration](../../clearml_sdk/task_sdk.md#configuration) for all hyperparameter logging options. - -## Log Artifacts - -ClearML lets you easily store the output products of a task: Model snapshot / weights file, a preprocessing of your data, feature representation of data and more! - -Essentially, artifacts are files (or Python objects) uploaded from a script and are stored alongside the Task. -These artifacts can be easily accessed by the web UI or programmatically. - -Artifacts can be stored anywhere, either on the ClearML server, or any object storage solution or shared folder. -See all [storage capabilities](../../integrations/storage.md). - - -### Adding Artifacts - -Upload a local file containing the preprocessed results of the data: -```python -task.upload_artifact(name='data', artifact_object='/path/to/preprocess_data.csv') -``` - -You can also upload an entire folder with all its content by passing the folder (the folder will be zipped and uploaded as a single zip file). -```python -task.upload_artifact(name='folder', artifact_object='/path/to/folder/') -``` - -Lastly, you can upload an instance of an object; Numpy/Pandas/PIL Images are supported with `npz`/`csv.gz`/`jpg` formats accordingly. -If the object type is unknown, ClearML pickles it and uploads the pickle file. - -```python -numpy_object = np.eye(100, 100) -task.upload_artifact(name='features', artifact_object=numpy_object) -``` - -For more artifact logging options, see [Artifacts](../../clearml_sdk/task_sdk.md#artifacts). - -### Using Artifacts - -Logged artifacts can be used by other Tasks, whether it's a pre-trained Model or processed data. -To use an artifact, first you have to get an instance of the Task that originally created it, -then you either download it and get its path, or get the artifact object directly. - -For example, using a previously generated preprocessed data. - -```python -preprocess_task = Task.get_task(task_id='preprocessing_task_id') -local_csv = preprocess_task.artifacts['data'].get_local_copy() -``` - -`task.artifacts` is a dictionary where the keys are the artifact names, and the returned object is the artifact object. -Calling `get_local_copy()` returns a local cached copy of the artifact. Therefore, next time you execute the code, you don't -need to download the artifact again. -Calling `get()` gets a deserialized pickled object. - -Check out the [artifacts retrieval](https://github.com/clearml/clearml/blob/master/examples/reporting/artifacts_retrieval.py) example code. - -### Models - -Models are a special kind of artifact. -Models created by popular frameworks (such as PyTorch, TensorFlow, Scikit-learn) are automatically logged by ClearML. -All snapshots are automatically logged. In order to make sure you also automatically upload the model snapshot (instead of saving its local path), -pass a storage location for the model files to be uploaded to. - -For example, upload all snapshots to an S3 bucket: -```python -task = Task.init( - project_name='examples', - task_name='storing model', - output_uri='s3://my_models/' -) -``` - -Now, whenever the framework (TensorFlow/Keras/PyTorch etc.) stores a snapshot, the model file is automatically uploaded to the bucket to a specific folder for the task. - -Loading models by a framework is also logged by the system; these models appear in a task's **Artifacts** tab, -under the "Input Models" section. - -Check out model snapshots examples for [TensorFlow](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py), -[PyTorch](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py), -[Keras](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py), -[scikit-learn](https://github.com/clearml/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py). - -#### Loading Models -Loading a previously trained model is quite similar to loading artifacts. - -```python -prev_task = Task.get_task(task_id='the_training_task') -last_snapshot = prev_task.models['output'][-1] -local_weights_path = last_snapshot.get_local_copy() -``` - -Like before, you have to get the instance of the task training the original weights files, then you can query the task for its output models (a list of snapshots), and get the latest snapshot. -:::note -Using TensorFlow, the snapshots are stored in a folder, meaning the `local_weights_path` will point to a folder containing your requested snapshot. -::: -As with artifacts, all models are cached, meaning the next time you run this code, no model needs to be downloaded. -Once one of the frameworks will load the weights file, the running task will be automatically updated with "Input Model" pointing directly to the original training Task's Model. -This feature lets you easily get a full genealogy of every trained and used model by your system! - -## Log Metrics - -Full metrics logging is the key to finding the best performing model! -By default, ClearML automatically captures and logs everything reported to TensorBoard and Matplotlib. - -Since not all metrics are tracked that way, you can also manually report metrics using a [`Logger`](../../fundamentals/logger.md) object. - -You can log everything, from time series data and confusion matrices to HTML, Audio, and Video, to custom plotly graphs! Everything goes! - -![Experiment plots](../../img/report_plotly.png#light-mode-only) -![Experiment plots](../../img/report_plotly_dark.png#dark-mode-only) - -Once everything is neatly logged and displayed, use the [comparison tool](../../webapp/webapp_exp_comparing.md) to find the best configuration! - - -## Track Tasks - -The task table is a powerful tool for creating dashboards and views of your own projects, your team's projects, or the entire development. - -![Task table](../../img/webapp_experiment_table.png#light-mode-only) -![Task table](../../img/webapp_experiment_table_dark.png#dark-mode-only) - - -### Creating Leaderboards -Customize the [task table](../../webapp/webapp_exp_table.md) to fit your own needs, adding desired views of parameters, metrics, and tags. -You can filter and sort based on parameters and metrics, so creating custom views is simple and flexible. - -Create a dashboard for a project, presenting the latest Models and their accuracy scores, for immediate insights. - -It can also be used as a live leaderboard, showing the best performing tasks' status, updated in real time. -This is helpful to monitor your projects' progress, and to share it across the organization. - -Any page is sharable by copying the URL from the address bar, allowing you to bookmark leaderboards or to send an exact view of a specific task or a comparison page. - -You can also tag Tasks for visibility and filtering allowing you to add more information on the execution of the task. -Later you can search based on task name in the search bar, and filter tasks based on their tags, parameters, status, and more. - -## What's Next? - -This covers the basics of ClearML! Running through this guide you've learned how to log Parameters, Artifacts and Metrics! - -If you want to learn more look at how we see the data science process in our [best practices](best_practices.md) page, -or check these pages out: - -- Scale you work and deploy [ClearML Agents](../../clearml_agent.md) -- Develop on remote machines with [ClearML Session](../../apps/clearml_session.md) -- Structure your work and put it into [Pipelines](../../pipelines/pipelines.md) -- Improve your tasks with [Hyperparameter Optimization](../../hpo.md) -- Check out ClearML's integrations with your favorite ML frameworks like [TensorFlow](../../integrations/tensorflow.md), - [PyTorch](../../integrations/pytorch.md), [Keras](../../integrations/keras.md), - and more - -## YouTube Playlist - -All these tips and tricks are also covered in ClearML's **Getting Started** series on YouTube. Go check it out :) - -[![Watch the video](https://img.youtube.com/vi/kyOfwVg05EM/hqdefault.jpg)](https://www.youtube.com/watch?v=kyOfwVg05EM&list=PLMdIlCuMqSTnoC45ME5_JnsJX0zWqDdlO&index=3) \ No newline at end of file diff --git a/docs/getting_started/logging_using_artifacts.md b/docs/getting_started/logging_using_artifacts.md new file mode 100644 index 00000000..3df11210 --- /dev/null +++ b/docs/getting_started/logging_using_artifacts.md @@ -0,0 +1,120 @@ +--- +title: Logging and Using Task Artifacts +--- + +:::note +This tutorial assumes that you've already set up [ClearML](../clearml_sdk/clearml_sdk_setup.md) +::: + + +ClearML lets you easily store a task's output products--or **Artifacts**: +* [Model](#models) snapshot / weights file +* Preprocessing of your data +* Feature representation of data +* And more! + +**Artifacts** are files or Python objects that are uploaded and stored alongside the Task. +These artifacts can be easily accessed by the web UI or programmatically. + +Artifacts can be stored anywhere, either on the ClearML Server, or any object storage solution or shared folder. +See all [storage capabilities](../integrations/storage.md). + + +## Adding Artifacts + +Let's create a [Task](../fundamentals/task.md) and add some artifacts to it. + +1. Create a task using [`Task.init()`](../references/sdk/task.md#taskinit) + + ```python + from clearml import Task + + task = Task.init(project_name='great project', task_name='task with artifacts') + ``` + +1. Upload a local **file** using [`Task.upload_folder()`](../references/sdk/task.md#upload_artifact) and specifying the artifact's + name and its path: + + ```python + task.upload_artifact(name='data', artifact_object='/path/to/preprocess_data.csv') + ``` + +1. Upload an **entire folder** with all its content by passing the folder path (the folder will be zipped and uploaded as a single zip file). + + ```python + task.upload_artifact(name='folder', artifact_object='/path/to/folder/') + ``` + +1. Upload an instance of an object. Numpy/Pandas/PIL Images are supported with `npz`/`csv.gz`/`jpg` formats accordingly. + If the object type is unknown, ClearML pickles it and uploads the pickle file. + + ```python + numpy_object = np.eye(100, 100) + task.upload_artifact(name='features', artifact_object=numpy_object) + ``` + +For more artifact logging options, see [Artifacts](../clearml_sdk/task_sdk.md#artifacts). + +### Using Artifacts + +Logged artifacts can be used by other Tasks, whether it's a pre-trained Model or processed data. +To use an artifact, first you have to get an instance of the Task that originally created it, +then you either download it and get its path, or get the artifact object directly. + +For example, using a previously generated preprocessed data. + +```python +preprocess_task = Task.get_task(task_id='preprocessing_task_id') +local_csv = preprocess_task.artifacts['data'].get_local_copy() +``` + +`task.artifacts` is a dictionary where the keys are the artifact names, and the returned object is the artifact object. +Calling `get_local_copy()` returns a local cached copy of the artifact. Therefore, next time you execute the code, you don't +need to download the artifact again. +Calling `get()` gets a deserialized pickled object. + +Check out the [artifacts retrieval](https://github.com/clearml/clearml/blob/master/examples/reporting/artifacts_retrieval.py) example code. + +### Models + +Models are a special kind of artifact. +Models created by popular frameworks (such as PyTorch, TensorFlow, Scikit-learn) are automatically logged by ClearML. +All snapshots are automatically logged. In order to make sure you also automatically upload the model snapshot (instead of saving its local path), +pass a storage location for the model files to be uploaded to. + +For example, upload all snapshots to an S3 bucket: +```python +task = Task.init( + project_name='examples', + task_name='storing model', + output_uri='s3://my_models/' +) +``` + +Now, whenever the framework (TensorFlow/Keras/PyTorch etc.) stores a snapshot, the model file is automatically uploaded to the bucket to a specific folder for the task. + +Loading models by a framework is also logged by the system; these models appear in a task's **Artifacts** tab, +under the "Input Models" section. + +Check out model snapshots examples for [TensorFlow](https://github.com/clearml/clearml/blob/master/examples/frameworks/tensorflow/tensorflow_mnist.py), +[PyTorch](https://github.com/clearml/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py), +[Keras](https://github.com/clearml/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py), +[scikit-learn](https://github.com/clearml/clearml/blob/master/examples/frameworks/scikit-learn/sklearn_joblib_example.py). + +#### Loading Models +Loading a previously trained model is quite similar to loading artifacts. + +```python +prev_task = Task.get_task(task_id='the_training_task') +last_snapshot = prev_task.models['output'][-1] +local_weights_path = last_snapshot.get_local_copy() +``` + +Like before, you have to get the instance of the task training the original weights files, then you can query the task for its output models (a list of snapshots), and get the latest snapshot. +:::note +Using TensorFlow, the snapshots are stored in a folder, meaning the `local_weights_path` will point to a folder containing your requested snapshot. +::: +As with artifacts, all models are cached, meaning the next time you run this code, no model needs to be downloaded. +Once one of the frameworks will load the weights file, the running task will be automatically updated with "Input Model" pointing directly to the original training Task's Model. +This feature lets you easily get a full genealogy of every trained and used model by your system! + diff --git a/docs/getting_started/mlops/mlops_first_steps.md b/docs/getting_started/mlops/mlops_first_steps.md deleted file mode 100644 index 34635cd3..00000000 --- a/docs/getting_started/mlops/mlops_first_steps.md +++ /dev/null @@ -1,225 +0,0 @@ ---- -title: First Steps ---- - -:::note -This tutorial assumes that you've already [signed up](https://app.clear.ml) to ClearML -::: - -ClearML provides tools for **automation**, **orchestration**, and **tracking**, all key in performing effective MLOps and LLMOps. - -Effective MLOps and LLMOps rely on the ability to scale work beyond one's own computer. Moving from your own machine can be time-consuming. -Even assuming that you have all the drivers and applications installed, you still need to manage multiple Python environments -for different packages / package versions, or worse - manage different Dockers for different package versions. - -Not to mention, when working on remote machines, executing experiments, tracking what's running where, and making sure machines -are fully utilized at all times become daunting tasks. - -This can create overhead that derails you from your core work! - -ClearML Agent was designed to deal with such issues and more! It is a tool responsible for executing tasks on remote machines: on-premises or in the cloud! ClearML Agent provides the means to reproduce and track tasks in your -machine of choice through the ClearML WebApp with no need for additional code. - -The agent will set up the environment for a specific Task's execution (inside a Docker, or bare-metal), install the -required Python packages, and execute and monitor the process. - - -## Set up an Agent - -1. Install the agent: - - ```bash - pip install clearml-agent - ``` - -1. Connect the agent to the server by [creating credentials](https://app.clear.ml/settings/workspace-configuration), then run this: - - ```bash - clearml-agent init - ``` - - :::note - If you've already created credentials, you can copy-paste the default agent section from [here](https://github.com/clearml/clearml-agent/blob/master/docs/clearml.conf#L15) (this is optional. If the section is not provided the default values will be used) - ::: - -1. Start the agent's daemon and assign it to a [queue](../../fundamentals/agents_and_queues.md#what-is-a-queue): - - ```bash - clearml-agent daemon --queue default - ``` - - A queue is an ordered list of Tasks that are scheduled for execution. The agent will pull Tasks from its assigned - queue (`default` in this case), and execute them one after the other. Multiple agents can listen to the same queue - (or even multiple queues), but only a single agent will pull a Task to be executed. - -:::tip Agent Deployment Modes -ClearML Agents can be deployed in: -* [Virtual environment mode](../../clearml_agent/clearml_agent_execution_env.md): Agent creates a new venv to execute a task. -* [Docker mode](../../clearml_agent/clearml_agent_execution_env.md#docker-mode): Agent executes a task inside a -Docker container. - -For more information, see [Running Modes](../../fundamentals/agents_and_queues.md#running-modes). -::: - -## Clone a Task -Tasks can be reproduced (cloned) for validation or as a baseline for further experimentation. -Cloning a task duplicates the task's configuration, but not its outputs. - -**To clone a task in the ClearML WebApp:** -1. Click on any project card to open its [task table](../../webapp/webapp_exp_table.md). -1. Right-click one of the tasks on the table. -1. Click **Clone** in the context menu, which will open a **CLONE TASK** window. -1. Click **CLONE** in the window. - -The newly cloned task will appear and its info panel will slide open. The cloned task is in draft mode, so -it can be modified. You can edit the Git / code references, control the Python packages to be installed, specify the -Docker container image to be used, or change the hyperparameters and configuration files. See [Modifying Tasks](../../webapp/webapp_exp_tuning.md#modifying-tasks) for more information about editing tasks in the UI. - -## Enqueue a Task -Once you have set up a task, it is now time to execute it. - -**To execute a task through the ClearML WebApp:** -1. Right-click your draft task (the context menu is also available through the Menu - button on the top right of the task's info panel) -1. Click **ENQUEUE,** which will open the **ENQUEUE TASK** window -1. In the window, select `default` in the queue menu -1. Click **ENQUEUE** - -This action pushes the task into the `default` queue. The task's status becomes *Pending* until an agent -assigned to the queue fetches it, at which time the task's status becomes *Running*. The agent executes the -task, and the task can be [tracked and its results visualized](../../webapp/webapp_exp_track_visual.md). - - -## Programmatic Interface - -The cloning, modifying, and enqueuing actions described above can also be performed programmatically. - -### First Steps -#### Access Previously Executed Tasks -All Tasks in the system can be accessed through their unique Task ID, or based on their properties using the [`Task.get_task`](../../references/sdk/task.md#taskget_task) -method. For example: -```python -from clearml import Task - -executed_task = Task.get_task(task_id='aabbcc') -``` - -Once a specific Task object has been obtained, it can be cloned, modified, and more. See [Advanced Usage](#advanced-usage). - -#### Clone a Task - -To duplicate a task, use the [`Task.clone`](../../references/sdk/task.md#taskclone) method, and input either a -Task object or the Task's ID as the `source_task` argument. -```python -cloned_task = Task.clone(source_task=executed_task) -``` - -#### Enqueue a Task -To enqueue the task, use the [`Task.enqueue`](../../references/sdk/task.md#taskenqueue) method, and input the Task object -with the `task` argument, and the queue to push the task into with `queue_name`. - -```python -Task.enqueue(task=cloned_task, queue_name='default') -``` - -### Advanced Usage -Before execution, use a variety of programmatic methods to manipulate a task object. - -#### Modify Hyperparameters -[Hyperparameters](../../fundamentals/hyperparameters.md) are an integral part of Machine Learning code as they let you -control the code without directly modifying it. Hyperparameters can be added from anywhere in your code, and ClearML supports multiple ways to obtain them! - -Users can programmatically change cloned tasks' parameters. - -For example: -```python -from clearml import Task - -cloned_task = Task.clone(task_id='aabbcc') -cloned_task.set_parameter(name='internal/magic', value=42) -``` - -#### Report Artifacts -Artifacts are files created by your task. Users can upload [multiple types of data](../../clearml_sdk/task_sdk.md#logging-artifacts), -objects and files to a task anywhere from code. - -```python -import numpy as np -from clearml import Task - -Task.current_task().upload_artifact(name='a_file', artifact_object='local_file.bin') -Task.current_task().upload_artifact(name='numpy', artifact_object=np.ones(4,4)) -``` - -Artifacts serve as a great way to pass and reuse data between tasks. Artifacts can be [retrieved](../../clearml_sdk/task_sdk.md#using-artifacts) -by accessing the Task that created them. These artifacts can be modified and uploaded to other tasks. - -```python -from clearml import Task - -executed_task = Task.get_task(task_id='aabbcc') -# artifact as a file -local_file = executed_task.artifacts['file'].get_local_copy() -# artifact as object -a_numpy = executed_task.artifacts['numpy'].get() -``` - -By facilitating the communication of complex objects between tasks, artifacts serve as the foundation of ClearML's [Data Management](../../clearml_data/clearml_data.md) -and [pipeline](../../pipelines/pipelines.md) solutions. - -#### Log Models -Logging models into the model repository is the easiest way to integrate the development process directly with production. -Any model stored by a supported framework (Keras / TensorFlow / PyTorch / Joblib etc.) will be automatically logged into ClearML. - -ClearML also supports methods to explicitly log models. Models can be automatically stored on a preferred storage medium -(S3 bucket, Google storage, etc.). - -#### Log Metrics -Log as many metrics as you want from your processes using the [Logger](../../fundamentals/logger.md) module. This -improves the visibility of your processes' progress. - -```python -from clearml import Logger - -Logger.current_logger().report_scalar( - graph='metric', - series='variant', - value=13.37, - iteration=counter -) -``` - -You can also retrieve reported scalars for programmatic analysis: -```python -from clearml import Task - -executed_task = Task.get_task(task_id='aabbcc') -# get a summary of the min/max/last value of all reported scalars -min_max_values = executed_task.get_last_scalar_metrics() -# get detailed graphs of all scalars -full_scalars = executed_task.get_reported_scalars() -``` - -#### Query Tasks -You can also search and query Tasks in the system. Use the [`Task.get_tasks`](../../references/sdk/task.md#taskget_tasks) -class method to retrieve Task objects and filter based on the specific values of the Task - status, parameters, metrics and more! - -```python -from clearml import Task - -tasks = Task.get_tasks( - project_name='examples', - task_name='partial_name_match', - task_filter={'status': 'in_progress'} -) -``` - -#### Manage Your Data -Data is probably one of the biggest factors that determines the success of a project. Associating a model's data with -the model's configuration, code, and results (such as accuracy) is key to deducing meaningful insights into model behavior. - -[ClearML Data](../../clearml_data/clearml_data.md) lets you version your data, so it's never lost, fetch it from every -machine with minimal code changes, and associate data to task results. - -Logging data can be done via command line, or programmatically. If any preprocessing code is involved, ClearML logs it -as well! Once data is logged, it can be used by other tasks. diff --git a/docs/getting_started/reproduce_tasks.md b/docs/getting_started/reproduce_tasks.md new file mode 100644 index 00000000..f580cc1a --- /dev/null +++ b/docs/getting_started/reproduce_tasks.md @@ -0,0 +1,82 @@ +--- +title: Reproduce Tasks +--- + +:::note +This tutorial assumes that you've already set up [ClearML](../clearml_sdk/clearml_sdk_setup.md) and [ClearML Agent](../clearml_agent/clearml_agent_setup.md). +::: + +Tasks can be reproduced--or **Cloned**--for validation or as a baseline for further experimentation. When you initialize a task in your +code, ClearML logs everything needed to reproduce your task and its environment: +* Uncommitted changes +* Used packages and their versions +* Parameters +* and more + +Cloning a task duplicates the task's configuration, but not its outputs. + +ClearML offers two ways to clone your task: +* [Via the WebApp](#via-the-webapp)--no further code required +* [Via programmatic interface](#via-programmatic-interface) using the `clearml` Python package + +Once you have cloned your task, you can modify its setup, and then execute it remotely on a machine of your choice using a ClearML Agent. + +## Via the WebApp + +**To clone a task in the ClearML WebApp:** +1. Click on any project card to open its [task table](../webapp/webapp_exp_table.md). +1. Right-click the task you want to reproduce. +1. Click **Clone** in the context menu, which will open a **CLONE TASK** window. +1. Click **CLONE** in the window. + +The newly cloned task's details page will open up. The cloned task is in *draft* mode, which means +it can be modified. You can edit any of the Task's setup details, including: +* Git and/or code references +* Python packages to be installed +* Container image to be used + +You can adjust the values of the task's hyperparameters and configuration files. See [Modifying Tasks](../webapp/webapp_exp_tuning.md#modifying-tasks) for more +information about editing tasks in the UI. + +### Enqueue a Task +Once you have set up a task, it is now time to execute it. + +**To execute a task through the ClearML WebApp:** +1. In the task's details page, click "Menu" Menu +1. Click **ENQUEUE** to open the **ENQUEUE TASK** window +1. In the window, select `default` in the `Queue` menu +1. Click **ENQUEUE** + +This action pushes the task into the `default` queue. The task's status becomes *Pending* until an agent +assigned to the queue fetches it, at which time the task's status becomes *Running*. The agent executes the +task, and the task can be [tracked and its results visualized](../webapp/webapp_exp_track_visual.md). + + +## Via Programmatic Interface + +The cloning, modifying, and enqueuing actions described above can also be performed programmatically using `clearml`. + + +### Clone the Task + +To duplicate the task, use [`Task.clone()`](../references/sdk/task.md#taskclone), and input either a +Task object or the Task's ID as the `source_task` argument. + +```python +cloned_task = Task.clone(source_task='qw03485je3hap903ere54') +``` + +The cloned task is in *draft* mode, which means it can be modified. For modification options, such as setting new parameter +values, see [Task SDK](../clearml_sdk/task_sdk.md). + +### Enqueue the Task +To enqueue the task, use [`Task.enqueue()`](../references/sdk/task.md#taskenqueue), and input the Task object +with the `task` argument, and the queue to push the task into with `queue_name`. + +```python +Task.enqueue(task=cloned_task, queue_name='default') +``` + +This action pushes the task into the `default` queue. The task's status becomes *Pending* until an agent +assigned to the queue fetches it, at which time the task's status becomes *Running*. The agent executes the +task, and the task can be [tracked and its results visualized](../webapp/webapp_exp_track_visual.md). \ No newline at end of file diff --git a/docs/getting_started/track_tasks.md b/docs/getting_started/track_tasks.md new file mode 100644 index 00000000..2dccb6a9 --- /dev/null +++ b/docs/getting_started/track_tasks.md @@ -0,0 +1,46 @@ +--- +title: Track Tasks +--- + +Every ClearML [task](../fundamentals/task.md) you create can be found in the **All Tasks** table and in its project's +task table. + +The task table is a powerful tool for creating dashboards and views of your own projects, your team's projects, or the +entire development. + +![Task table](../img/webapp_experiment_table.png#light-mode-only) +![Task table](../img/webapp_experiment_table_dark.png#dark-mode-only) + +Customize the [task table](../webapp/webapp_exp_table.md) to fit your own needs by adding views of parameters, metrics, and tags. +Filter and sort based on various criteria, such as parameters and metrics, making it simple to create custom +views. This allows you to: + +* Create a dashboard for a project, presenting the latest model accuracy scores, for immediate insights. +* Create a live leaderboard displaying the best-performing tasks, updated in real time +* Monitor a projects' progress and share it across the organization. + +## Creating Leaderboards + +To create a leaderboard: + +1. Select a project in the ClearML WebApp and go to its task table +1. Customize the column selection. Click "Settings" Setting Gear + to view and select columns to display. +1. Filter tasks by name using the search bar to find tasks containing any search term +1. Filter by other categories by clicking "Filter" Filter + on the relevant column. There are a few types of filters: + * Value set - Choose which values to include from a list of all values in the column + * Numerical ranges - Insert minimum and/or maximum value + * Date ranges - Insert starting and/or ending date and time + * Tags - Choose which tags to filter by from a list of all tags used in the column. + * Filter by multiple tag values using the **ANY** or **ALL** options, which correspond to the logical "AND" and "OR" respectively. These + options appear on the top of the tag list. + * Filter by the absence of a tag (logical "NOT") by clicking its checkbox twice. An `X` will appear in the tag's checkbox. +1. Enable auto-refresh for real-time monitoring + +For more detailed instructions, see the [Tracking Leaderboards Tutorial](../guides/ui/building_leader_board.md). + +## Sharing Leaderboards + +Bookmark the URL of your customized leaderboard to save and share your view. The URL contains all parameters and values +for your specific leaderboard view. \ No newline at end of file diff --git a/docs/guides/clearml-task/clearml_task_tutorial.md b/docs/guides/clearml-task/clearml_task_tutorial.md index 085f352c..99b86e0f 100644 --- a/docs/guides/clearml-task/clearml_task_tutorial.md +++ b/docs/guides/clearml-task/clearml_task_tutorial.md @@ -7,7 +7,7 @@ on a remote or local machine, from a remote repository and your local machine. ### Prerequisites -- [`clearml`](../../getting_started/ds/ds_first_steps.md) Python package installed and configured +- [`clearml`](../../clearml_sdk/clearml_sdk_setup) Python package installed and configured - [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) running on at least one machine (to execute the task), configured to listen to `default` queue ### Executing Code from a Remote Repository diff --git a/docs/guides/clearml_agent/executable_exp_containers.md b/docs/guides/clearml_agent/executable_exp_containers.md index 35cd57da..884bc53a 100644 --- a/docs/guides/clearml_agent/executable_exp_containers.md +++ b/docs/guides/clearml_agent/executable_exp_containers.md @@ -9,7 +9,7 @@ script. ## Prerequisites * [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) installed and configured -* [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured +* [`clearml`](../../clearml_sdk/clearml_sdk_setup#install-clearml) installed and configured * [clearml](https://github.com/clearml/clearml) repo cloned (`git clone https://github.com/clearml/clearml.git`) ## Creating the ClearML Task diff --git a/docs/guides/clearml_agent/exp_environment_containers.md b/docs/guides/clearml_agent/exp_environment_containers.md index 0398e017..388d932e 100644 --- a/docs/guides/clearml_agent/exp_environment_containers.md +++ b/docs/guides/clearml_agent/exp_environment_containers.md @@ -11,7 +11,7 @@ be used when running optimization tasks. ## Prerequisites * [`clearml-agent`](../../clearml_agent/clearml_agent_setup.md#installation) installed and configured -* [`clearml`](../../getting_started/ds/ds_first_steps.md#install-clearml) installed and configured +* [`clearml`](../../clearml_sdk/clearml_sdk_setup#install-clearml) installed and configured * [clearml](https://github.com/clearml/clearml) repo cloned (`git clone https://github.com/clearml/clearml.git`) ## Creating the ClearML Task diff --git a/docs/guides/frameworks/tensorflow/integration_keras_tuner.md b/docs/guides/frameworks/tensorflow/integration_keras_tuner.md index 4635afd9..5db4d120 100644 --- a/docs/guides/frameworks/tensorflow/integration_keras_tuner.md +++ b/docs/guides/frameworks/tensorflow/integration_keras_tuner.md @@ -3,10 +3,10 @@ title: Keras Tuner --- :::tip -If you are not already using ClearML, see [Getting Started](../../../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + Integrate ClearML into code that uses [Keras Tuner](https://www.tensorflow.org/tutorials/keras/keras_tuner). By specifying `ClearMLTunerLogger` (see [kerastuner.py](https://github.com/clearml/clearml/blob/master/clearml/external/kerastuner.py)) as the Keras Tuner logger, ClearML automatically logs scalars and hyperparameter optimization. diff --git a/docs/integrations/accelerate.md b/docs/integrations/accelerate.md index 6be0f9ab..8d5d685e 100644 --- a/docs/integrations/accelerate.md +++ b/docs/integrations/accelerate.md @@ -9,7 +9,7 @@ such as required packages and uncommitted changes, and supports reporting scalar ## Setup -To use Accelerate's ClearML tracker, make sure that `clearml` is [installed and set up](../getting_started/ds/ds_first_steps.md#install-clearml) +To use Accelerate's ClearML tracker, make sure that `clearml` is [installed and set up](../clearml_sdk/clearml_sdk_setup#install-clearml) in your environment, and use the `log_with` parameter when instantiating an `Accelerator`: ```python diff --git a/docs/integrations/autokeras.md b/docs/integrations/autokeras.md index a92eb852..dd90106f 100644 --- a/docs/integrations/autokeras.md +++ b/docs/integrations/autokeras.md @@ -3,7 +3,7 @@ title: AutoKeras --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup +If you are not already using ClearML, see [Getting Started](../clearml_sdk/clearml_sdk_setup) for setup instructions. ::: diff --git a/docs/integrations/catboost.md b/docs/integrations/catboost.md index ed5bd5df..77476e6b 100644 --- a/docs/integrations/catboost.md +++ b/docs/integrations/catboost.md @@ -3,7 +3,7 @@ title: CatBoost --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup +If you are not already using ClearML, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup) for setup instructions. ::: diff --git a/docs/integrations/click.md b/docs/integrations/click.md index cf9298bd..c1169615 100644 --- a/docs/integrations/click.md +++ b/docs/integrations/click.md @@ -3,7 +3,7 @@ title: Click --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup +If you are not already using ClearML, see [ClearML Setup](../clearml_sdk/clearml_sdk_setup) for setup instructions. ::: diff --git a/docs/integrations/fastai.md b/docs/integrations/fastai.md index e8fd03e5..62bc5e16 100644 --- a/docs/integrations/fastai.md +++ b/docs/integrations/fastai.md @@ -3,8 +3,7 @@ title: Fast.ai --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: ClearML integrates seamlessly with [fast.ai](https://www.fast.ai/), automatically logging its models and scalars. diff --git a/docs/integrations/hydra.md b/docs/integrations/hydra.md index d8a05c04..faaa41b0 100644 --- a/docs/integrations/hydra.md +++ b/docs/integrations/hydra.md @@ -3,8 +3,7 @@ title: Hydra --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: diff --git a/docs/integrations/ignite.md b/docs/integrations/ignite.md index 9b2de832..683292ab 100644 --- a/docs/integrations/ignite.md +++ b/docs/integrations/ignite.md @@ -3,8 +3,7 @@ title: PyTorch Ignite --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: [PyTorch Ignite](https://pytorch.org/ignite/index.html) is a library for training and evaluating neural networks in diff --git a/docs/integrations/jsonargparse.md b/docs/integrations/jsonargparse.md index 8f348e45..42cc2fa2 100644 --- a/docs/integrations/jsonargparse.md +++ b/docs/integrations/jsonargparse.md @@ -3,11 +3,11 @@ title: jsonargparse --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [jsonargparse](https://github.com/omni-us/jsonargparse) is a Python package for creating command-line interfaces. ClearML integrates seamlessly with `jsonargparse` and automatically logs its command-line parameters and connected configuration files. diff --git a/docs/integrations/keras.md b/docs/integrations/keras.md index 88a9d182..87c1c140 100644 --- a/docs/integrations/keras.md +++ b/docs/integrations/keras.md @@ -3,10 +3,10 @@ title: Keras --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + ClearML integrates with [Keras](https://keras.io/) out-of-the-box, automatically logging its models, scalars, TensorFlow definitions, and TensorBoard outputs. diff --git a/docs/integrations/keras_tuner.md b/docs/integrations/keras_tuner.md index d75cffc1..705526b8 100644 --- a/docs/integrations/keras_tuner.md +++ b/docs/integrations/keras_tuner.md @@ -3,10 +3,10 @@ title: Keras Tuner --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [Keras Tuner](https://www.tensorflow.org/tutorials/keras/keras_tuner) is a library that helps you pick the optimal set of hyperparameters for training your models. ClearML integrates seamlessly with `kerastuner` and automatically logs task scalars, the output model, and hyperparameter optimization summary. diff --git a/docs/integrations/langchain.md b/docs/integrations/langchain.md index f4fef37d..c85f7551 100644 --- a/docs/integrations/langchain.md +++ b/docs/integrations/langchain.md @@ -3,10 +3,10 @@ title: LangChain --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [LangChain](https://github.com/langchain-ai/langchain) is a popular framework for developing applications powered by language models. You can integrate ClearML into your LangChain code using the built-in `ClearMLCallbackHandler`. This class is used to create a ClearML Task to log LangChain assets and metrics. diff --git a/docs/integrations/lightgbm.md b/docs/integrations/lightgbm.md index ddba0057..19e4eb23 100644 --- a/docs/integrations/lightgbm.md +++ b/docs/integrations/lightgbm.md @@ -3,10 +3,10 @@ title: LightGBM --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + ClearML integrates seamlessly with [LightGBM](https://github.com/microsoft/LightGBM), automatically logging its models, metric plots, and parameters. diff --git a/docs/integrations/matplotlib.md b/docs/integrations/matplotlib.md index 06714ff8..dde8e0cd 100644 --- a/docs/integrations/matplotlib.md +++ b/docs/integrations/matplotlib.md @@ -3,10 +3,10 @@ title: Matplotlib --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [Matplotlib](https://matplotlib.org/) is a Python library for data visualization. ClearML automatically captures plots and images created with `matplotlib`. diff --git a/docs/integrations/megengine.md b/docs/integrations/megengine.md index 05e2bed4..af734eb8 100644 --- a/docs/integrations/megengine.md +++ b/docs/integrations/megengine.md @@ -3,10 +3,10 @@ title: MegEngine --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + ClearML integrates seamlessly with [MegEngine](https://github.com/MegEngine/MegEngine), automatically logging its models. All you have to do is simply add two lines of code to your MegEngine script: diff --git a/docs/integrations/mmcv.md b/docs/integrations/mmcv.md index 8c77ca70..b9833820 100644 --- a/docs/integrations/mmcv.md +++ b/docs/integrations/mmcv.md @@ -7,10 +7,10 @@ title: MMCV v1.x ::: :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [MMCV](https://github.com/open-mmlab/mmcv/tree/1.x) is a computer vision framework developed by OpenMMLab. You can integrate ClearML into your code using the `mmcv` package's [`ClearMLLoggerHook`](https://mmcv.readthedocs.io/en/master/_modules/mmcv/runner/hooks/logger/clearml.html) class. This class is used to create a ClearML Task and to automatically log metrics. diff --git a/docs/integrations/mmengine.md b/docs/integrations/mmengine.md index 09d64256..733625f6 100644 --- a/docs/integrations/mmengine.md +++ b/docs/integrations/mmengine.md @@ -3,10 +3,10 @@ title: MMEngine --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [MMEngine](https://github.com/open-mmlab/mmengine) is a library for training deep learning models based on PyTorch. MMEngine supports ClearML through a builtin logger: It automatically logs task environment information, such as required packages and uncommitted changes, and supports reporting scalars, parameters, and debug samples. diff --git a/docs/integrations/monai.md b/docs/integrations/monai.md index 3dc98233..8b82e036 100644 --- a/docs/integrations/monai.md +++ b/docs/integrations/monai.md @@ -3,10 +3,10 @@ title: MONAI --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [MONAI](https://github.com/Project-MONAI/MONAI) is a PyTorch-based, open-source framework for deep learning in healthcare imaging. You can integrate ClearML into your code using MONAI's built-in handlers: [`ClearMLImageHandler`, `ClearMLStatsHandler`](#clearmlimagehandler-and-clearmlstatshandler), and [`ModelCheckpoint`](#modelcheckpoint). diff --git a/docs/integrations/pytorch.md b/docs/integrations/pytorch.md index 59191fc9..7f5ed5a0 100644 --- a/docs/integrations/pytorch.md +++ b/docs/integrations/pytorch.md @@ -3,10 +3,10 @@ title: PyTorch --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + ClearML integrates seamlessly with [PyTorch](https://pytorch.org/), automatically logging its models. All you have to do is simply add two lines of code to your PyTorch script: diff --git a/docs/integrations/pytorch_lightning.md b/docs/integrations/pytorch_lightning.md index 476489bd..68432d5b 100644 --- a/docs/integrations/pytorch_lightning.md +++ b/docs/integrations/pytorch_lightning.md @@ -3,10 +3,10 @@ title: PyTorch Lightning --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [PyTorch Lightning](https://github.com/Lightning-AI/lightning) is a framework that simplifies the process of training and deploying PyTorch models. ClearML seamlessly integrates with PyTorch Lightning, automatically logging PyTorch models, parameters supplied by [LightningCLI](https://lightning.ai/docs/pytorch/stable/cli/lightning_cli.html), and more. diff --git a/docs/integrations/scikit_learn.md b/docs/integrations/scikit_learn.md index 5a6afbab..3af78f22 100644 --- a/docs/integrations/scikit_learn.md +++ b/docs/integrations/scikit_learn.md @@ -3,10 +3,10 @@ title: scikit-learn --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + ClearML integrates seamlessly with [scikit-learn](https://scikit-learn.org/stable/), automatically logging models created with `joblib`. diff --git a/docs/integrations/seaborn.md b/docs/integrations/seaborn.md index ca2e1a2c..54b65583 100644 --- a/docs/integrations/seaborn.md +++ b/docs/integrations/seaborn.md @@ -3,10 +3,10 @@ title: Seaborn --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [seaborn](https://seaborn.pydata.org/) is a Python library for data visualization. ClearML automatically captures plots created using `seaborn`. All you have to do is add two lines of code to your script: diff --git a/docs/integrations/tensorboard.md b/docs/integrations/tensorboard.md index a0921c3b..317c983f 100644 --- a/docs/integrations/tensorboard.md +++ b/docs/integrations/tensorboard.md @@ -3,9 +3,10 @@ title: TensorBoard --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md). +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + [TensorBoard](https://www.tensorflow.org/tensorboard) is TensorFlow's data visualization toolkit. ClearML automatically captures all data logged to TensorBoard. All you have to do is add two lines of code to your script: diff --git a/docs/integrations/tensorboardx.md b/docs/integrations/tensorboardx.md index c8bf97bf..673b2c7b 100644 --- a/docs/integrations/tensorboardx.md +++ b/docs/integrations/tensorboardx.md @@ -3,7 +3,7 @@ title: TensorboardX --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md). +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: [TensorboardX](https://tensorboardx.readthedocs.io/en/latest/tutorial.html#what-is-tensorboard-x) is a data diff --git a/docs/integrations/tensorflow.md b/docs/integrations/tensorflow.md index 2c30f353..b87358bb 100644 --- a/docs/integrations/tensorflow.md +++ b/docs/integrations/tensorflow.md @@ -3,10 +3,10 @@ title: TensorFlow --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: + ClearML integrates with [TensorFlow](https://www.tensorflow.org/) out-of-the-box, automatically logging its models, definitions, scalars, as well as TensorBoard outputs. diff --git a/docs/integrations/xgboost.md b/docs/integrations/xgboost.md index 5a9ae5ad..8039bb68 100644 --- a/docs/integrations/xgboost.md +++ b/docs/integrations/xgboost.md @@ -3,8 +3,7 @@ title: XGBoost --- :::tip -If you are not already using ClearML, see [Getting Started](../getting_started/ds/ds_first_steps.md) for setup -instructions. +If you are not already using ClearML, see [ClearML Setup instructions](../clearml_sdk/clearml_sdk_setup). ::: ClearML integrates seamlessly with [XGBoost](https://xgboost.readthedocs.io/en/stable/), automatically logging its models, diff --git a/docusaurus.config.js b/docusaurus.config.js index cc9a9d21..c9f11b3e 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -92,7 +92,7 @@ module.exports = { position: 'left' }, { - to: '/docs/getting_started/ds/ds_first_steps', + to: '/docs/getting_started/auto_log_exp', label: 'Using ClearML', position: 'left' }, diff --git a/sidebars.js b/sidebars.js index 879b6fb4..55ea71f9 100644 --- a/sidebars.js +++ b/sidebars.js @@ -96,12 +96,12 @@ module.exports = { collapsible: true, label: 'Where do I start?', items: [ - {'Data Scientists': [ - 'getting_started/ds/ds_first_steps', - 'getting_started/ds/ds_second_steps', - ]}, + 'getting_started/auto_log_exp', + 'getting_started/track_tasks', + 'getting_started/reproduce_tasks', + 'getting_started/logging_using_artifacts', {'MLOps and LLMOps': [ - 'getting_started/mlops/mlops_first_steps', + 'getting_started/mlops/mlops_second_steps', ]} ], @@ -615,6 +615,7 @@ module.exports = { 'hyperdatasets/code_examples' ], installationSidebar: [ + 'clearml_sdk/clearml_sdk_setup', { type: 'category', collapsible: true,