diff --git a/docs/deploying_clearml/clearml_config_for_clearml_server.md b/docs/deploying_clearml/clearml_config_for_clearml_server.md index f0404e59..c7ffcacd 100644 --- a/docs/deploying_clearml/clearml_config_for_clearml_server.md +++ b/docs/deploying_clearml/clearml_config_for_clearml_server.md @@ -95,7 +95,7 @@ The configuration file's location depends upon the operating system: * Mac - `$HOME/clearml.conf` * Windows - `\User\\clearml.conf` -## Add ClearML to a configuration file +## Add ClearML to a Configuration File The setup wizard may indicate that a configuration file already exists. For example, if a **ClearML Agent** was previously configured, then a configuration file was created. The wizard does not edit or overwrite existing configuration files. diff --git a/docs/fundamentals/agents_and_queues.md b/docs/fundamentals/agents_and_queues.md index e202d315..2b5396ec 100644 --- a/docs/fundamentals/agents_and_queues.md +++ b/docs/fundamentals/agents_and_queues.md @@ -8,7 +8,7 @@ coupled with execution queues, addresses both these needs. The ClearML Agent is the base for **Automation** in ClearML and can be leveraged to build automated pipelines, launch custom services (e.g. a [monitor and alert service](https://github.com/allegroai/clearml/tree/master/examples/services/monitoring)) and more. -## What does a ClearML Agent do? +## What Does a ClearML Agent Do? An agent (also referred to as a Worker) allows users to execute code on any machine it's installed on, thus facilitating the scaling of data science work beyond one's own machine. The agent takes care of deploying the code to the target machine as well as setting up the entire execution environment: @@ -33,7 +33,7 @@ A ClearML Agent can service multiple queues in either of the following modes: * Strict priority: The agent services the higher priority queue before servicing lower priority ones. * Round robin: The agent pulls a single task from a queue then moves to service the next queue. -## Agent and Queue workflow +## Agent and Queue Workflow ![image](../img/clearml_agent_flow_diagram.png) @@ -52,7 +52,7 @@ The diagram above demonstrates a typical flow where an agent executes a task: While the agent is running, it continuously reports system metrics to the ClearML Server (These can be monitored in the **Workers and Queues** page). -## Resource management +## Resource Management Installing an Agent on machines allows it to monitor all the machine's status (GPU \ CPU \ Memory \ Network \ Disk IO). When managing multiple machines, this allows users to have an overview of their entire HW resources. What is the status of each machine, what is the expected workload on each machine and so on. @@ -70,7 +70,7 @@ Task to one of your queues, according to the amount of resources you want to all With queues and ClearML Agent, you can easily add and remove machines from the cluster, and you can reuse machines without the need for any dedicated containers or images. -## Additional features +## Additional Features Agents can be deployed bare-metal, with multiple instances allocating specific GPUs to the agents. They can also be deployed as dockers in a Kubernetes cluster. diff --git a/docs/fundamentals/hpo.md b/docs/fundamentals/hpo.md index 387d3212..269a410a 100644 --- a/docs/fundamentals/hpo.md +++ b/docs/fundamentals/hpo.md @@ -63,7 +63,7 @@ optimization. * **Custom** - `automation.optimization.SearchStrategy` - Use a custom class and inherit from the ClearML automation base strategy class -## Defining a hyperparameter optimization search example +## Defining a Hyperparameter Optimization Search Example 1. Import ClearML's automation modules: diff --git a/docs/fundamentals/pipelines.md b/docs/fundamentals/pipelines.md index c1293b25..302bc912 100644 --- a/docs/fundamentals/pipelines.md +++ b/docs/fundamentals/pipelines.md @@ -7,7 +7,7 @@ Tasks in a pipeline can leverage other tasks' work products such as artifacts an Pipelines are controlled by a *Controller Task* that holds the logic of the pipeline execution steps. -## How do pipelines work? +## How Do Pipelines Work? Before running a pipeline, we need to configure a Controller Task, in which the pipeline is defined. Pipelines are made up of steps. Each step consists of a task that already exists in the ClearML Server and is used as a template. The @@ -25,7 +25,7 @@ create customized, step-specific callbacks. ![Pipeline chart](../img/fundamentals_pipeline.png) -## Simple DAG pipelines +## Simple DAG Pipelines For a simple, DAG based logic, use the off-the-shelf [`PipelineController`](../references/sdk/automation_controller_pipelinecontroller.md) class to define the DAG (see an example [here](../guides/pipeline/pipeline_controller)). Once the `PipelineController` object is populated and configured, @@ -44,7 +44,7 @@ method. Alternatively, step-specific callback functions can be specified with th `post_execute_callback` parameters of the [`add_step`](../references/sdk/automation_controller_pipelinecontroller.md#add_step) method. -## Advanced pipelines +## Advanced Pipelines Since a pipeline *Controller Task* is itself a ClearML Task, it can be used as a pipeline step and can be used to create more complicated workflows, such as pipelines running other pipelines, or a pipeline running multiple tasks concurrently. @@ -56,7 +56,7 @@ It could also be useful to run a pipeline that runs tasks concurrently, training values simultaneously. See the [Tabular training pipeline](../guides/frameworks/pytorch/notebooks/table/tabular_training_pipeline.md) example of a pipeline with concurrent steps. -## Custom pipelines +## Custom Pipelines In cases where a DAG is insufficient (for example, when needing to launch one pipeline, then, if performance is inadequate, rerun pipeline again), users can apply custom logic, using generic methods to enqueue tasks, implemented in python code. diff --git a/docs/guides/frameworks/keras/jupyter.md b/docs/guides/frameworks/keras/jupyter.md index 88d559b5..e38f1719 100644 --- a/docs/guides/frameworks/keras/jupyter.md +++ b/docs/guides/frameworks/keras/jupyter.md @@ -31,7 +31,7 @@ They appear in **RESULTS** **>** **PLOTS**. ![image](../../../img/examples_keras_jupyter_03b.png) -## Debug samples +## Debug Samples The example calls Matplotlib methods to log debug sample images. They appear in **RESULTS** **>** **DEBUG SAMPLES**. @@ -63,7 +63,7 @@ The TensorFlow Definitions appear in the **TF_DEFINE** subsection. ![image](../../../img/examples_keras_jupyter_21.png) -## CONSOLE +## Console Text printed to the console for training appears in **RESULTS** **>** **CONSOLE**. diff --git a/docs/guides/frameworks/keras/keras_tensorboard.md b/docs/guides/frameworks/keras/keras_tensorboard.md index aa896967..4fee7c10 100644 --- a/docs/guides/frameworks/keras/keras_tensorboard.md +++ b/docs/guides/frameworks/keras/keras_tensorboard.md @@ -46,7 +46,7 @@ Text printed to the console for training progress, as well as all other console ![image](../../../img/examples_keras_03.png) -## Configuration objects +## Configuration Objects In the experiment code, a configuration dictionary is connected to the Task by calling the [Task.connect](../../../references/sdk/task.md#connect) method. diff --git a/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md b/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md index bf1c178b..44d84445 100644 --- a/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md +++ b/docs/guides/frameworks/pytorch/tensorboard_toy_pytorch.md @@ -7,7 +7,7 @@ example demonstrates the integration of **ClearML** into code, which creates a T debug sample images. When the script runs, it creates an experiment named `pytorch tensorboard toy example`, which is associated with the `examples` project. -## Debug samples +## Debug Samples The debug sample images appear according to metric, in the experiment page in the **ClearML web UI** under **RESULTS** **>** **DEBUG SAMPLES**. diff --git a/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md b/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md index 3ba93627..59559787 100644 --- a/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md +++ b/docs/guides/frameworks/pytorch_lightning/pytorch_lightning_example.md @@ -1,5 +1,5 @@ --- -title: Pytorch Lightning +title: PyTorch Lightning --- The [pytorch-lightning](https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch-lightning/pytorch_lightning_example.py) diff --git a/docs/hyperdatasets/annotations.md b/docs/hyperdatasets/annotations.md index 21e96b65..8fd34056 100644 --- a/docs/hyperdatasets/annotations.md +++ b/docs/hyperdatasets/annotations.md @@ -10,7 +10,7 @@ Annotation Tasks can be used to efficiently organize the annotation of frames in For information about how to view, create, and manage annotations using the WebApp, see [Annotating Images and Videos](webapp/webapp_annotator.md#annotating-images-and-video). -## Frame objects +## Frame Objects Frame objects are labeled Regions of Interest (ROIs), which can be bounded by polygons (including rectangles), ellipses, or key points. These ROIs are useful for object detection, classification, or semantic segmentation. @@ -18,14 +18,14 @@ or key points. These ROIs are useful for object detection, classification, or se Frame objects can include ROI labels, confidence levels, and masks for semantic segmentation. In ClearML Enterprise, one or more labels and sources dictionaries can be associated with an ROI (although multiple source ROIs are not frequently used). -## Frame labels +## Frame Labels Frame labels are applied to an entire frame, not a region in a frame. ## Usage -### Adding a frame object +### Adding a Frame Object To add a frame object annotation to a SingleFrame, use the `SingleFrame.add_annotation` method. @@ -43,7 +43,7 @@ When adding an annotation there are a few options for entering the annotation's * `ellipse2d_xyrrt` - A List consisting of cx, cy, rx, ry, and theta for an ellipse * And more! See `SingleFrame.add_annotation` for further options. -### Adding a Frame label +### Adding a Frame Label Adding a frame label is similar to creating a frame objects, except that coordinates don't need to be specified, since the whole frame is being referenced. diff --git a/docs/hyperdatasets/dataset.md b/docs/hyperdatasets/dataset.md index 5e37938f..d678a535 100644 --- a/docs/hyperdatasets/dataset.md +++ b/docs/hyperdatasets/dataset.md @@ -17,7 +17,7 @@ metadata and data. These parent-child version relationships can be represented as version trees with a root-level parent. A Dataset can contain one or more trees. -## Dataset version state +## Dataset Version State Dataset versions can have either **Draft** or **Published** status. diff --git a/docs/hyperdatasets/single_frames.md b/docs/hyperdatasets/single_frames.md index 915ea169..53b8956e 100644 --- a/docs/hyperdatasets/single_frames.md +++ b/docs/hyperdatasets/single_frames.md @@ -52,7 +52,7 @@ For more information, see [Previews](previews.md). For more information, see [Custom Metadata](custom_metadata.md). -## Frame structure +## Frame Structure The panel below describes the details contained within a `frame`: @@ -294,7 +294,7 @@ myDatasetVersion.update_frames(frames) ``` -### Deleting frames +### Deleting Frames To delete a SingleFrame, use the `DatasetVersion.delete_frames` method. diff --git a/docs/webapp/webapp_exp_reproducing.md b/docs/webapp/webapp_exp_reproducing.md index 049b0210..c5f68fb2 100644 --- a/docs/webapp/webapp_exp_reproducing.md +++ b/docs/webapp/webapp_exp_reproducing.md @@ -1,5 +1,5 @@ --- -title: Reproducing experiments +title: Reproducing Experiments --- Reproduce experiments on local or remote machines, in one of the following ways: diff --git a/docs/webapp/webapp_exp_table.md b/docs/webapp/webapp_exp_table.md index 1772b17c..1ef9fe4c 100644 --- a/docs/webapp/webapp_exp_table.md +++ b/docs/webapp/webapp_exp_table.md @@ -14,7 +14,7 @@ in the active experiments and models tables. See [Archiving](webapp_archiving). ![Experiment table](../img/webapp_exp_table_01.png) -## Experiments table columns +## Experiments Table Columns The experiments table default and customizable columns are described in the following table. @@ -36,7 +36,7 @@ The experiments table default and customizable columns are described in the foll -## Customizing the experiments table +## Customizing the Experiments Table The experiments table can be customized by: * Showing / hiding default columns @@ -69,24 +69,24 @@ all the experiments in the project. The customizations of these two views are sa -### Adding metrics and / or hyperparameters +### Adding Metrics and / or Hyperparameters ![Experiment table customization gif](../img/gif/webapp_exp_table_cust.gif) Add metrics and / or hyperparameters columns to the experiments table. The metrics and hyperparameters depend upon the experiments in the table. -#### To add metrics: +#### To Add Metrics: * Click Setting Gear **>** **+ METRICS** **>** Expand a metric **>** Select the **LAST** (value), **MIN** (minimal value), and / or **MAX** (maximal value) checkboxes. -#### To add hyperparameters: +#### To Add Hyperparameters: * Click Setting Gear **>** **+ HYPER PARAMETERS** **>** Expand a section **>** Select the hyperparameter checkboxes. -### Filtering experiments +### Filtering Experiments ![Filtering table gif](../img/gif/filter_screenshots.gif) @@ -106,7 +106,7 @@ Once a filter is applied to a column, its filter icon will appear with a highlig -### Using other customization features +### Using Other Customization Features **To use other customization features:** @@ -116,7 +116,7 @@ Once a filter is applied to a column, its filter icon will appear with a highlig * Column resizing - In the column heading, drag to a new size. * Column autofit - In the column heading, double click a column separator. -## Experiment actions +## Experiment Actions The following table describes the actions that can be done from the experiments table, including the [states](../fundamentals/task.md#task-states-and-state-transitions) that allow each operation. @@ -151,7 +151,7 @@ appears when hovering over an action icon. ![Experiment table batch operations](../img/webapp_exp_table_batch_operations.png) -## Creating an experiment leaderboard +## Creating an Experiment Leaderboard Filter & sort the experiments of any project to create a leaderboard that can be shared and stored. This leaderboard updates in real time with experiment performance and outputs. diff --git a/docs/webapp/webapp_model_modifying.md b/docs/webapp/webapp_model_modifying.md index 079d9a81..21e7d98f 100644 --- a/docs/webapp/webapp_model_modifying.md +++ b/docs/webapp/webapp_model_modifying.md @@ -5,7 +5,7 @@ title: Modifying Models In the models table, modify models that have a status of *Draft* (status *Published* is read-only). Modify the model configuration and label enumeration. -## Model configuration +## Model Configuration **To edit the model configuration:** @@ -14,7 +14,7 @@ configuration and label enumeration. ![image](../img/webapp_models_05a.png) -### Label enumeration +### Label Enumeration For each class, label enumeration contains the class name (key) and value. diff --git a/docs/webapp/webapp_model_viewing.md b/docs/webapp/webapp_model_viewing.md index 256f72d4..ef1bc5a3 100644 --- a/docs/webapp/webapp_model_viewing.md +++ b/docs/webapp/webapp_model_viewing.md @@ -4,7 +4,7 @@ title: Viewing Model Details In the models table, click on a model to view its general information, configuration, and label enumeration. -## General model information +## General Model Information General information includes: * Model URL @@ -16,11 +16,11 @@ If a model is a local file, it is downloadable. If a model is in another type of ![image](../img/webapp_models_02.png) -## Model configuration +## Model Configuration ![image](../img/webapp_models_05.png) -## Label enumeration +## Label Enumeration For each class, label enumeration contains the class name (key) and value. diff --git a/docs/webapp/webapp_profile.md b/docs/webapp/webapp_profile.md index d312c6d4..010ac3fd 100644 --- a/docs/webapp/webapp_profile.md +++ b/docs/webapp/webapp_profile.md @@ -10,7 +10,7 @@ Use the Profile page to manage a **ClearML** user account, including: * [Switch workspaces](#switching-workspaces) - If using multiple workspaces (are a member of more than one **ClearML Hosted Service** team), switch workspaces. * [Invite new teammates](#inviting-new-teammates) - Collaborate with new users by inviting them to a **ClearML Hosted Service** workspace. -## Setting user preferences +## Setting User Preferences The **HiDPI browser scale override** adjusts scaling on High-DPI monitors to improve the Web UI experience. It is enabled by default, but can be disabled. @@ -29,7 +29,7 @@ Users that use their own **ClearML Server** can choose whether to send anonymous * **Region** - The region for AWS S3. * **Host (Endpoint)** - The host for non-AWS S3 servers. -## Creating ClearML credentials +## Creating ClearML Credentials **ClearML** credentials include: * Access key @@ -53,7 +53,7 @@ switch to it. 1. In **App Credentials** **>** **+ Create new credentials**. -## Switching workspaces +## Switching Workspaces :::note Switching workspaces does not apply to users of a self-hosted **ClearML Server** @@ -67,7 +67,7 @@ Switching workspaces does not apply to users of a self-hosted **ClearML Server** Click the workspace to switch to. * Profile page - In the **WORKSPACES** section, click **SWITCH TO WORKSPACE** **>** Click the workspace to switch to. -## Inviting new teammates +## Inviting New Teammates :::note Inviting new teammates does not apply to users of a self-hosted **ClearML Server**. @@ -88,7 +88,7 @@ section shows the current members of the team, and whether the team has reached 1. Send the invitation hyperlink to an invitee. -## Leaving a workspace +## Leaving a Workspace A member of a workspace can leave the workspace and no longer be a member of that team. diff --git a/docs/webapp/webapp_workers_queues.md b/docs/webapp/webapp_workers_queues.md index a362f4bb..4714272b 100644 --- a/docs/webapp/webapp_workers_queues.md +++ b/docs/webapp/webapp_workers_queues.md @@ -12,7 +12,7 @@ With the **Workers and Queues** page, users can: -## Resources utilization +## Resources Utilization **To monitor resource utilization:** @@ -34,7 +34,7 @@ With the **Workers and Queues** page, users can: -## Worker utilization +## Worker Utilization Optimize worker use by monitoring worker utilization in the **Workers** tab. @@ -45,7 +45,7 @@ Optimize worker use by monitoring worker utilization in the **Workers** tab. -## Queue utilization +## Queue Utilization **To monitor all queues:** @@ -64,7 +64,7 @@ Optimize worker use by monitoring worker utilization in the **Workers** tab. -## Queue management +## Queue Management In the **Queues** tab, do any of the following: