Merge branch 'main' of https://github.com/allegroai/clearml-docs into edits_2
@ -232,7 +232,7 @@ ranging from 2 GB to 12 GB (see [clearml-fractional-gpu repository](https://gith
|
||||
|
||||
This example runs the ClearML Ubuntu 22 with CUDA 12.3 container on GPU 0, which is limited to use up to 8GB of its memory.
|
||||
:::note
|
||||
--pid=host is required to allow the driver to differentiate between the container's processes and other host processes when limiting memory usage
|
||||
`--pid=host` is required to allow the driver to differentiate between the container's processes and other host processes when limiting memory usage
|
||||
:::
|
||||
1. Run the following command inside the container to verify that the fractional gpu memory limit is working correctly:
|
||||
```bash
|
||||
|
@ -212,7 +212,7 @@ Example:
|
||||
ClearML serving instances send serving statistics (count/latency) automatically to Prometheus and Grafana can be used
|
||||
to visualize and create live dashboards.
|
||||
|
||||
The default docker-compose installation is preconfigured with Prometheus and Grafana. Notice that by default data/ate
|
||||
The default `docker-compose` installation is preconfigured with Prometheus and Grafana. Notice that by default data/ate
|
||||
of both containers is *not* persistent. To add persistence, adding a volume mount is recommended.
|
||||
|
||||
You can also add many custom metrics on the input/predictions of your models. Once a model endpoint is registered,
|
||||
|
@ -22,7 +22,7 @@ The values in the ClearML configuration file can be overridden by environment va
|
||||
and command-line arguments.
|
||||
:::
|
||||
|
||||
# Editing Your Configuration File
|
||||
## Editing Your Configuration File
|
||||
|
||||
To add, change, or delete options, edit your configuration file.
|
||||
|
||||
@ -1548,7 +1548,7 @@ environment {
|
||||
}
|
||||
```
|
||||
|
||||
### files section
|
||||
### files section
|
||||
|
||||
**`files`** (*dict*)
|
||||
|
||||
|
@ -5,8 +5,8 @@ title: Linux and macOS
|
||||
Deploy the ClearML Server in Linux or macOS using the pre-built Docker image.
|
||||
|
||||
For ClearML docker images, including previous versions, see [https://hub.docker.com/r/allegroai/clearml](https://hub.docker.com/r/allegroai/clearml).
|
||||
However, pulling the ClearML Docker image directly is not required. ClearML provides a docker-compose YAML file that does this.
|
||||
The docker-compose file is included in the instructions on this page.
|
||||
However, pulling the ClearML Docker image directly is not required. ClearML provides a `docker-compose` YAML file that does this.
|
||||
The `docker-compose` file is included in the instructions on this page.
|
||||
|
||||
For information about upgrading ClearML Server in Linux or macOS, see [here](upgrade_server_linux_mac.md).
|
||||
|
||||
@ -134,7 +134,7 @@ Deploying the server requires a minimum of 8 GB of memory, 16 GB is recommended.
|
||||
sudo chown -R $(whoami):staff /opt/clearml
|
||||
```
|
||||
|
||||
2. Download the ClearML Server docker-compose YAML file.
|
||||
2. Download the ClearML Server `docker-compose` YAML file:
|
||||
```
|
||||
sudo curl https://raw.githubusercontent.com/clearml/clearml-server/master/docker/docker-compose.yml -o /opt/clearml/docker-compose.yml
|
||||
```
|
||||
|
@ -54,7 +54,7 @@ Deploying the server requires a minimum of 8 GB of memory, 16 GB is recommended.
|
||||
mkdir c:\opt\clearml\logs
|
||||
```
|
||||
|
||||
1. Save the ClearML Server docker-compose YAML file.
|
||||
1. Save the ClearML Server `docker-compose` YAML file.
|
||||
|
||||
```
|
||||
curl https://raw.githubusercontent.com/clearml/clearml-server/master/docker/docker-compose-win10.yml -o c:\opt\clearml\docker-compose-win10.yml
|
||||
|
@ -13,7 +13,7 @@ without any coding. Applications are installed on top of the ClearML Server.
|
||||
To run application you will need the following:
|
||||
* RAM: Make sure you have at least 400 MB of RAM per application instance.
|
||||
* Applications Service: Make sure that the applications agent service is up and running on your server:
|
||||
* If you are using a docker-compose solution, make sure that the clearml-apps-agent service is running.
|
||||
* If you are using a `docker-compose` solution, make sure that the clearml-apps-agent service is running.
|
||||
* If you are using a Kubernetes cluster, check for the clearml-clearml-enterprise-apps component.
|
||||
* Installation Files: Each application has its installation zip file. Make sure you have the relevant files for the
|
||||
applications you wish to install.
|
||||
|
@ -13,11 +13,11 @@ The Application Gateway is available under the ClearML Enterprise plan.
|
||||
* Credentials for the ClearML/allegroai docker repository
|
||||
* A valid ClearML Server installation
|
||||
|
||||
## Host configurations
|
||||
## Host Configurations
|
||||
|
||||
### Docker installation
|
||||
### Docker Installation
|
||||
|
||||
Installing docker and docker-compose might vary depending on the specific operating system you’re using. Here is an example for AmazonLinux:
|
||||
Installing `docker` and `docker-compose` might vary depending on the specific operating system you’re using. Here is an example for AmazonLinux:
|
||||
|
||||
```
|
||||
sudo dnf -y install docker
|
||||
@ -33,9 +33,9 @@ sudo docker login
|
||||
|
||||
Use the ClearML/allegroai dockerhub credentials when prompted by docker login.
|
||||
|
||||
### Docker-compose file
|
||||
### Docker-compose File
|
||||
|
||||
This is an example of the docker-compose file you will need:
|
||||
This is an example of the `docker-compose` file you will need:
|
||||
|
||||
```
|
||||
version: '3.5'
|
||||
@ -103,17 +103,17 @@ Edit it according to the following guidelines:
|
||||
* `CLEARML_API_ACCESS_KEY`: ClearML server api key
|
||||
* `CLEARML_API_SECRET_KEY`: ClearML server secret key
|
||||
* `ROUTER_URL`: URL for this router that was previously configured in the load balancer starting with `https://`
|
||||
* `ROUTER_NAME`: unique name for this router
|
||||
* `AUTH_ENABLED`: enable or disable http calls authentication when the router is communicating with the ClearML server
|
||||
* `SSL_VERIFY`: enable or disable SSL certificate validation when the router is communicating with the ClearML server
|
||||
* `AUTH_COOKIE_NAME`: the cookie name used by the ClearML server to store the ClearML authentication cookie. This can usually be found in the `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in the ClearML server installation (`/opt/allegro/config/envoy/envoy.yaml`) (see below)
|
||||
* `AUTH_SECURE_ENABLED`: enable the Set-Cookie `secure` parameter
|
||||
* `AUTH_BASE64_JWKS_KEY`: value form `k` key in the `jwks.json` file in the ClearML server installation
|
||||
* `LISTEN_QUEUE_NAME`: (optional) name of queue to check for tasks (if none, every task is checked)
|
||||
* `EXTRA_BASH_COMMAND`: command to be launched before starting the router
|
||||
* `TCP_ROUTER_ADDRESS`: router external address, can be an IP or the host machine or a load balancer hostname, depends on network configuration
|
||||
* `TCP_PORT_START`: start port for the TCP Session feature
|
||||
* `TCP_PORT_END`: end port port for the TCP Session feature
|
||||
* `ROUTER_NAME`: Unique name for this router
|
||||
* `AUTH_ENABLED`: Enable or disable http calls authentication when the router is communicating with the ClearML server
|
||||
* `SSL_VERIFY`: Enable or disable SSL certificate validation when the router is communicating with the ClearML server
|
||||
* `AUTH_COOKIE_NAME`: Cookie name used by the ClearML server to store the ClearML authentication cookie. This can usually be found in the `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in the ClearML server installation (`/opt/allegro/config/envoy/envoy.yaml`) (see below)
|
||||
* `AUTH_SECURE_ENABLED`: Enable the Set-Cookie `secure` parameter
|
||||
* `AUTH_BASE64_JWKS_KEY`: Value form `k` key in the `jwks.json` file in the ClearML server installation
|
||||
* `LISTEN_QUEUE_NAME`: (*optional*) Name of queue to check for tasks (if none, every task is checked)
|
||||
* `EXTRA_BASH_COMMAND`: Command to be launched before starting the router
|
||||
* `TCP_ROUTER_ADDRESS`: Router external address, can be an IP or the host machine or a load balancer hostname, depends on network configuration
|
||||
* `TCP_PORT_START`: Start port for the TCP Session feature
|
||||
* `TCP_PORT_END`: End port for the TCP Session feature
|
||||
|
||||
Run the following command to start the router:
|
||||
|
||||
@ -121,11 +121,11 @@ Run the following command to start the router:
|
||||
sudo docker compose --env-file runtime.env up -d
|
||||
```
|
||||
|
||||
:::Note How to find my jwkskey
|
||||
:::note How to find my jwkskey
|
||||
|
||||
The *JSON Web Key Set* (*JWKS*) is a set of keys containing the public keys used to verify any JSON Web Token (JWT).
|
||||
|
||||
In a docker-compose server installation, this can be found in the `CLEARML__secure__auth__token_secret` env var in the apiserver server component.
|
||||
In a `docker-compose` server installation, this can be found in the `CLEARML__secure__auth__token_secret` env var in the apiserver server component.
|
||||
|
||||
:::
|
||||
|
||||
|
@ -32,9 +32,9 @@ https://raw.githubusercontent.com/clearml/clearml-enterprise-helm-charts/gh-page
|
||||
--password <GITHUB_TOKEN>
|
||||
```
|
||||
|
||||
### Prepare values
|
||||
### Prepare Values
|
||||
|
||||
Before installing the TTR create an helm-override files named `task-traffic-router.values-override.yaml`:
|
||||
Before installing the TTR, create a `helm-override` files named `task-traffic-router.values-override.yaml`:
|
||||
|
||||
```
|
||||
imageCredentials:
|
||||
@ -55,20 +55,20 @@ tcpSession:
|
||||
end:
|
||||
```
|
||||
|
||||
Edit it accordingly to this guidelines:
|
||||
Edit it accordingly to these guidelines:
|
||||
|
||||
* `clearml.apiServerUrlReference`: url usually starting with `https://api.`
|
||||
* `clearml.apiServerUrlReference`: URL usually starting with `https://api.`
|
||||
* `clearml.apiServerKey`: ClearML server api key
|
||||
* `clearml.apiServerSecret`: ClearML server secret key
|
||||
* `ingress.hostName`: url of router we configured previously for loadbalancer starting with `https://`
|
||||
* `clearml.sslVerify`: enable or disable SSL certificate validation on apiserver calls check
|
||||
* `clearml.authCookieName`: value from `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in ClearML server installation.
|
||||
* `clearml.jwksKey`: value form `k` key in `jwks.json` file in ClearML server installation (see below)
|
||||
* `tcpSession.routerAddress`: router external address can be an IP or the host machine or a loadbalancer hostname, depends on the network configuration
|
||||
* `tcpSession.portRange.start`: start port for the TCP Session feature
|
||||
* `tcpSession.portRange.end`: end port port for the TCP Session feature
|
||||
* `ingress.hostName`: URL of router we configured previously for load balancer starting with `https://`
|
||||
* `clearml.sslVerify`: Enable or disable SSL certificate validation on apiserver calls check
|
||||
* `clearml.authCookieName`: Value from `value_prefix` key starting with `allegro_token` in `envoy.yaml` file in ClearML server installation.
|
||||
* `clearml.jwksKey`: Value form `k` key in `jwks.json` file in ClearML server installation (see below)
|
||||
* `tcpSession.routerAddress`: Router external address can be an IP or the host machine or a load balancer hostname, depends on the network configuration
|
||||
* `tcpSession.portRange.start`: Start port for the TCP Session feature
|
||||
* `tcpSession.portRange.end`: End port for the TCP Session feature
|
||||
|
||||
::: How to find my jwkskey
|
||||
:::note How to find my jwkskey
|
||||
|
||||
The *JSON Web Key Set* (*JWKS*) is a set of keys containing the public keys used to verify any JSON Web Token (JWT).
|
||||
|
||||
|
@ -36,7 +36,7 @@ them before exporting.
|
||||
Execute the data tool within the `apiserver` container.
|
||||
|
||||
Open a bash session inside the `apiserver` container of the server:
|
||||
* In docker-compose:
|
||||
* In `docker-compose`:
|
||||
|
||||
```commandline
|
||||
sudo docker exec -it clearml-apiserver /bin/bash
|
||||
|
@ -100,9 +100,10 @@ Install the ClearML chart with the required configuration:
|
||||
1. Prepare the `overrides.yaml` file and input the following content. Make sure to replace `<BASE_DOMAIN>` and `<SSO_*>`
|
||||
with a valid domain that will have records pointing to the ingress controller accordingly.
|
||||
The credentials specified in `<SUPERVISOR_USER_KEY>` and `<SUPERVISOR_USER_SECRET>` can be used to log in as the
|
||||
supervisor user in the web UI.
|
||||
supervisor user in the web UI.
|
||||
|
||||
Note that the `<SUPERVISOR_USER_EMAIL>` value must be explicitly quoted. To do so, put `\\"` around the quoted value.
|
||||
For example `"\\"email@example.com\\””`
|
||||
For example `"\\"email@example.com\\””`.
|
||||
|
||||
```
|
||||
imageCredentials:
|
||||
@ -192,7 +193,7 @@ Install the ClearML chart with the required configuration:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
2. Install ClearML
|
||||
2. Install ClearML:
|
||||
|
||||
```
|
||||
helm install -n clearml \\
|
||||
@ -305,9 +306,9 @@ spec:
|
||||
kubernetes.io/metadata.name: clearml
|
||||
```
|
||||
|
||||
## Applications Installation
|
||||
## Application Installation
|
||||
|
||||
To install ClearML GUI applications, follow these steps:
|
||||
To install ClearML GUI applications:
|
||||
|
||||
1. Get the apps to install and the installation script by downloading and extracting the archive provided by ClearML
|
||||
|
||||
@ -491,7 +492,7 @@ To install the ClearML Agent Chart, follow these steps:
|
||||
-d '{"name":"default"}'
|
||||
```
|
||||
|
||||
### Tenant Namespace isolation with NetworkPolicies
|
||||
### Tenant Namespace Isolation with NetworkPolicies
|
||||
|
||||
To ensure network isolation for each tenant, you need to create a `NetworkPolicy` in the tenant namespace. This way
|
||||
the entire namespace/tenant will not accept any connection from other namespaces.
|
||||
|
@ -43,7 +43,7 @@ should be reviewed and modified prior to the server installation
|
||||
## Installing ClearML Server
|
||||
### Preliminary Steps
|
||||
|
||||
1. Install Docker CE
|
||||
1. Install Docker CE:
|
||||
|
||||
```
|
||||
https://docs.docker.com/install/linux/docker-ce/ubuntu/
|
||||
@ -113,10 +113,10 @@ should be reviewed and modified prior to the server installation
|
||||
sudo systemctl enable disable-thp
|
||||
```
|
||||
|
||||
1. Restart the machine
|
||||
1. Restart the machine.
|
||||
|
||||
### Installing the Server
|
||||
1. Remove any previous installation of ClearML Server
|
||||
1. Remove any previous installation of ClearML Server:
|
||||
|
||||
```
|
||||
sudo rm -R /opt/clearml/
|
||||
@ -141,7 +141,7 @@ should be reviewed and modified prior to the server installation
|
||||
sudo mkdir -pv /opt/allegro/config/onprem_poc
|
||||
```
|
||||
|
||||
1. Copy the following ClearML configuration files to `/opt/allegro`
|
||||
1. Copy the following ClearML configuration files to `/opt/allegro`:
|
||||
* `constants.env`
|
||||
* `docker-compose.override.yml`
|
||||
* `docker-compose.yml`
|
||||
@ -165,10 +165,13 @@ should be reviewed and modified prior to the server installation
|
||||
sudo docker login -u=$DOCKERHUB_USER -p=$DOCKERHUB_PASSWORD
|
||||
```
|
||||
|
||||
1. Start the `docker-compose` by changing directories to the directory containing the docker-compose files and running the following command:
|
||||
sudo docker-compose --env-file constants.env up -d
|
||||
|
||||
1. Verify web access by browsing to your URL (IP address) and port 8080.
|
||||
1. Start the `docker-compose` by changing directories to the directory containing the `docker-compose` files and running the following command:
|
||||
|
||||
```
|
||||
sudo docker-compose --env-file constants.env up -d
|
||||
```
|
||||
|
||||
1. Verify web access by browsing to your URL (IP address) and port 8080:
|
||||
|
||||
```
|
||||
http://<server_ip_here>:8080
|
||||
@ -191,7 +194,10 @@ the following subdomains should be forwarded to the corresponding ports on the s
|
||||
* `https://app.<domain>` should be forwarded to port 8080
|
||||
* `https://files.<domain>` should be forwarded to port 8081
|
||||
|
||||
|
||||
:::warning
|
||||
**Critical: Ensure no other ports are open to maintain the highest level of security.**
|
||||
:::
|
||||
|
||||
Additionally, ensure that the following URLs are correctly configured in the server's environment file:
|
||||
|
||||
|
@ -8,7 +8,7 @@ It covers the following:
|
||||
* Set up security groups and IAM role
|
||||
* Create EC2 instance with required disks
|
||||
* Install dependencies and mount disks
|
||||
* Deploy ClearML version using docker-compose
|
||||
* Deploy ClearML version using `docker-compose`
|
||||
* Set up load balancer and DNS
|
||||
* Set up server backup
|
||||
|
||||
@ -117,10 +117,10 @@ Instance requirements:
|
||||
## Load Balancer
|
||||
|
||||
1. Create a TLS certificate:
|
||||
1. Choose a domain name to be used with the server. The main URL that will be used by the system’s users will be app.\<domain\>
|
||||
1. Choose a domain name to be used with the server. The main URL that will be used by the system’s users will be `app.<domain>`
|
||||
2. Create a certificate, with the following DNS names:
|
||||
1. \<domain name\>
|
||||
2. \*.\<domain name\>
|
||||
1. `<domain name>`
|
||||
2. `*.<domain name>`
|
||||
|
||||
2. Create the `envoy` target group for the server:
|
||||
1. Port: 10000
|
||||
@ -284,7 +284,7 @@ log would usually indicate the reason for the failure.
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Removing app containers
|
||||
### Removing App Containers
|
||||
|
||||
To remove old application containers, add the following to the cron:
|
||||
|
||||
|
@ -31,7 +31,7 @@ The pip package also includes `clearml-data`. It can help you keep track of your
|
||||
|
||||
Both the 2 magic lines and the data tool will send all of their information to a ClearML server. This server then keeps an overview of your experiment runs and data sets over time, so you can always go back to a previous experiment, see how it was created and even recreate it exactly. Keep track of your best models by creating leaderboards based on your own metrics, and you can even directly compare multiple experiment runs, helping you to figure out the best way forward for your models.
|
||||
|
||||
To get started with a server right away, you can make use of the free tier. And when your needs grow, we've got you covered too! Just check out our website to find a tier that fits your organisation best. But, because we're open source, you can also host your own completely for free. We have AWS images, Google Cloud images, you can run it on docker-compose locally or even, if you really hate yourself, run it on a self-hosted kubernetes cluster using our helm charts.
|
||||
To get started with a server right away, you can make use of the free tier. And when your needs grow, we've got you covered too! Just check out our website to find a tier that fits your organisation best. But, because we're open source, you can also host your own completely for free. We have AWS images, Google Cloud images, you can run it on `docker-compose` locally or even, if you really hate yourself, run it on a self-hosted kubernetes cluster using our helm charts.
|
||||
|
||||
So, to recap: to get started, all you need is a pip package and a server to store everything. Easy right? But MLOps is much more than experiment and data management. It's also about automation and orchestration, which is exactly where the `clearml-agent` comes into play.
|
||||
|
||||
|
@ -18,22 +18,26 @@ The example does the following:
|
||||
|
||||
The loss and accuracy metric scalar plots appear in **SCALARS**, along with the resource utilization plots, which are titled **:monitor: machine**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Plots
|
||||
|
||||
The example calls Matplotlib methods to create several sample plots, and TensorBoard methods to plot histograms for layer density.
|
||||
They appear in **PLOTS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Debug Samples
|
||||
|
||||
The example calls Matplotlib methods to log debug sample images. They appear in **DEBUG SAMPLES**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Hyperparameters
|
||||
|
||||
@ -55,17 +59,20 @@ task_params['hidden_dim'] = 512
|
||||
|
||||
Parameter dictionaries appear in **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **General**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
The TensorFlow Definitions appear in the **TF_DEFINE** subsection.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Console
|
||||
|
||||
Text printed to the console for training appears in **CONSOLE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Artifacts
|
||||
|
||||
@ -74,9 +81,11 @@ created using Keras.
|
||||
|
||||
The task info panel shows model tracking, including the model name and design in **ARTIFACTS** **>** **Output Model**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
Clicking on the model name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can view
|
||||
the model's details and access the model.
|
||||
|
||||

|
||||

|
||||

|
@ -25,31 +25,36 @@ The example script does the following:
|
||||
The loss and accuracy metric scalar plots appear in **SCALARS**, along with the resource utilization plots,
|
||||
which are titled **:monitor: machine**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Histograms
|
||||
|
||||
Histograms for layer density appear in **PLOTS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Hyperparameters
|
||||
|
||||
ClearML automatically logs command line options generated with `argparse`, and TensorFlow Definitions.
|
||||
ClearML automatically logs command line options generated with `argparse` and TensorFlow Definitions.
|
||||
|
||||
Command line options appear in **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **Args**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
TensorFlow Definitions appear in **TF_DEFINE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Console
|
||||
|
||||
Text printed to the console for training progress, as well as all other console output, appear in **CONSOLE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Configuration Objects
|
||||
|
||||
@ -64,4 +69,5 @@ task.connect_configuration(
|
||||
|
||||
It appears in **CONFIGURATION** **>** **CONFIGURATION OBJECTS** **>** **MyConfig**.
|
||||
|
||||

|
||||

|
||||

|
@ -12,16 +12,19 @@ and `matplotlib` to create a scatter diagram. When the script runs, it creates a
|
||||
ClearML automatically logs the scatter plot, which appears in the [task's page](../../../webapp/webapp_exp_track_visual.md)
|
||||
in the ClearML web UI, under **PLOTS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Artifacts
|
||||
|
||||
Models created by the task appear in the task's **ARTIFACTS** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
Clicking on the model name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can
|
||||
view the model's details and access the model.
|
||||
|
||||
|
||||

|
||||

|
||||

|
@ -16,30 +16,35 @@ The script does the following:
|
||||
The loss and accuracy metric scalar plots appear in the task's page in the **ClearML web UI**, under
|
||||
**SCALARS**. The also includes resource utilization plots, which are titled **:monitor: machine**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Hyperparameters
|
||||
|
||||
ClearML automatically logs command line options defined with `argparse`. They appear in **CONFIGURATION** **>**
|
||||
**HYPERPARAMETERS** **>** **Args**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Console
|
||||
|
||||
Text printed to the console for training progress, as well as all other console output, appear in **CONSOLE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Artifacts
|
||||
|
||||
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
|
||||
models and any snapshots created using PyTorch.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
Clicking on the model's name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can
|
||||
view the model's details and access the model.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
|
@ -14,5 +14,6 @@ the `examples` project.
|
||||
ClearML automatically captures the video data that is added to the `SummaryWriter` object, using the `add_video` method.
|
||||
The video appears in the task's **DEBUG SAMPLES** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
|
@ -44,28 +44,33 @@ When the script runs, it logs:
|
||||
ClearML logs the scalars from training each network. They appear in the task's page in the **ClearML web UI**, under
|
||||
**SCALARS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Summary of Hyperparameter Optimization
|
||||
|
||||
ClearML automatically logs the parameters of each task run in the hyperparameter search. They appear in tabular
|
||||
form in **PLOTS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Artifacts
|
||||
|
||||
ClearML automatically stores the output model. It appears in **ARTIFACTS** **>** **Output Model**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
Model details, such as snap locations, appear in the **MODELS** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
The model configuration is stored with the model.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Configuration Objects
|
||||
|
||||
@ -73,12 +78,14 @@ The model configuration is stored with the model.
|
||||
|
||||
ClearML automatically logs the TensorFlow Definitions, which appear in **CONFIGURATION** **>** **HYPERPARAMETERS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
### Configuration
|
||||
|
||||
The Task configuration appears in **CONFIGURATION** **>** **General**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
|
||||
|
@ -20,24 +20,29 @@ In the **ClearML Web UI**, the PR Curve summaries appear in the task's page unde
|
||||
|
||||
* Blue PR curves
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
* Green PR curves
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
* Red PR curves
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Hyperparameters
|
||||
|
||||
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATION** **>** **HYPERPARAMETERS** **>** **TF_DEFINE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Console
|
||||
|
||||
All other console output appears in **CONSOLE**.
|
||||
All console output appears in **CONSOLE** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
@ -14,25 +14,29 @@ project.
|
||||
The `tf.summary.scalar` output appears in the ClearML web UI, in the task's
|
||||
**SCALARS**. Resource utilization plots, which are titled **:monitor: machine**, also appear in the **SCALARS** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Plots
|
||||
|
||||
The `tf.summary.histogram` output appears in **PLOTS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Debug Samples
|
||||
|
||||
ClearML automatically tracks images and text output to TensorFlow. They appear in **DEBUG SAMPLES**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Hyperparameters
|
||||
|
||||
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATION** **>** **HYPERPARAMETERS** **>**
|
||||
**TF_DEFINE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
|
||||
|
@ -13,30 +13,35 @@ When the script runs, it creates a task named `Tensorflow v2 mnist with summarie
|
||||
The loss and accuracy metric scalar plots appear in the task's page in the **ClearML web UI** under
|
||||
**SCALARS**. Resource utilization plots, which are titled **:monitor: machine**, also appear in the **SCALARS** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Hyperparameters
|
||||
|
||||
ClearML automatically logs TensorFlow Definitions. They appear in **CONFIGURATION** **>** **HYPERPARAMETERS**
|
||||
**>** **TF_DEFINE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Console
|
||||
|
||||
All console output appears in **CONSOLE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Artifacts
|
||||
|
||||
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
|
||||
models and any snapshots created using TensorFlow.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
Clicking on a model's name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can
|
||||
view the model's details and access the model.
|
||||
|
||||
|
||||

|
||||

|
||||

|
@ -13,7 +13,8 @@ the `examples` project.
|
||||
ClearML automatically captures scalars logged with XGBoost, which can be visualized in plots in the
|
||||
ClearML WebApp, in the task's **SCALARS** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Models
|
||||
|
||||
@ -21,14 +22,17 @@ ClearML automatically captures the model logged using the `xgboost.save` method,
|
||||
|
||||
View saved snapshots in the task's **ARTIFACTS** tab.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
To view the model details, click the model name in the **ARTIFACTS** page, which will open the model's info tab. Alternatively, download the model.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Console
|
||||
|
||||
All console output during the script's execution appears in the task's **CONSOLE** page.
|
||||
|
||||

|
||||

|
||||

|
@ -18,25 +18,30 @@ classification dataset using XGBoost
|
||||
The feature importance plot and tree plot appear in the task's page in the **ClearML web UI**, under
|
||||
**PLOTS**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
|
||||
## Console
|
||||
|
||||
All other console output appear in **CONSOLE**.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
## Artifacts
|
||||
|
||||
Models created by the task appear in the task's **ARTIFACTS** tab. ClearML automatically logs and tracks
|
||||
models and any snapshots created using XGBoost.
|
||||
|
||||

|
||||

|
||||

|
||||
|
||||
Clicking on the model's name takes you to the [model's page](../../../webapp/webapp_model_viewing.md), where you can
|
||||
view the model's details and access the model.
|
||||
|
||||

|
||||

|
||||

|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 39 KiB |
BIN
docs/img/examples_keras_00_dark.png
Normal file
After Width: | Height: | Size: 40 KiB |
Before Width: | Height: | Size: 86 KiB After Width: | Height: | Size: 76 KiB |
BIN
docs/img/examples_keras_00a_dark.png
Normal file
After Width: | Height: | Size: 78 KiB |
Before Width: | Height: | Size: 129 KiB After Width: | Height: | Size: 120 KiB |
BIN
docs/img/examples_keras_01_dark.png
Normal file
After Width: | Height: | Size: 126 KiB |
Before Width: | Height: | Size: 83 KiB After Width: | Height: | Size: 115 KiB |
BIN
docs/img/examples_keras_02_dark.png
Normal file
After Width: | Height: | Size: 119 KiB |
Before Width: | Height: | Size: 83 KiB After Width: | Height: | Size: 104 KiB |
BIN
docs/img/examples_keras_jupyter_03_dark.png
Normal file
After Width: | Height: | Size: 107 KiB |
Before Width: | Height: | Size: 90 KiB After Width: | Height: | Size: 99 KiB |
BIN
docs/img/examples_keras_jupyter_03a_dark.png
Normal file
After Width: | Height: | Size: 100 KiB |
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 43 KiB |
BIN
docs/img/examples_keras_jupyter_04_dark.png
Normal file
After Width: | Height: | Size: 43 KiB |
Before Width: | Height: | Size: 269 KiB After Width: | Height: | Size: 128 KiB |
BIN
docs/img/examples_keras_jupyter_07_dark.png
Normal file
After Width: | Height: | Size: 130 KiB |
Before Width: | Height: | Size: 120 KiB After Width: | Height: | Size: 118 KiB |
BIN
docs/img/examples_keras_jupyter_08_dark.png
Normal file
After Width: | Height: | Size: 123 KiB |
Before Width: | Height: | Size: 48 KiB After Width: | Height: | Size: 47 KiB |
BIN
docs/img/examples_keras_jupyter_20_dark.png
Normal file
After Width: | Height: | Size: 48 KiB |
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 71 KiB |
BIN
docs/img/examples_keras_jupyter_21_dark.png
Normal file
After Width: | Height: | Size: 73 KiB |
Before Width: | Height: | Size: 94 KiB After Width: | Height: | Size: 72 KiB |
BIN
docs/img/examples_keras_jupyter_23_dark.png
Normal file
After Width: | Height: | Size: 73 KiB |
Before Width: | Height: | Size: 48 KiB After Width: | Height: | Size: 41 KiB |
BIN
docs/img/examples_keras_jupyter_24_dark.png
Normal file
After Width: | Height: | Size: 41 KiB |
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 46 KiB |
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 46 KiB |
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 38 KiB |
BIN
docs/img/examples_sklearn_joblib_example_01_dark.png
Normal file
After Width: | Height: | Size: 38 KiB |
Before Width: | Height: | Size: 50 KiB After Width: | Height: | Size: 44 KiB |
BIN
docs/img/examples_sklearn_joblib_example_02_dark.png
Normal file
After Width: | Height: | Size: 44 KiB |
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 59 KiB |
BIN
docs/img/examples_sklearn_joblib_example_06_dark.png
Normal file
After Width: | Height: | Size: 59 KiB |
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 78 KiB |
BIN
docs/img/examples_tensorboard_pr_curve_01_dark.png
Normal file
After Width: | Height: | Size: 81 KiB |
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 78 KiB |
BIN
docs/img/examples_tensorboard_pr_curve_02_dark.png
Normal file
After Width: | Height: | Size: 81 KiB |
Before Width: | Height: | Size: 77 KiB After Width: | Height: | Size: 77 KiB |
BIN
docs/img/examples_tensorboard_pr_curve_03_dark.png
Normal file
After Width: | Height: | Size: 81 KiB |
Before Width: | Height: | Size: 85 KiB After Width: | Height: | Size: 68 KiB |
BIN
docs/img/examples_tensorboard_pr_curve_04_dark.png
Normal file
After Width: | Height: | Size: 70 KiB |
Before Width: | Height: | Size: 189 KiB After Width: | Height: | Size: 48 KiB |
BIN
docs/img/examples_tensorboard_pr_curve_05_dark.png
Normal file
After Width: | Height: | Size: 49 KiB |
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 71 KiB |
BIN
docs/img/examples_tensorboard_toy_01_dark.png
Normal file
After Width: | Height: | Size: 73 KiB |
Before Width: | Height: | Size: 108 KiB After Width: | Height: | Size: 97 KiB |
BIN
docs/img/examples_tensorboard_toy_03_dark.png
Normal file
After Width: | Height: | Size: 98 KiB |
Before Width: | Height: | Size: 103 KiB After Width: | Height: | Size: 174 KiB |
BIN
docs/img/examples_tensorboard_toy_04_dark.png
Normal file
After Width: | Height: | Size: 185 KiB |
Before Width: | Height: | Size: 221 KiB After Width: | Height: | Size: 388 KiB |
BIN
docs/img/examples_tensorboard_toy_05_dark.png
Normal file
After Width: | Height: | Size: 387 KiB |
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 30 KiB |
BIN
docs/img/examples_tensorboardx_debug_dark.png
Normal file
After Width: | Height: | Size: 30 KiB |
Before Width: | Height: | Size: 76 KiB After Width: | Height: | Size: 69 KiB |
BIN
docs/img/examples_tensorflow_mnist_01_dark.png
Normal file
After Width: | Height: | Size: 69 KiB |
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 42 KiB |
BIN
docs/img/examples_tensorflow_mnist_03_dark.png
Normal file
After Width: | Height: | Size: 42 KiB |
Before Width: | Height: | Size: 262 KiB After Width: | Height: | Size: 113 KiB |
BIN
docs/img/examples_tensorflow_mnist_05_dark.png
Normal file
After Width: | Height: | Size: 116 KiB |
Before Width: | Height: | Size: 70 KiB After Width: | Height: | Size: 112 KiB |
BIN
docs/img/examples_tensorflow_mnist_06_dark.png
Normal file
After Width: | Height: | Size: 116 KiB |
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 43 KiB |
BIN
docs/img/examples_tensorflow_mnist_10_dark.png
Normal file
After Width: | Height: | Size: 43 KiB |
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 38 KiB |
BIN
docs/img/examples_xgboost_metric_artifacts_dark.png
Normal file
After Width: | Height: | Size: 37 KiB |
Before Width: | Height: | Size: 53 KiB After Width: | Height: | Size: 39 KiB |
BIN
docs/img/examples_xgboost_metric_console_dark.png
Normal file
After Width: | Height: | Size: 39 KiB |
Before Width: | Height: | Size: 50 KiB After Width: | Height: | Size: 42 KiB |
BIN
docs/img/examples_xgboost_metric_model_dark.png
Normal file
After Width: | Height: | Size: 42 KiB |
Before Width: | Height: | Size: 53 KiB After Width: | Height: | Size: 56 KiB |
BIN
docs/img/examples_xgboost_metric_scalars_dark.png
Normal file
After Width: | Height: | Size: 59 KiB |
Before Width: | Height: | Size: 49 KiB After Width: | Height: | Size: 44 KiB |
BIN
docs/img/examples_xgboost_sample_03_dark.png
Normal file
After Width: | Height: | Size: 44 KiB |