Small edits (#738)

This commit is contained in:
pollfly 2023-12-26 15:49:35 +02:00 committed by GitHub
parent 25709e30a7
commit 4456da4019
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 22 additions and 19 deletions

View File

@ -19,7 +19,7 @@ line arguments, Python module dependencies, and a requirements.txt file!
1. Execute `clearml-task`, specifying the ClearML target project and task name, along with your script (and repository / commit / branch).
Optionally, specify an execution queue and Docker image to use.
1. `clearml-task` does its magic! It creates a new task on the [ClearML Server](../deploying_clearml/clearml_server.md),
1. `clearml-task` does its magic! It creates a new [ClearML Task](../fundamentals/task.md),
and, if so directed, enqueues it for execution by a ClearML Agent.
1. While the Task is running on the remote machine, all its console outputs are logged in real-time, alongside your
TensorBoard and matplotlib. You can track your script's progress and results in the [ClearML Web UI](../webapp/webapp_overview.md)

View File

@ -18,21 +18,24 @@ demonstrates how to do the following:
### Downloading the Data
You first need to obtain a local copy of the CIFAR dataset.
You first need to obtain a local copy of the CIFAR dataset.
The code below downloads the data and `dataset_path` contains the path to the downloaded data:
```python
from clearml import StorageManager
```python
from clearml import StorageManager
manager = StorageManager()
dataset_path = manager.get_local_copy(
remote_url="https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
)
manager = StorageManager()
dataset_path = manager.get_local_copy(
remote_url="https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
)
```
This script downloads the data and `dataset_path` contains the path to the downloaded data.
### Creating the Dataset
The following code creates a data processing task called `cifar_dataset` in the `dataset examples` project, which
can be viewed in the [WebApp](../../webapp/datasets/webapp_dataset_viewing.md).
```python
from clearml import Dataset
@ -42,23 +45,24 @@ dataset = Dataset.create(
)
```
This creates a data processing task called `cifar_dataset` in the `dataset examples` project, which
can be viewed in the WebApp.
### Adding Files
Add the downloaded files to the current dataset:
```python
dataset.add_files(path=dataset_path)
```
This adds the downloaded files to the current dataset.
### Uploading the Files
Upload the dataset:
```python
dataset.upload()
```
This uploads the dataset to the ClearML Server by default. The dataset's destination can be changed by specifying the
By default, the dataset is uploaded to the ClearML File Server. The dataset's destination can be changed by specifying the
target storage with the `output_url` parameter of the [`upload`](../../references/sdk/dataset.md#upload) method.
### Finalizing the Dataset

View File

@ -7,9 +7,8 @@ on a remote or local machine, from a remote repository and your local machine.
### Prerequisites
- `clearml` Python package installed
- `clearml-agent` running on at least one machine (to execute the experiment), configured to listen to default queue
- [`clearml`](../../getting_started/ds/ds_first_steps.md) Python package installed and configured
- [`clearml-agent`](../../clearml_agent.md#installation) running on at least one machine (to execute the experiment), configured to listen to `default` queue
### Executing Code from a Remote Repository
@ -34,9 +33,9 @@ or add the `--packages "<package_name>"` option to the command (for example: `--
:::
Now `clearml-task` does all the heavy-lifting!
1. It creates a new Task on the [ClearML Server](../../deploying_clearml/clearml_server.md).
1. It creates a new [ClearML Task](../../fundamentals/task.md)
1. `clearml-task` enqueues the task in the selected execution queue, where a [ClearML Agent](../../clearml_agent.md)
assigned to that queue executes the task.
assigned to that queue executes the task
Your output should look something like this: