mirror of
https://github.com/clearml/clearml-docs
synced 2025-04-02 12:21:08 +00:00
Small edits (#738)
This commit is contained in:
parent
25709e30a7
commit
4456da4019
@ -19,7 +19,7 @@ line arguments, Python module dependencies, and a requirements.txt file!
|
||||
|
||||
1. Execute `clearml-task`, specifying the ClearML target project and task name, along with your script (and repository / commit / branch).
|
||||
Optionally, specify an execution queue and Docker image to use.
|
||||
1. `clearml-task` does its magic! It creates a new task on the [ClearML Server](../deploying_clearml/clearml_server.md),
|
||||
1. `clearml-task` does its magic! It creates a new [ClearML Task](../fundamentals/task.md),
|
||||
and, if so directed, enqueues it for execution by a ClearML Agent.
|
||||
1. While the Task is running on the remote machine, all its console outputs are logged in real-time, alongside your
|
||||
TensorBoard and matplotlib. You can track your script's progress and results in the [ClearML Web UI](../webapp/webapp_overview.md)
|
||||
|
@ -18,21 +18,24 @@ demonstrates how to do the following:
|
||||
|
||||
### Downloading the Data
|
||||
|
||||
You first need to obtain a local copy of the CIFAR dataset.
|
||||
You first need to obtain a local copy of the CIFAR dataset.
|
||||
The code below downloads the data and `dataset_path` contains the path to the downloaded data:
|
||||
|
||||
```python
|
||||
from clearml import StorageManager
|
||||
```python
|
||||
from clearml import StorageManager
|
||||
|
||||
manager = StorageManager()
|
||||
dataset_path = manager.get_local_copy(
|
||||
remote_url="https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
|
||||
)
|
||||
manager = StorageManager()
|
||||
dataset_path = manager.get_local_copy(
|
||||
remote_url="https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
|
||||
)
|
||||
```
|
||||
|
||||
This script downloads the data and `dataset_path` contains the path to the downloaded data.
|
||||
|
||||
### Creating the Dataset
|
||||
|
||||
The following code creates a data processing task called `cifar_dataset` in the `dataset examples` project, which
|
||||
can be viewed in the [WebApp](../../webapp/datasets/webapp_dataset_viewing.md).
|
||||
|
||||
```python
|
||||
from clearml import Dataset
|
||||
|
||||
@ -42,23 +45,24 @@ dataset = Dataset.create(
|
||||
)
|
||||
```
|
||||
|
||||
This creates a data processing task called `cifar_dataset` in the `dataset examples` project, which
|
||||
can be viewed in the WebApp.
|
||||
|
||||
### Adding Files
|
||||
|
||||
Add the downloaded files to the current dataset:
|
||||
|
||||
```python
|
||||
dataset.add_files(path=dataset_path)
|
||||
```
|
||||
|
||||
This adds the downloaded files to the current dataset.
|
||||
|
||||
### Uploading the Files
|
||||
|
||||
Upload the dataset:
|
||||
|
||||
```python
|
||||
dataset.upload()
|
||||
```
|
||||
This uploads the dataset to the ClearML Server by default. The dataset's destination can be changed by specifying the
|
||||
|
||||
By default, the dataset is uploaded to the ClearML File Server. The dataset's destination can be changed by specifying the
|
||||
target storage with the `output_url` parameter of the [`upload`](../../references/sdk/dataset.md#upload) method.
|
||||
|
||||
### Finalizing the Dataset
|
||||
|
@ -7,9 +7,8 @@ on a remote or local machine, from a remote repository and your local machine.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- `clearml` Python package installed
|
||||
|
||||
- `clearml-agent` running on at least one machine (to execute the experiment), configured to listen to default queue
|
||||
- [`clearml`](../../getting_started/ds/ds_first_steps.md) Python package installed and configured
|
||||
- [`clearml-agent`](../../clearml_agent.md#installation) running on at least one machine (to execute the experiment), configured to listen to `default` queue
|
||||
|
||||
### Executing Code from a Remote Repository
|
||||
|
||||
@ -34,9 +33,9 @@ or add the `--packages "<package_name>"` option to the command (for example: `--
|
||||
:::
|
||||
|
||||
Now `clearml-task` does all the heavy-lifting!
|
||||
1. It creates a new Task on the [ClearML Server](../../deploying_clearml/clearml_server.md).
|
||||
1. It creates a new [ClearML Task](../../fundamentals/task.md)
|
||||
1. `clearml-task` enqueues the task in the selected execution queue, where a [ClearML Agent](../../clearml_agent.md)
|
||||
assigned to that queue executes the task.
|
||||
assigned to that queue executes the task
|
||||
|
||||
Your output should look something like this:
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user