mirror of
https://github.com/clearml/clearml-docs
synced 2025-06-26 18:17:44 +00:00
Small edits (#476)
This commit is contained in:
@@ -2,13 +2,13 @@
|
||||
title: Dataset Management with CLI and SDK
|
||||
---
|
||||
|
||||
In this tutorial, we are going to manage the CIFAR dataset with `clearml-data` CLI, and then use ClearML's [`Dataset`](../../references/sdk/dataset.md)
|
||||
In this tutorial, you are going to manage the CIFAR dataset with `clearml-data` CLI, and then use ClearML's [`Dataset`](../../references/sdk/dataset.md)
|
||||
class to ingest the data.
|
||||
|
||||
## Creating the Dataset
|
||||
|
||||
### Downloading the Data
|
||||
Before we can register the CIFAR dataset with `clearml-data`, we need to obtain a local copy of it.
|
||||
Before registering the CIFAR dataset with `clearml-data`, you need to obtain a local copy of it.
|
||||
|
||||
Execute this python script to download the data
|
||||
```python
|
||||
@@ -43,7 +43,7 @@ New dataset created id=ee1c35f60f384e65bc800f42f0aca5ec
|
||||
Where `ee1c35f60f384e65bc800f42f0aca5ec` is the dataset ID.
|
||||
|
||||
## Adding Files
|
||||
Add the files we just downloaded to the dataset:
|
||||
Add the files that were just downloaded to the dataset:
|
||||
|
||||
```
|
||||
clearml-data add --files <dataset_path>
|
||||
@@ -72,7 +72,7 @@ In the panel's **CONTENT** tab, you can see a table summarizing version contents
|
||||
|
||||
## Using the Dataset
|
||||
|
||||
Now that we have a new dataset registered, we can consume it.
|
||||
Now that a new dataset is registered, you can consume it.
|
||||
|
||||
The [data_ingestion.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/data_ingestion.py) example
|
||||
script demonstrates using the dataset within Python code.
|
||||
@@ -103,6 +103,6 @@ hyperparameters. Passing `alias=<dataset_alias_string>` stores the dataset’s I
|
||||
you can easily track which dataset the task is using.
|
||||
|
||||
The Dataset's [`get_local_copy`](../../references/sdk/dataset.md#get_local_copy) method will return a path to the cached,
|
||||
downloaded dataset. Then we provide the path to PyTorch's dataset object.
|
||||
downloaded dataset. Then the dataset path is input to PyTorch's `datasets` object.
|
||||
|
||||
The script then trains a neural network to classify images using the dataset created above.
|
||||
@@ -18,7 +18,7 @@ demonstrates how to do the following:
|
||||
|
||||
### Downloading the Data
|
||||
|
||||
We first need to obtain a local copy of the CIFAR dataset.
|
||||
You first need to obtain a local copy of the CIFAR dataset.
|
||||
|
||||
```python
|
||||
from clearml import StorageManager
|
||||
@@ -79,7 +79,7 @@ In the panel's **CONTENT** tab, you can see a table summarizing version contents
|
||||
|
||||
## Data Ingestion
|
||||
|
||||
Now that we have a new dataset registered, we can consume it!
|
||||
Now that a new dataset is registered, you can consume it!
|
||||
|
||||
The [data_ingestion.py](https://github.com/allegroai/clearml/blob/master/examples/datasets/data_ingestion.py) script
|
||||
demonstrates data ingestion using the dataset created in the first script.
|
||||
|
||||
@@ -62,7 +62,7 @@ ClearML automatically tracks images logged to TensorboardLogger. They appear in
|
||||
|
||||
## Ignite ClearMLLogger
|
||||
|
||||
PyTorch Ignite also offers a dedicated `ClearMLLogger` handler to log metrics, text, model / optimizer parameters, plots, and model
|
||||
PyTorch Ignite also supports a dedicated `ClearMLLogger` handler to log metrics, text, model / optimizer parameters, plots, and model
|
||||
checkpoints during training and validation.
|
||||
|
||||
For more information, see the [PyTorch Ignite ClearMLLogger](pytorch_ignite_mnist.md)
|
||||
|
||||
@@ -5,7 +5,7 @@ title: PyTorch Ignite ClearMLLogger
|
||||
The `ignite` repository contains the [mnist_with_clearml_logger.py](https://github.com/pytorch/ignite/blob/master/examples/contrib/mnist/mnist_with_clearml_logger.py)
|
||||
example script that uses [ignite](https://github.com/pytorch/ignite) and integrates **ClearMLLogger** and its [helper handlers](https://pytorch.org/ignite/generated/ignite.contrib.handlers.clearml_logger.html).
|
||||
|
||||
PyTorch Ignite offers a `ClearMLLogger` handler to log metrics, text, model / optimizer parameters, plots, and model
|
||||
PyTorch Ignite supports a `ClearMLLogger` handler to log metrics, text, model / optimizer parameters, plots, and model
|
||||
checkpoints during training and validation.
|
||||
|
||||
The example script does the following:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: ClearML Agent on Google Colab
|
||||
---
|
||||
|
||||
[Google Colab](https://colab.research.google.com) is a common development environment for data scientists. It offers a convenient IDE as well as
|
||||
[Google Colab](https://colab.research.google.com) is a common development environment for data scientists. It supports a convenient IDE as well as
|
||||
compute provided by google.
|
||||
|
||||
Users can transform a Google Colab instance into an available resource in ClearML using [ClearML Agent](../../clearml_agent.md).
|
||||
|
||||
@@ -51,7 +51,7 @@ task.register_artifact('train', df, metadata={'counting': 'legs', 'max legs': 69
|
||||
```
|
||||
|
||||
By changing the artifact, and calling the [Task.get_registered_artifacts](../../references/sdk/task.md#get_registered_artifacts)
|
||||
method to retrieve it, we can see that ClearML tracked the change.
|
||||
method to retrieve it, you can see that ClearML tracked the change.
|
||||
|
||||
```python
|
||||
# change the artifact object
|
||||
|
||||
Reference in New Issue
Block a user