mirror of
https://github.com/clearml/clearml-docs
synced 2025-03-03 02:32:49 +00:00
Small edits (#870)
This commit is contained in:
parent
2a92231e82
commit
b08ae93e9f
@ -7,7 +7,7 @@ In this example we'll create a simple dataset and demonstrate basic actions on i
|
||||
## Prerequisites
|
||||
1. First, make sure that you have cloned the [clearml](https://github.com/allegroai/clearml) repository. It contains all
|
||||
the needed files.
|
||||
1. Open terminal and change directory to the cloned repository's examples folder
|
||||
1. Open terminal and change directory to the cloned repository's examples folder:
|
||||
|
||||
```
|
||||
cd clearml/examples/reporting
|
||||
@ -140,7 +140,7 @@ Using ClearML Data, you can create child datasets that inherit the content of ot
|
||||
1 files removed
|
||||
```
|
||||
|
||||
1. Close and finalize the dataset
|
||||
1. Close and finalize the dataset:
|
||||
|
||||
```bash
|
||||
clearml-data close
|
||||
|
@ -241,7 +241,8 @@ To replace the URL of each model, execute the following commands:
|
||||
sudo docker exec -it clearml-mongo /bin/bash
|
||||
```
|
||||
|
||||
1. Create the following script inside the Docker shell (as well as the URL protocol if you aren't using `s3`):
|
||||
1. Create the following script inside the Docker shell (as well as the URL protocol if you aren't using `s3`).
|
||||
Make sure to replace `<old-bucket-name>` and `<new-bucket-name>`.
|
||||
|
||||
```bash
|
||||
cat <<EOT >> script.js
|
||||
@ -250,7 +251,6 @@ To replace the URL of each model, execute the following commands:
|
||||
db.model.save(e);});
|
||||
EOT
|
||||
```
|
||||
Make sure to replace `<old-bucket-name>` and `<new-bucket-name>`.
|
||||
|
||||
1. Run the script against the backend DB:
|
||||
|
||||
@ -273,7 +273,7 @@ To fix this, the registered URL of each model needs to be replaced with its curr
|
||||
sudo docker exec -it clearml-mongo /bin/bash
|
||||
```
|
||||
|
||||
1. Create the following script inside the Docker shell:
|
||||
1. Create the following script inside the Docker shell (Make sure to replace `<old-bucket-name>` and `<new-bucket-name>`, as well as the URL protocol prefixes if you aren't using S3):
|
||||
```bash
|
||||
cat <<EOT >> script.js
|
||||
db.model.find({uri:{$regex:/^s3/}}).forEach(function(e,i) {
|
||||
@ -281,7 +281,6 @@ To fix this, the registered URL of each model needs to be replaced with its curr
|
||||
db.model.save(e);});
|
||||
EOT
|
||||
```
|
||||
Make sure to replace `<old-bucket-name>` and `<new-bucket-name>`, as well as the URL protocol prefixes if you aren't using S3.
|
||||
1. Run the script against the backend DB:
|
||||
|
||||
```bash
|
||||
|
@ -94,37 +94,37 @@ myDataset = DatasetVersion.get_current(dataset_name='myDataset')
|
||||
|
||||
### Deleting Datasets
|
||||
|
||||
Use the [`Dataset.delete`](../references/hyperdataset/hyperdataset.md#datasetdelete) method to delete a Dataset.
|
||||
Use the [`Dataset.delete`](../references/hyperdataset/hyperdataset.md#datasetdelete) class method to delete a Dataset:
|
||||
|
||||
Delete an empty Dataset (no versions).
|
||||
* Delete an empty Dataset (no versions):
|
||||
|
||||
```python
|
||||
Dataset.delete(dataset_name='MyDataset', delete_all_versions=False, force=False)
|
||||
```
|
||||
```python
|
||||
Dataset.delete(dataset_name='MyDataset', delete_all_versions=False, force=False)
|
||||
```
|
||||
|
||||
Delete a Dataset containing only versions whose status is *Draft*.
|
||||
* Delete a Dataset containing only versions whose status is *Draft*:
|
||||
|
||||
```python
|
||||
Dataset.delete(dataset_name='MyDataset', delete_all_versions=True, force=False)
|
||||
```
|
||||
```python
|
||||
Dataset.delete(dataset_name='MyDataset', delete_all_versions=True, force=False)
|
||||
```
|
||||
|
||||
Delete a Dataset even if it contains versions whose status is *Published*.
|
||||
* Delete a Dataset even if it contains versions whose status is *Published*:
|
||||
|
||||
```python
|
||||
Dataset.delete(dataset_name='MyDataset', delete_all_versions=True, force=True)
|
||||
```
|
||||
```python
|
||||
Dataset.delete(dataset_name='MyDataset', delete_all_versions=True, force=True)
|
||||
```
|
||||
|
||||
Delete a Dataset and the sources associated with its deleted frames:
|
||||
* Delete a Dataset and the sources associated with its deleted frames:
|
||||
|
||||
```python
|
||||
Dataset.delete(
|
||||
dataset_name='MyDataset', delete_all_versions=True, force=True, delete_sources=True
|
||||
)
|
||||
```
|
||||
```python
|
||||
Dataset.delete(
|
||||
dataset_name='MyDataset', delete_all_versions=True, force=True, delete_sources=True
|
||||
)
|
||||
```
|
||||
|
||||
This supports deleting sources located in AWS S3, GCP, and Azure Storage (not local storage). The `delete_sources`
|
||||
parameter is ignored if `delete_all_versions` is `False`. You can view the deletion process' progress by passing
|
||||
`show_progress=True` (`tqdm` required).
|
||||
This supports deleting sources located in AWS S3, GCP, and Azure Storage (not local storage). The `delete_sources`
|
||||
parameter is ignored if `delete_all_versions` is `False`. You can view the deletion process' progress by passing
|
||||
`show_progress=True` (`tqdm` required).
|
||||
|
||||
### Tagging Datasets
|
||||
|
||||
|
@ -38,22 +38,22 @@ during training and validation.
|
||||
Integrate ClearML with the following steps:
|
||||
1. Create a `ClearMLLogger` object:
|
||||
|
||||
```python
|
||||
from ignite.contrib.handlers.clearml_logger import *
|
||||
```python
|
||||
from ignite.contrib.handlers.clearml_logger import *
|
||||
|
||||
clearml_logger = ClearMLLogger(task_name="ignite", project_name="examples")
|
||||
```
|
||||
|
||||
This creates a [ClearML Task](../fundamentals/task.md) called `ignite` in the `examples` project, which captures your
|
||||
script's information, including Git details, uncommitted code, python environment.
|
||||
clearml_logger = ClearMLLogger(task_name="ignite", project_name="examples")
|
||||
```
|
||||
|
||||
This creates a [ClearML Task](../fundamentals/task.md) called `ignite` in the `examples` project, which captures your
|
||||
script's information, including Git details, uncommitted code, python environment.
|
||||
|
||||
You can also pass the following parameters to the `ClearMLLogger` object:
|
||||
* `task_type` – The type of experiment (see [task types](../fundamentals/task.md#task-types)).
|
||||
* `report_freq` – The histogram processing frequency (handles histogram values every X calls to the handler). Affects
|
||||
`GradsHistHandler` and `WeightsHistHandler` (default: 100).
|
||||
* `histogram_update_freq_multiplier` – The histogram report frequency (report first X histograms and once every X
|
||||
reports afterwards) (default: 10).
|
||||
* `histogram_granularity` - Histogram sampling granularity (default: 50).
|
||||
You can also pass the following parameters to the `ClearMLLogger` object:
|
||||
* `task_type` – The type of experiment (see [task types](../fundamentals/task.md#task-types)).
|
||||
* `report_freq` – The histogram processing frequency (handles histogram values every X calls to the handler). Affects
|
||||
`GradsHistHandler` and `WeightsHistHandler` (default: 100).
|
||||
* `histogram_update_freq_multiplier` – The histogram report frequency (report first X histograms and once every X
|
||||
reports afterwards) (default: 10).
|
||||
* `histogram_granularity` - Histogram sampling granularity (default: 50).
|
||||
|
||||
1. Attach the `ClearMLLogger` to output handlers to log metrics:
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user