From 03d3b014a83dc286963e3fa964fcfd8685ae8ea7 Mon Sep 17 00:00:00 2001 From: Fred DeSerres-Blanchet Date: Wed, 2 Nov 2022 10:10:56 -0400 Subject: [PATCH] Dataset: Include another use case for `add_external_files` (#356) --- docs/clearml_data/clearml_data_sdk.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/clearml_data/clearml_data_sdk.md b/docs/clearml_data/clearml_data_sdk.md index c01a1bd6..5e471a99 100644 --- a/docs/clearml_data/clearml_data_sdk.md +++ b/docs/clearml_data/clearml_data_sdk.md @@ -169,15 +169,22 @@ dataset.add_files( ### add_external_files() To add files or folders to the current dataset, leaving them in their original location, use the [`Dataset.add_external_files`](../references/sdk/dataset.md#add_external_files) -method. Input the `source_url` argument, which can be a link from cloud storage (`s3://`, `gs://`, `azure://`) -or local / network storage (`file://`). +method. Input the `source_url` argument, which can be a link or a list of links from cloud storage (`s3://`, `gs://`, `azure://`) +or local / network storage (`file://`). ```python dataset = Dataset.create(dataset_name="my dataset", dataset_project="example project") dataset.add_external_files( source_url="s3://my/bucket/path_to_folder_or_file", dataset_path="/my_dataset/new_folder/" -) +) +dataset.add_external_files( + source_url=[ + "s3://my/bucket/path_to_folder_or_file", + "s3://my/bucket/path_to_another_folder_or_file", + ], + dataset_path="/my_dataset/new_folder/" +) ``` There is an option to add a set of files based on wildcard matching of a single string or a list of wildcards, using the