diff --git a/docs/fundamentals/projects.md b/docs/fundamentals/projects.md new file mode 100644 index 00000000..1ac64541 --- /dev/null +++ b/docs/fundamentals/projects.md @@ -0,0 +1,67 @@ +--- +title: Projects +--- + +Projects are contextual containers for [tasks](task.md) and [models](artifacts.md#models) (as well as [dataviews](../hyperdatasets/dataviews.md) +when hyperdatasets are enabled), providing a logical structure similar to file system folders. +An often useful method is to categorize components into projects according to models or objectives. +Grouping into projects helps in identifying tasks, models, and dataviews when queried. + +Projects can be divided into sub-projects (and sub-sub-projects, etc.) just like files and subdirectories on a +computer, making organization easier. + +Projects contain a textual description field for noting relevant information. The WebApp supports markdown rendering +of the description (see [overview](../webapp/webapp_project_overview.md)). + +In addition, the project's default output URI can be specified. When new experiments from +the project are executed, the model checkpoints (snapshots) and artifacts are stored in the default output location. + +## WebApp + +Users can create and modify projects, and see project details in the WebApp (see [WebApp Home](../webapp/webapp_home.md)). +The project's description can be edited in the [overview](../webapp/webapp_overview.md) page. Each project's experiments, +models, and dataviews, can be viewed in the project's [experiments table](../webapp/webapp_exp_table.md), + [models table](../webapp/webapp_model_table.md), and [dataviews table](../hyperdatasets/webapp/webapp_dataviews.md). + +## Usage + +### Creating sub-projects + +When [initializing a task](task.md#task-creation), its project needs to be specified. If the project entered does not exist, it will be created. +Projects can contain sub-projects, just like folders can contain sub-folders. Input into the `project_name` +parameter a target project path. The project path should follow the project tree hierarchy, in which the project and +sub-projects are slash (`/`) delimited. + +For example: + +```python +from clearml import Task + +Task.init(project_name='main_project/sub_project', task_name='test') +``` + +Nesting projects works on multiple levels. For example: `project_name=main_project/sub_project/sub_sub_project` + +Projects can also be created using the [`projects.create`](../references/api/endpoints.md#post-projectscreate) REST API call. + +### View all projects in system + +To view all projects in the system, use the `Task` class method `get_projects`: + +```python +project_list = Task.get_projects() +``` + +This returns a list of project sorted by last update time. + +### More actions + +For additional ways to work with projects, use the REST API `projects` resource. Some of the available actions include: +* [`projects.create`](../references/api/endpoints.md#post-projectscreate) and [`projects.delete`](../references/api/endpoints.md#post-projectsdelete) - create and delete projects +* [`projects.get_hyper_parameters`](../references/api/endpoints.md#post-projectsget_hyper_parameters) - get a list of all hyperparameter sections and names used in a project +* [`projects.merge_projects`](../references/api/endpoints.md#post-projectsmerge) - merge projects into a single project + +See more in the [REST API reference](../references/api/endpoints.md#projects). + + + diff --git a/docs/fundamentals/task.md b/docs/fundamentals/task.md index 2eee4802..8b010cb3 100644 --- a/docs/fundamentals/task.md +++ b/docs/fundamentals/task.md @@ -15,11 +15,8 @@ All the information captured by a task is by default uploaded to the [ClearML Se and it can be visualized in the [ClearML WebApp](../webapp/webapp_overview.md) (UI). ClearML can also be configured to upload model checkpoints, artifacts, and charts to cloud storage (see [Storage](../integrations/storage.md)). -In the UI and code, tasks are grouped into projects, which are logical entities similar to folders. Users can decide +In the UI and code, tasks are grouped into [projects](projects.md), which are logical entities similar to folders. Users can decide how to group tasks, though different models or objectives are usually grouped into different projects. -Projects can be divided into sub-projects (and sub-sub-projects, etc.) just like files and subdirectories on a -computer, making experiment organization easier. In the WebApp, every project has an [**Overview**](../webapp/webapp_project_overview.md) -tab, where a project description can be written and shared. Tasks that are in the system can be accessed and utilized with code. To [access a task](#accessing-tasks), it can be identified either by a project name & task name combination or by a unique ID. diff --git a/sidebars.js b/sidebars.js index b5587e14..c04cad03 100644 --- a/sidebars.js +++ b/sidebars.js @@ -13,7 +13,7 @@ module.exports = { 'Where do I start?': [{'Data Scientists': ['getting_started/ds/ds_first_steps', 'getting_started/ds/ds_second_steps', 'getting_started/ds/best_practices']}, {'MLOps': ['getting_started/mlops/mlops_first_steps','getting_started/mlops/mlops_second_steps','getting_started/mlops/mlops_best_practices']}] }, 'getting_started/architecture']}, - {'ClearML Fundamentals': ['fundamentals/task', 'fundamentals/hyperparameters', 'fundamentals/artifacts', 'fundamentals/logger', 'fundamentals/agents_and_queues', + {'ClearML Fundamentals': ['fundamentals/projects', 'fundamentals/task', 'fundamentals/hyperparameters', 'fundamentals/artifacts', 'fundamentals/logger', 'fundamentals/agents_and_queues', 'fundamentals/hpo', 'fundamentals/pipelines']}, 'clearml_sdk', 'clearml_agent', @@ -59,7 +59,7 @@ module.exports = { 'guides/guidemain', {'Automation': ['guides/automation/manual_random_param_search_example', 'guides/automation/task_piping']}, {'Data Management': ['guides/data management/data_man_simple', 'guides/data management/data_man_folder_sync', 'guides/data management/data_man_cifar_classification']}, - {'ClearML Task': ['guides/clearml-task/clearml_task_tutorial']}, + {'Clearml Task': ['guides/clearml-task/clearml_task_tutorial']}, {'Distributed': ['guides/distributed/distributed_pytorch_example', 'guides/distributed/subprocess_example']}, {'Docker': ['guides/docker/extra_docker_shell_script']}, {'Frameworks': [ @@ -106,7 +106,7 @@ module.exports = { ], rnSidebar: { - 'Release Notes': ['release_notes/ver_1_1','release_notes/ver_1_0', 'release_notes/ver_0_17', 'release_notes/ver_0_16', 'release_notes/ver_0_15', 'release_notes/ver_0_14', + 'Release Notes': ['release_notes/ver_1_0', 'release_notes/ver_0_17', 'release_notes/ver_0_16', 'release_notes/ver_0_15', 'release_notes/ver_0_14', 'release_notes/ver_0_13', 'release_notes/ver_0_12', 'release_notes/ver_0_11', 'release_notes/ver_0_10', 'release_notes/ver_0_9', ],