mirror of
https://github.com/clearml/clearml
synced 2025-03-03 02:32:11 +00:00
Documentation
This commit is contained in:
allegroai
parent
68b2e9e242
commit
3f63394b51
@ -25,12 +25,12 @@ your experimentation logs, outputs, and data to one centralized server.
|
||||
|
||||
* Git repository, branch, commit id, entry point and local git diff
|
||||
* Python environment (including specific packages & versions)
|
||||
* StdOut and StdErr
|
||||
* stdout and stderr
|
||||
* Resource Monitoring (CPU/GPU utilization, temperature, IO, network, etc.)
|
||||
* Hyper-parameters
|
||||
* ArgParser for command line parameters with currently used values
|
||||
* Explicit parameters dictionary
|
||||
* Tensorflow Defines (absl-py)
|
||||
* Tensorflow Defines (absl-py)
|
||||
* Initial model weights file
|
||||
* Model snapshots
|
||||
* Tensorboard/TensorboardX scalars, metrics, histograms, images (with audio coming soon)
|
||||
@ -39,8 +39,7 @@ your experimentation logs, outputs, and data to one centralized server.
|
||||
* Seamless integration (including version control) with **Jupyter Notebook**
|
||||
and [*PyCharm* remote debugging](https://github.com/allegroai/trains-pycharm-plugin)
|
||||
|
||||
**Detailed overview of TRAINS offering and system design can be found [here](https://github.com/allegroai/trains/blob/master/docs/brief.md).**
|
||||
|
||||
**Additionally, log data explicitly using [TRAINS Explicit Logging](https://github.com/allegroai/trains/blob/master/docs/logger.md).**
|
||||
|
||||
## Using TRAINS <a name="using-trains"></a>
|
||||
|
||||
|
||||
56
docs/faq.md
56
docs/faq.md
@ -51,8 +51,7 @@ scikit-learn
|
||||
|
||||
TRAINS API
|
||||
|
||||
[How can I use the TRAINS API to fetch data?](#api)
|
||||
|
||||
* [How can I use the TRAINS API to fetch data?](#api)
|
||||
|
||||
## General Information
|
||||
|
||||
@ -92,23 +91,7 @@ export TRAINS_API_HOST="http://localhost:8008"
|
||||
|
||||
### How can I track OS environment variables with experiments? <a name="track-env-vars"></a>
|
||||
|
||||
Set the OS environment variable `TRAINS_LOG_ENVIRONMENT` to either a list of environment variables to track, a wildcard for all environment variables,
|
||||
or unset it with no value and TRAINS does not log environment variables.
|
||||
|
||||
For example, to log the `PWD` and `PYTHONPATH` environment variables:
|
||||
```bash
|
||||
$ export TRAINS_LOG_ENVIRONMENT="PWD,PYTHONPATH"
|
||||
```
|
||||
|
||||
For example, to log all environment variables:
|
||||
```bash
|
||||
$ export TRAINS_LOG_ENVIRONMENT="*"
|
||||
```
|
||||
|
||||
For example, do not log any environment variables (the default):
|
||||
```bash
|
||||
$ export TRAINS_LOG_ENVIRONMENT=
|
||||
```
|
||||
Set the OS environment variable `TRAINS_LOG_ENVIRONMENT` with the variables you need track. See [Specifying Environment Variables to Track](https://github.com/allegroai/trains/blob/master/docs/logger.md#specifying-environment-variables-to-track).
|
||||
|
||||
## Models
|
||||
|
||||
@ -132,10 +115,10 @@ Task.current_task().set_model_label_enumeration( {"label": int(0), } )
|
||||
|
||||
### Can I store the model configuration file as well? <a name="store-model-configuration"></a>
|
||||
|
||||
Yes! Use the `Task.set_model_design()` method:
|
||||
Yes! Use the `Task.set_model_config()` method:
|
||||
|
||||
```python
|
||||
Task.current_task().set_model_design("a very long text with the configuration file's content")
|
||||
Task.current_task().set_model_config("a very long text with the configuration file's content")
|
||||
```
|
||||
|
||||
### I am training multiple models at the same time, but I only see one of them. What happened? <a name="only-last-model-appears"></a>
|
||||
@ -170,13 +153,7 @@ If you still don't care, just ignore this message - it is merely a warning.
|
||||
|
||||
### I do not use Argparser for hyper-parameters. Do you have a solution? <a name="dont-want-argparser"></a>
|
||||
|
||||
Yes! TRAINS supports using a Python dictionary for hyper-parameter logging. Just use:
|
||||
|
||||
```python
|
||||
parameters_dict = Task.current_task().connect(parameters_dict)
|
||||
```
|
||||
|
||||
From this point onward, not only are the dictionary key/value pairs stored as part of the experiment, but any changes to the dictionary will be automatically updated in the task's information.
|
||||
Yes! TRAINS supports [logging an experiment parameter dictionary](https://github.com/allegroai/trains/blob/master/docs/logger.md#logging-an-experiment-parameter-dictionary).
|
||||
|
||||
|
||||
### I noticed that all of my experiments appear as `Training`. Are there other options? <a name="other-experiment-types"></a>
|
||||
@ -260,28 +237,7 @@ Task.current_task().get_logger().report_vector(
|
||||
|
||||
### I want to add more graphs, not just with Tensorboard. Is this supported? <a name="more-graph-types"></a>
|
||||
|
||||
Yes! Use a [Logger](https://github.com/allegroai/trains/blob/master/trains/logger.py) object. An instance can be always be retrieved using the `Task.current_task().get_logger()` method:
|
||||
|
||||
```python
|
||||
# Get a logger object
|
||||
logger = Task.current_task().get_logger()
|
||||
|
||||
# Report some scalar
|
||||
logger.report_scalar("loss", "classification", iteration=42, value=1.337)
|
||||
```
|
||||
|
||||
#### **TRAINS supports:**
|
||||
|
||||
* Scalars
|
||||
* Plots
|
||||
* 2D/3D Scatter Diagrams
|
||||
* Histograms
|
||||
* Surface Diagrams
|
||||
* Confusion Matrices
|
||||
* Images
|
||||
* Text logs
|
||||
|
||||
For a more detailed example, see [here](https://github.com/allegroai/trains/blob/master/examples/manual_reporting.py).
|
||||
Yes! Use the [Logger](https://github.com/allegroai/trains/blob/master/trains/logger.py) module. For more information, see [TRAINS Explicit Logging](https://github.com/allegroai/trains/blob/master/docs/logger.md).
|
||||
|
||||
## Git and Storage
|
||||
|
||||
|
||||
1518
docs/logger.md
Normal file
1518
docs/logger.md
Normal file
File diff suppressed because it is too large
Load Diff
@ -12,6 +12,7 @@ class SetupUploadMixin(object):
|
||||
self, bucket_name, host=None, access_key=None, secret_key=None, region=None, multipart=True, https=True):
|
||||
"""
|
||||
Setup upload options (currently only S3 is supported)
|
||||
|
||||
:param bucket_name: AWS bucket name
|
||||
:type bucket_name: str
|
||||
:param host: Hostname. Only required in case a Non-AWS S3 solution such as a local Minio server is used)
|
||||
|
||||
@ -53,7 +53,7 @@ class Logger(object):
|
||||
|
||||
This is how we send graphs/plots/text to the system, later we can compare the performance of different tasks.
|
||||
|
||||
**Usage: Task.get_logger()**
|
||||
**Usage:** :func:`Logger.current_logger` or :func:`Task.get_logger`
|
||||
"""
|
||||
SeriesInfo = SeriesInfo
|
||||
_stdout_proxy = None
|
||||
@ -64,7 +64,7 @@ class Logger(object):
|
||||
"""
|
||||
**Do not construct Logger manually!**
|
||||
|
||||
please use Logger.current_logger()
|
||||
please use :func:`Logger.current_logger`
|
||||
"""
|
||||
assert isinstance(private_task, _Task), \
|
||||
'Logger object cannot be instantiated externally, use Logger.current_logger()'
|
||||
@ -139,7 +139,7 @@ class Logger(object):
|
||||
"""
|
||||
Return a logger object for the current task. Can be called from anywhere in the code
|
||||
|
||||
:return Singleton Logger object for the current running task
|
||||
:return: Singleton Logger object for the current running task
|
||||
"""
|
||||
from .task import Task
|
||||
task = Task.current_task()
|
||||
|
||||
@ -50,20 +50,20 @@ class Task(_Task):
|
||||
Task (experiment) object represents the current running experiments and connects all the different parts into \
|
||||
a fully reproducible experiment
|
||||
|
||||
Common usage is calling Task.init() to initialize the main task.
|
||||
Common usage is calling :func:`Task.init` to initialize the main task.
|
||||
The main task is development / remote execution mode-aware, and supports connecting various SDK objects
|
||||
such as Models etc. In development mode, the main task supports task reuse (see Task.init() for more
|
||||
such as Models etc. In development mode, the main task supports task reuse (see :func:`Task.init` for more
|
||||
information in development mode features).
|
||||
Any subsequent call to Task.init() will return the already-initialized main task
|
||||
Any subsequent call to :func:`Task.init` will return the already-initialized main task
|
||||
and will not create a new main task.
|
||||
|
||||
Sub-tasks, meaning tasks which are not the main task and are not development / remote execution mode aware, can be
|
||||
created using Task.create(). These tasks do no support task reuse and any call
|
||||
to Task.create() will always create a new task.
|
||||
created using :func:`Task.create`. These tasks do no support task reuse and any call
|
||||
to :func:`Task.create` will always create a new task.
|
||||
|
||||
You can also query existing tasks in the system by calling Task.get_task().
|
||||
You can also query existing tasks in the system by calling :func:`Task.get_task`.
|
||||
|
||||
**Usage: Task.init(...), Task.create() or Task.get_task(...)**
|
||||
**Usage:** :func:`Task.init` or :func:`Task.get_task`
|
||||
"""
|
||||
|
||||
TaskTypes = _Task.TaskTypes
|
||||
@ -142,8 +142,14 @@ class Task(_Task):
|
||||
Notice! The reused task will be reset. (when running remotely, the usual behaviour applies)
|
||||
If reuse_last_task_id is of type string, it will assume this is the task_id to reuse!
|
||||
Note: A closed or published task will not be reused, and a new task will be created.
|
||||
:param output_uri: Default location for output models (currently support folder/S3/GS/ ).
|
||||
:param output_uri: Default location for output models (currently support folder/S3/GS/Azure ).
|
||||
notice: sub-folders (task_id) is created in the destination folder for all outputs.
|
||||
|
||||
Usage example: /mnt/share/folder, s3://bucket/folder , gs://bucket-name/folder,
|
||||
azure://company.blob.core.windows.net/folder/
|
||||
|
||||
Note: When using cloud storage, make sure you install the accompany packages.
|
||||
For example: trains[s3], trains[gs], trains[azure]
|
||||
:param auto_connect_arg_parser: Automatically grab the ArgParser and connect it with the task.
|
||||
if set to false, you can manually connect the ArgParser with task.connect(parser)
|
||||
:param auto_connect_frameworks: If true automatically patch MatplotLib, Keras callbacks, and TensorBoard/X to
|
||||
@ -633,17 +639,39 @@ class Task(_Task):
|
||||
if self.is_main_task():
|
||||
self.__register_at_exit(None)
|
||||
|
||||
def add_artifact(self, name, artifact):
|
||||
def register_artifact(self, name, artifact, metadata=None):
|
||||
"""
|
||||
Add artifact for the current Task, used mostly for Data Audition.
|
||||
Currently supported artifacts object types: pandas.DataFrame
|
||||
|
||||
:param name: name of the artifacts. can override previous artifacts if name already exists
|
||||
:type name: str
|
||||
:param artifact: artifact object, supported artifacts object types: pandas.DataFrame
|
||||
:type artifact: pandas.DataFrame
|
||||
:param str name: name of the artifacts. Notice! it will override previous artifacts if name already exists.
|
||||
:param pandas.DataFrame artifact: artifact object, supported artifacts object types: pandas.DataFrame
|
||||
:param dict metadata: dictionary of key value to store with the artifact (visible in the UI)
|
||||
"""
|
||||
self._artifacts_manager.add_artifact(name=name, artifact=artifact)
|
||||
self._artifacts_manager.register_artifact(name=name, artifact=artifact, metadata=metadata)
|
||||
|
||||
def unregister_artifact(self, name):
|
||||
"""
|
||||
Remove artifact from the watch list. Notice this will not remove the artifacts from the Task.
|
||||
It will only stop monitoring the artifact,
|
||||
the last snapshot of the artifact will be taken immediately in the background.
|
||||
"""
|
||||
self._artifacts_manager.unregister_artifact(name=name)
|
||||
|
||||
def upload_artifact(self, name, artifact_object=None, artifact_file=None, metadata=None):
|
||||
"""
|
||||
Add static artifact to Task. Artifact file/object will be uploaded in the background
|
||||
|
||||
:param str name: Artifact name. Notice! it will override previous artifact if name already exists
|
||||
:param object artifact_object: Artifact object to upload. Currently supports Numpy, PIL.Image.
|
||||
Numpy will be stored as .npz, and Image as .png file.
|
||||
Use None if uploading a file directly with 'artifact_file'.
|
||||
:param str artifact_file: path to artifact file to upload. None means not applicable.
|
||||
Notice you wither artifact object or artifact_file
|
||||
:param dict metadata: Simple key/value dictionary to store on the artifact
|
||||
:return: True if artifact is supported
|
||||
"""
|
||||
raise ValueError("Not implemented yet")
|
||||
|
||||
def is_current_task(self):
|
||||
"""
|
||||
|
||||
Loading…
Reference in New Issue
Block a user