Initial beta version

This commit is contained in:
allegroai
2019-06-10 20:00:28 +03:00
parent 3cb9de58c3
commit f595afe6c8
121 changed files with 34975 additions and 0 deletions

54
docs/contributing.md Normal file
View File

@@ -0,0 +1,54 @@
# Guidelines for Contributing
Firstly, we thank you for taking the time to contribute!
The following is a set of guidelines for contributing to TRAINS.
These are primarily guidelines, not rules.
Use your best judgment and feel free to propose changes to this document in a pull request.
## Reporting Bugs
This section guides you through submitting a bug report for TRAINS.
By following these guidelines, you
help maintainers and the community understand your report, reproduce the behavior, and find related reports.
Before creating bug reports, please check whether the bug you want to report already appears [here](link to issues).
You may discover that you do not need to create a bug report.
When you are creating a bug report, please include as much detail as possible.
**Note**: If you find a **Closed** issue that may be the same issue which you are currently experiencing,
then open a **New** issue and include a link to the original (Closed) issue in the body of your new one.
Explain the problem and include additional details to help maintainers reproduce the problem:
* **Use a clear and descriptive title** for the issue to identify the problem.
* **Describe the exact steps necessary to reproduce the problem** in as much detail as possible. Please do not just summarize what you did. Make sure to explain how you did it.
* **Provide the specific environment setup.** Include the `pip freeze` output, specific environment variables, Python version, and other relevant information.
* **Provide specific examples to demonstrate the steps.** Include links to files or GitHub projects, or copy/paste snippets which you use in those examples.
* **If you are reporting any TRAINS crash,** include a crash report with a stack trace from the operating system. Make sure to add the crash report in the issue and place it in a [code block](https://help.github.com/en/articles/getting-started-with-writing-and-formatting-on-github#multiple-lines),
a [file attachment](https://help.github.com/articles/file-attachments-on-issues-and-pull-requests/), or just put it in a [gist](https://gist.github.com/) (and provide link to that gist).
* **Describe the behavior you observed after following the steps** and the exact problem with that behavior.
* **Explain which behavior you expected to see and why.**
* **For Web-App issues, please include screenshots and animated GIFs** which recreate the described steps and clearly demonstrate the problem. You can use [LICEcap](https://www.cockos.com/licecap/) to record GIFs on macOS and Windows, and [silentcast](https://github.com/colinkeenan/silentcast) or [byzanz](https://github.com/threedaymonk/byzanz) on Linux.
## Suggesting Enhancements
This section guides you through submitting an enhancement suggestion for TRAINS, including
completely new features and minor improvements to existing functionality.
By following these guidelines, you help maintainers and the community understand your suggestion and find related suggestions.
Enhancement suggestions are tracked as GitHub issues. After you determine which repository your enhancement suggestion is related to, create an issue on that repository and provide the following:
* **A clear and descriptive title** for the issue to identify the suggestion.
* **A step-by-step description of the suggested enhancement** in as much detail as possible.
* **Specific examples to demonstrate the steps.** Include copy/pasteable snippets which you use in those examples as [Markdown code blocks](https://help.github.com/articles/markdown-basics/#multiple-lines).
* **Describe the current behavior and explain which behavior you expected to see instead and why.**
* **Include screenshots or animated GIFs** which help you demonstrate the steps or point out the part of TRAINS which the suggestion is related to. You can use [LICEcap](https://www.cockos.com/licecap/) to record GIFs on macOS and Windows, and [silentcast](https://github.com/colinkeenan/silentcast) or [byzanz](https://github.com/threedaymonk/byzanz) on Linux.

160
docs/faq.md Normal file
View File

@@ -0,0 +1,160 @@
# FAQ
**Can I store more information on the models? For example, can I store enumeration of classes?**
YES!
Use the SDK `set_model_label_enumeration` method:
```python
Task.current_task().set_model_label_enumeration( {label: int(0), } )
```
**Can I store the model configuration file as well?**
YES!
Use the SDK `set_model_design` method:
```python
Task.current_task().set_model_design( a very long text of the configuration file content )
```
**I want to add more graphs, not just with Tensorboard. Is this supported?**
YES!
Use an SDK [Logger](link to git) object. An instance can be always be retrieved with `Task.current_task().get_logger()`:
```python
logger = Task.current_task().get_logger()
logger.report_scalar("loss", "classification", iteration=42, value=1.337)
```
TRAINS supports scalars, plots, 2d/3d scatter diagrams, histograms, surface diagrams, confusion matrices, images, and text logging.
An example can be found [here](docs/manual_log.py).
**I noticed that all of my experiments appear as “Training”. Are there other options?**
YES!
When creating experiments and calling `Task.init`, you can pass an experiment type.
The currently supported types are `Task.TaskTypes.training` and `Task.TaskTypes.testing`:
```python
task = Task.init(project_name, task_name, Task.TaskTypes.testing)
```
If you feel we should add a few more, let us know in the [issues]() section.
**I noticed I keep getting a message “warning: uncommitted code”. What does it mean?**
TRAINS not only detects your current repository and git commit,
but it also warns you if you are using uncommitted code. TRAINS does this
because uncommitted code means it will be difficult to reproduce this experiment.
**Is there something you can do about uncommitted code running?**
YES!
TRAINS currently stores the git diff together with the project.
The Web-App will soon present the git diff as well. This is coming very soon!
**I read that there is a feature for centralized model storage. How do I use it?**
Pass the `output_uri` parameter to `Task.init`, for example:
```python
Task.init(project_name, task_name, output_uri=/mnt/shared/folder)
```
All of the stored snapshots are copied into a subfolder whose name contains the task ID, for example:
`/mnt/shared/folder/task_6ea4f0b56d994320a713aeaf13a86d9d/models/`
Other options include:
```python
Task.init(project_name, task_name, output_uri=s3://bucket/folder)
```
```python
Task.init(project_name, task_name, output_uri=gs://bucket/folder)
```
These require configuring the cloud storage credentials in `~/trains.conf` (see an [example](v)).
**I am training multiple models at the same time, but I only see one of them. What happened?**
This will be fixed in a future version. Currently, TRAINS does support multiple models
from the same task/experiment so you can find all the models in the project Models tab.
In the Task view, we only present the last one.
**Can I log input and output models manually?**
YES!
See [InputModel]() and [OutputModel]().
For example:
```python
input_model = InputModel.import_model(link_to_initial_model_file)
Task.current_task().connect(input_model)
OutputModel(Task.current_task()).update_weights(link_to_new_model_file_here)
```
**I am using Jupyter Notebook. Is this supported?**
YES!
Jupyter Notebook is supported.
**I do not use ArgParser for hyper-parameters. Do you have a solution?**
YES!
TRAINS supports using a Python dictionary for hyper-parameter logging.
```python
parameters_dict = Task.current_task().connect(parameters_dict)
```
From this point onward, not only are the dictionary key/value pairs stored, but also any change to the dictionary is automatically stored.
**Git is not well supported in Jupyter. We just gave up on properly committing our code. Do you have a solution?**
YES!
Check our [trains-jupyter-plugin](). It is a Jupyter plugin that allows you to commit your notebook directly from Jupyter. It also saves the Python version of the code and creates an updated `requirements.txt` so you know which packages you were using.
**Can I use TRAINS with scikit-learn?**
YES!
scikit-learn is supported. Everything you do is logged, with the caveat that models are not logged automatically.
Models are not logged automatically because, in most cases, scikit-learn is simply pickling the object to files so there is no underlying frame to connect to.
**I am working with PyCharm and remotely debugging a machine, but the git repo is not detected. Do you have a solution?**
YES!
This is such a common occurrence that we created a PyCharm plugin that allows for a remote debugger to grab your local repository / commit ID. See our [trains-pycharm-plugin]() repository for instructions and [latest release]().
**How do I know a new version came out?**
Unfortunately, TRAINS currently does not support auto-update checks. We hope to add this soon.
**Sometimes I see experiments as running while they are not. What is it?**
When the Python process exits in an orderly fashion, TRAINS closes the experiment.
If a process crashes, then sometimes the stop signal is missed. You can safely right click on the experiment in the Web-App and stop it.
**In the experiment log tab, Im missing the first log lines. Where are they?**
Unfortunately, due to speed/optimization issues, we opted to display only the last several hundreds. The full log can be downloaded from the Web-App.