allegroai
|
19a24c4b20
|
Fix default argument behavior to match argparse behavior (if value is the same as default, use default value and type)
|
2020-05-13 20:29:35 +03:00 |
|
allegroai
|
e1f6e5ec6a
|
Fix typo in warning
|
2020-05-13 20:26:43 +03:00 |
|
allegroai
|
cb6ecb458a
|
Version bump to v0.14.3
|
2020-05-10 09:49:14 +03:00 |
|
allegroai
|
7962b041c0
|
Fix code examples in docstrings
|
2020-05-08 22:16:20 +03:00 |
|
allegroai
|
5d04b21a79
|
Fix upload_object_via_stream() in Azure storage driver
|
2020-05-08 22:13:39 +03:00 |
|
allegroai
|
0298b84030
|
Refactor histogram code for PyTorch Ignite integration
|
2020-05-08 22:12:49 +03:00 |
|
allegroai
|
966cd6118a
|
Add Model and Task type-annotations
|
2020-05-08 22:08:48 +03:00 |
|
allegroai
|
f90f8f06e2
|
Fix Task.get_tasks() return type to Task
|
2020-05-08 22:07:43 +03:00 |
|
allegroai
|
a5ff2ba9c8
|
Add Logger histogram mode (stack/group/relative)
|
2020-05-08 22:05:33 +03:00 |
|
allegroai
|
5a85d40fc7
|
Add Logger type-annotations
|
2020-05-08 21:34:47 +03:00 |
|
allegroai
|
3de95063ee
|
Remove auto_connect_arg_parser constraint when ArgumentParser.parse_arg() is used before initializing task
|
2020-05-08 21:33:54 +03:00 |
|
allegroai
|
8b5377ee4f
|
Add PEP610 support for new pip>=20.1 git reference feature
|
2020-05-08 21:31:34 +03:00 |
|
allegroai
|
6991e06e14
|
Fix RGBA upload with default JPEG files
|
2020-05-08 21:30:02 +03:00 |
|
allegroai
|
031d26a6cf
|
Support '#' and '?' in image names
|
2020-05-08 21:28:48 +03:00 |
|
allegroai
|
3f52a6088a
|
Version bump to v0.14.2
|
2020-04-28 15:32:33 +03:00 |
|
allegroai
|
3d9683f290
|
Fix logger creation guard
|
2020-04-28 10:44:40 +03:00 |
|
allegroai
|
7dc27b9a5e
|
Rename utilities dir
|
2020-04-28 10:43:47 +03:00 |
|
allegroai
|
6746144803
|
Use UID in message if user name cannot be retrieved
|
2020-04-26 23:22:46 +03:00 |
|
allegroai
|
7085a2bb74
|
Report GPU memory usage per process (and sub-process)
|
2020-04-26 23:21:27 +03:00 |
|
allegroai
|
0a7a32f2eb
|
Support caching of extracted zip artifacts
|
2020-04-26 23:19:42 +03:00 |
|
allegroai
|
6ff3cc0ee4
|
Fix duplicate suffix for artifacts
|
2020-04-26 23:17:51 +03:00 |
|
allegroai
|
302a8cbf75
|
Report memory usage only for the experiment process (and sub-processes)
|
2020-04-26 23:16:13 +03:00 |
|
allegroai
|
7e839204d1
|
Raise error in case of missing artifact directory
|
2020-04-26 23:14:33 +03:00 |
|
allegroai
|
617e5f7c1d
|
Improve Jupyter support by ignoring IPython directives in converted python script (like % and ! lines)
|
2020-04-26 23:13:00 +03:00 |
|
allegroai
|
d9f2f01bbd
|
Make sure trains is included in Jupyter requirements
|
2020-04-26 23:11:51 +03:00 |
|
allegroai
|
dbb3346332
|
Improve module requirements detection
|
2020-04-26 23:10:45 +03:00 |
|
allegroai
|
9726f782f2
|
Add requirement parsing optimization using entry point by default
|
2020-04-26 22:58:13 +03:00 |
|
allegroai
|
7e4fba9d10
|
Fix scikit-learn module detection
|
2020-04-26 22:54:49 +03:00 |
|
allegroai
|
13ce783fa3
|
Check for updates based on session version
|
2020-04-26 22:53:18 +03:00 |
|
allegroai
|
466b2859d9
|
Upload debug assets with series / title including dot or slash did not add iteration index to the filename
|
2020-04-16 16:49:21 +03:00 |
|
allegroai
|
f7b80a0da2
|
matplotlib with no logger should not use the last iteration reported by the resource monitor
|
2020-04-16 16:48:19 +03:00 |
|
allegroai
|
cb139f2d17
|
Fix Tensorflow V1/V2 audio support
|
2020-04-16 16:46:02 +03:00 |
|
allegroai
|
215aa32239
|
Detect exception in debugger session, set Task to Failed
|
2020-04-16 16:43:10 +03:00 |
|
allegroai
|
4eaa77dbdb
|
Fix single log request exceeds max packet size, set limit to 1MB per request
|
2020-04-16 16:41:18 +03:00 |
|
allegroai
|
7f4b100042
|
Fix text encoding utf-8 and pr_curve broken in Tensorboard support
|
2020-04-16 16:40:14 +03:00 |
|
allegroai
|
4bb17ca420
|
Fix renaming/deleting model file right after saving will break async upload (trains/issues#123)
|
2020-04-13 19:03:15 +03:00 |
|
allegroai
|
12659307a8
|
Fix update_weights() to use model upload target file when passed
|
2020-04-13 19:00:35 +03:00 |
|
allegroai
|
4b9c5c235c
|
Update docstrings
|
2020-04-13 18:58:39 +03:00 |
|
allegroai
|
648779380c
|
Add media (audio) support for both Logger and Tensorboard bind
|
2020-04-09 13:14:14 +03:00 |
|
allegroai
|
7ac7e088a1
|
Add trace feature
|
2020-04-09 13:12:50 +03:00 |
|
allegroai
|
0df3d38862
|
Fix self references in configuration when environment variables exist
|
2020-04-09 13:11:21 +03:00 |
|
allegroai
|
3ac7dbdb49
|
Refactor shutdown sequence
|
2020-04-09 13:10:29 +03:00 |
|
allegroai
|
7dae058359
|
Optimize locking for TaskHandler, avoid lock when shutting down
|
2020-04-09 13:08:46 +03:00 |
|
allegroai
|
d9aa83380f
|
Stop resource monitoring before signaling task stop
|
2020-04-09 13:07:26 +03:00 |
|
allegroai
|
ab263bb59f
|
Raise ValueError if Task.get_logger() is called after task was closed
|
2020-04-09 13:06:06 +03:00 |
|
allegroai
|
3c4925d605
|
Fix resource monitor and check if task is valid
|
2020-04-09 13:00:24 +03:00 |
|
allegroai
|
7f00e45d6c
|
Do not recreate logger after Task was closed/exited
|
2020-04-09 12:59:00 +03:00 |
|
allegroai
|
3f6fb5379a
|
Revert fork patching as signal is not enough and is not called from forked processes
|
2020-04-09 12:57:50 +03:00 |
|
allegroai
|
5eb4ae6600
|
Use a daemon thread for the log
|
2020-04-09 12:56:55 +03:00 |
|
allegroai
|
1b901b7d13
|
Fix logger in case a packet was dropped before it was overwritten
|
2020-04-09 12:56:02 +03:00 |
|
allegroai
|
aa737e6b5b
|
Make sure task is marked as started in remote execution (just in case)
|
2020-04-09 12:53:43 +03:00 |
|
allegroai
|
f61cbdeb39
|
Check if join was successful when waiting for std flush pool
|
2020-04-09 12:51:34 +03:00 |
|
allegroai
|
2f395cc76b
|
Use sub-process fork signal hooks instead of os._exit patch
|
2020-04-09 12:50:04 +03:00 |
|
allegroai
|
004f925454
|
ThreadPool should be terminated, not closed, otherwise it might hang
|
2020-04-09 12:47:38 +03:00 |
|
allegroai
|
9916c93ce0
|
Add 10sec timeout for stdout/stderr flush at end of process
|
2020-04-09 12:46:30 +03:00 |
|
allegroai
|
1718aa20d4
|
Add thread_waited_join waited join for Thread/Process Pools
|
2020-04-09 12:45:06 +03:00 |
|
allegroai
|
23bd6097a8
|
Add nicer stdout log flush
|
2020-04-09 12:42:45 +03:00 |
|
allegroai
|
9a0a84a83e
|
Do not wait for logs if we are aborting the task manually (i.e. ctrl-C)
|
2020-04-09 12:41:10 +03:00 |
|
allegroai
|
98ce0bbe43
|
Change TaskHandler.close() wait default to False as it should not wait for logs to flush
|
2020-04-09 12:39:09 +03:00 |
|
allegroai
|
b3c9872a3f
|
Intercept SystemExit and do nothing so we could kill the thread
|
2020-04-09 12:33:16 +03:00 |
|
allegroai
|
5ec4d80493
|
Disconnect stdout/stderr logger on exit
|
2020-04-09 12:31:43 +03:00 |
|
allegroai
|
de9c88bc2d
|
Do not try to wait for Lock
|
2020-04-09 12:30:42 +03:00 |
|
allegroai
|
337e60a376
|
Kill repo/package detection thread on exit
|
2020-04-09 12:28:57 +03:00 |
|
allegroai
|
b2c2002c40
|
Create dev task manually when constructing the Task
|
2020-04-09 12:27:13 +03:00 |
|
allegroai
|
11420adce7
|
Log reports at the end of the task
|
2020-04-09 12:24:37 +03:00 |
|
allegroai
|
ffedb219d5
|
Local modules (except trains) imported from a folder inside the git project should not be logged as "local packages", they should be ignored
|
2020-04-09 12:21:37 +03:00 |
|
allegroai
|
07daf8f5e6
|
Fix logger sometimes getting stuck at end of experiment
|
2020-04-09 12:05:56 +03:00 |
|
allegroai
|
e6f29428eb
|
Add StorageManager
|
2020-04-09 12:03:41 +03:00 |
|
allegroai
|
e1fc9b3dc8
|
ThreadPool should be terminated, not closed, otherwise it might hang
|
2020-04-09 11:39:03 +03:00 |
|
allegroai
|
070fd8149a
|
Store the version that matching the Session API so we do not reload every time
|
2020-04-09 11:35:51 +03:00 |
|
allegroai
|
a425a70fc6
|
Add api.ssl_error_count_verbosity and make sure SSL retries are taken care by the session
|
2020-04-09 11:33:55 +03:00 |
|
allegroai
|
101e5393d1
|
Fix TRAINS_VCS_ROOT path conversion
|
2020-04-01 19:06:30 +03:00 |
|
allegroai
|
41ca1a2e49
|
Fix requirements detection to make sure trains is detected even if we execute without actually being installed
|
2020-04-01 19:04:57 +03:00 |
|
allegroai
|
01772430d6
|
Ignore virtual-environment folder that might be inside the project's directory
|
2020-04-01 19:02:54 +03:00 |
|
allegroai
|
6de3d4b6fd
|
Ignore local modules imported from a folder inside the git project
|
2020-04-01 19:01:21 +03:00 |
|
allegroai
|
172ed62d41
|
Add Task.get_tasks() filtering support
|
2020-04-01 18:54:16 +03:00 |
|
allegroai
|
581edf1098
|
Version bump to v0.14.1
|
2020-03-24 20:36:57 +02:00 |
|
allegroai
|
c4719f2e2f
|
Add type annotations and fix docstrings
|
2020-03-23 23:26:46 +02:00 |
|
allegroai
|
766c8ab24f
|
Add Task.models property
|
2020-03-23 23:25:55 +02:00 |
|
allegroai
|
0211d233d4
|
Deprecate Task.set_model_config(), Task.get_model_config_text() and Task.get_model_config_dict()
|
2020-03-23 23:25:16 +02:00 |
|
allegroai
|
023f1721c1
|
Add Task.get_models() retrieving stored models on previously executed tasks
|
2020-03-22 18:19:07 +02:00 |
|
allegroai
|
332e9e2f63
|
Fix Tensorflow direct V2.1 multiple FileWriters
|
2020-03-22 18:17:16 +02:00 |
|
allegroai
|
493cce443a
|
Reuse Model objects if we are storing local files (reduce clutter)
|
2020-03-22 18:15:32 +02:00 |
|
allegroai
|
4e2564cd3a
|
Support reusing Models. Use trains.Model as general purpose registered Model.
|
2020-03-22 18:13:56 +02:00 |
|
allegroai
|
63507c82f7
|
Fix Model.download_model_weights() to reuse previously downloaded file
|
2020-03-22 18:11:30 +02:00 |
|
allegroai
|
477665ee33
|
Fix storage_uri handling in Model.update()
|
2020-03-22 18:05:05 +02:00 |
|
allegroai
|
abc9b512f7
|
Fix logging typos
|
2020-03-22 18:03:25 +02:00 |
|
allegroai
|
7817ef5cda
|
Fix joblib binding
|
2020-03-20 10:30:13 +02:00 |
|
allegroai
|
5db53ba643
|
Support multiple EventWriter in TensorFlow eager mode (TF 2.0+)
|
2020-03-20 10:29:18 +02:00 |
|
allegroai
|
b4050ecf25
|
Fix TensorFlow NaN/Inf values support
|
2020-03-20 10:27:52 +02:00 |
|
allegroai
|
babaf9f1ce
|
Add OpenMPI/Slurm support
|
2020-03-20 10:23:00 +02:00 |
|
allegroai
|
0adbd79975
|
Fix StorageHelper upload on shutdown
|
2020-03-20 10:20:44 +02:00 |
|
allegroai
|
dc915d0241
|
Fix support for Task init/close multiple times
|
2020-03-20 10:20:06 +02:00 |
|
allegroai
|
667ddcab88
|
Fix import for services that do not exist in old versions
|
2020-03-20 10:16:48 +02:00 |
|
allegroai
|
3b1d2d3258
|
Version bump to v0.14.0
|
2020-03-12 19:42:48 +02:00 |
|
allegroai
|
afad6a42ea
|
Add initial slurm support (multiple nodes sharing the same task id)
|
2020-03-12 18:12:16 +02:00 |
|
allegroai
|
5b29aa194c
|
Make sure artifact temporary files names are valid file names
|
2020-03-12 18:10:03 +02:00 |
|
allegroai
|
84a34428b6
|
Add trains-init support for config file env override (as well as argument)
|
2020-03-12 18:09:03 +02:00 |
|
allegroai
|
b3dff9a4eb
|
Support setting task initial iteration for continuing previous runs
|
2020-03-12 17:40:29 +02:00 |
|
allegroai
|
f3531c1af2
|
Allow Task.set_credentials() to override configuration file in dev mode
|
2020-03-12 17:22:09 +02:00 |
|