Commit Graph

702 Commits

Author SHA1 Message Date
allegroai
df143f1b4e Fix add UTC to time stamp in comment #152 2020-07-06 21:00:47 +03:00
allegroai
2b75c933ee Fix detect installed packages when Trains is not imported directly. 2020-07-06 21:00:01 +03:00
allegroai
2f5b519cd8 Tensorboard text logging report as debug samples (.txt files), instead of as console output. 2020-07-04 22:55:29 +03:00
allegroai
934771184d Fix git diff larger than 500kb , git section will contain warning, and entire git diff will be uploaded as artifact named auxiliary_git_diff 2020-07-04 22:53:26 +03:00
allegroai
d1744785b1 flake8 2020-07-04 22:52:09 +03:00
allegroai
87f7b5d50f Fix limit git diff to 500kb 2020-07-02 01:28:15 +03:00
allegroai
6129e09b50 Fix Hyper-Parameter optimization get_top_experiments() order 2020-07-02 01:28:15 +03:00
allegroai
59b78b288e Add TRAINS_SUPPRESS_UPDATE_MESSAGE #157 2020-07-02 01:28:15 +03:00
allegroai
5cc09d83fd Fix Task.get_task should not auto log console outputs 2020-07-02 01:28:15 +03:00
allegroai
b07a17890d Add task.upload_artifact auto_pickle=True #153
Fix multiple argparse calls before task.init
Add mark_started force argument #159
2020-07-02 01:28:15 +03:00
allegroai
efd843d863 Fix auto logging multiple argparse calls before Task.init 2020-07-02 01:28:15 +03:00
allegroai
b2fb5a3425 Improve Tensorboard histogram scale 2020-07-02 01:28:15 +03:00
allegroai
5d13ec0dfe Fix Tensorboard 2d convolution histogram, improve histogram accuracy on very small histograms.
Add Logger._set_tensorboard_series_prefix add series prefix for all TB reports
2020-06-23 23:08:08 +03:00
allegroai
f6a8736ef6 Fix PyHocon parsing of dictionary keys starting with a numeric value would break the parser 2020-06-23 23:07:05 +03:00
allegroai
a99709d5d0 Version bump to v0.15.1 2020-06-21 23:31:25 +03:00
allegroai
f3621feabb Allow Task.get_tasks() to filter by list of project ids 2020-06-21 23:31:14 +03:00
allegroai
2c5e2bb849 Add Slack Monitor service 2020-06-21 23:30:07 +03:00
allegroai
575f391ba7 Fix ssh://user@domain@server.com/ in git repo link (Issue #149) 2020-06-20 22:15:08 +03:00
allegroai
d2d3e595af Fix incorrect entry point detection when called from Trains wrapper (such as TrainsLogger Ignite/Lightning) 2020-06-19 20:51:46 +03:00
allegroai
5fbfa1d6e2 Add safety guards for Model creation / cache lookup 2020-06-19 19:09:36 +03:00
allegroai
9f9452ecf4 Automation Parameter serialization type name 2020-06-19 19:09:27 +03:00
allegroai
0626d807a7 Add WeightsFileHandler callback type enum 2020-06-19 19:09:17 +03:00
allegroai
9a7850b23d Fix anonymous named Models and input model registration based on remote url instead of local file (when StorageManager is used to download) 2020-06-19 00:52:31 +03:00
allegroai
7ab93e7dba Removed automagic reuse Model objects 2020-06-19 00:46:43 +03:00
allegroai
71003e02b2 hyper-parameter Parameter serialization type is now named "type" 2020-06-19 00:46:01 +03:00
allegroai
01d497451e Fix models and projects api client access 2020-06-19 00:44:38 +03:00
allegroai
2004b64a5b Change Task.wait_for_status() defaults & Add TrainsJob.is_failed() 2020-06-18 01:08:54 +03:00
allegroai
29b4b7338b Fix Logger.tensorboard_single_series_per_graph 2020-06-18 01:08:30 +03:00
allegroai
0454804721 WeightsFileHandler callback documentation 2020-06-18 01:08:10 +03:00
Allegro AI
1475f0e98d
Merge pull request #145 from H4dr1en/patch-3
Add task.wait_for_status()
2020-06-17 21:47:28 +03:00
allegroai
9ee596f48b version bump 2020-06-16 22:02:05 +03:00
allegroai
e03d421b46 Allow auto-model logging callbacks to drop logging 2020-06-16 22:01:52 +03:00
allegroai
bec31c7ac4 Fix matplotlib plot margins 2020-06-15 22:25:32 +03:00
allegroai
9c1d08b826 Fix TensorFlow >=2 histogram binding 2020-06-15 22:23:09 +03:00
H4dr1en
547f69da4b
Add task.wait_for_status() 2020-06-15 15:19:58 +02:00
allegroai
20a9f0997d Add direct plotly figure reporting (see issue #136) 2020-06-14 00:01:30 +03:00
allegroai
8a5f6b7d02 Fix Google CoLab code/package detection 2020-06-13 22:12:28 +03:00
allegroai
aa61fa3f06 Improve pytorch ignite integration 2020-06-13 22:10:59 +03:00
allegroai
a5b1ed0330 Improve matplotlib integration, issue #140 2020-06-13 22:09:45 +03:00
allegroai
2784a48c47 Embed jsonmodels v2.4 package 2020-06-13 22:08:57 +03:00
allegroai
7173a16a16 Update docstrings 2020-06-11 21:15:40 +03:00
allegroai
570b8f19b3 Reuse model id, upload to the correct destination, and remove multiple comments 2020-06-11 17:21:48 +03:00
allegroai
ac986d05f1 Add APIClient support 2020-06-11 15:08:45 +03:00
allegroai
feb82c30a7 Allow providing alternate model ID when getting an output model 2020-06-11 15:07:25 +03:00
allegroai
6a28a6e21d Change callback structure, access thorough designated class WeightsFileHandler.ModelInfo
See https://github.com/pytorch/ignite/issues/1056
2020-06-11 15:05:11 +03:00
allegroai
5475d00d52 Add missing Task types in scheme 2020-06-11 14:47:14 +03:00
allegroai
bd2a0dcc2c Fix trains-init wizard, allow at most two credentials verification tries 2020-06-11 14:41:10 +03:00
allegroai
d5c7ef7eff Do not call make deterministic on import 2020-06-11 14:39:57 +03:00
H4dr1en
91fbc1cf87
Add set_parent method to Tasks 2020-06-09 11:17:42 +02:00
allegroai
00e56faa96 Version bump to v0.15.0 2020-06-01 19:56:21 +03:00
allegroai
7dad7e57e4 Fix automation BOHB budget display calculation, Job.started() and daemon sleep 2020-06-01 11:02:45 +03:00
allegroai
2066d9ff9d Fix type-hints and docstrings 2020-06-01 11:00:42 +03:00
allegroai
43dac458df Change OptimizerBOHB argument max_job_execution_minutes to time_limit_per_job. Add budget remaining reporting 2020-05-31 13:28:03 +03:00
allegroai
2089dde55a Add Task.running_locally() and Task.execute_remotely() 2020-05-31 12:13:54 +03:00
allegroai
dcd16abd3b Add new task types and support for Server API version 2.8 2020-05-31 12:11:48 +03:00
allegroai
92d003657b Fix WeightsFileHandler support for model=None 2020-05-31 12:08:14 +03:00
allegroai
38230626c2 Fix Job.elapsed() 2020-05-31 12:07:12 +03:00
allegroai
f86198bbe5 Add callback for framework save/load binding 2020-05-31 12:06:15 +03:00
allegroai
b865fc0072 If sys.argv doesn't point into a git repo, take file calling Task.init(). Support running code from a module (i.e. -m module) 2020-05-31 12:05:09 +03:00
allegroai
183ad248cf Add type hints, remove/ignore pep8 warnings 2020-05-31 12:02:15 +03:00
allegroai
7b3a554fe9 Fix exact_match_regex() in case of empty pattern 2020-05-31 12:01:03 +03:00
allegroai
77d71819e8 Add Session.get_clients() 2020-05-31 12:00:00 +03:00
allegroai
9259e4efeb Fix type hints, add ignore/fix pep8 warnings 2020-05-31 11:59:05 +03:00
allegroai
7440799bb0 Remove deprecated function call StorageHelper._test_bucket_config() 2020-05-31 11:55:58 +03:00
allegroai
0a0d816bd5 Update docstring 2020-05-25 20:41:15 +03:00
allegroai
cc6bb18e68 Add default port to bandster optimization constructor 2020-05-24 08:20:33 +03:00
allegroai
96f899d028 Fix documentation and layout (PEP8) 2020-05-24 08:16:12 +03:00
allegroai
70cdb13252 Support Task.get_task() without project name (i.e. all projects) 2020-05-24 08:10:47 +03:00
allegroai
7ad4ec2314 Add extra_layout argument for all loggers, setting layout configuration for plotly objects 2020-05-24 08:09:33 +03:00
allegroai
95105cbe6a Add automation support including hyper-parameters optimization 2020-05-22 12:00:07 +03:00
allegroai
b457b9aaad Fix archive support: StorageManager should only try to extract .zip files, Model should not auto extract package 2020-05-22 11:54:46 +03:00
allegroai
2393ac5f7f Add raise_on_error (default=False) argument to Artifat.get_local_copy() 2020-05-22 11:13:13 +03:00
allegroai
03bf764dc7 Make sure Task.close() will wait for repository package analysis (timeout 5 minutes) 2020-05-22 11:04:12 +03:00
allegroai
91bc4b92cd Limit max requirement size to 0.5mb 2020-05-22 11:02:52 +03:00
allegroai
11443e20f8 Fix report histogram xlabels without labels will add "trace 0" in legend instead of series name 2020-05-22 11:01:54 +03:00
allegroai
88cddcfe1d Add raise_on_error (default=False) argument to Model.get_local_copy() 2020-05-22 11:00:17 +03:00
allegroai
163f0c8587 Add utilities.attrs using attr<19.2 flags, supporting attr>=19.2 with no deprecation warnings 2020-05-22 10:53:13 +03:00
allegroai
2848e9190e Fix NaN, Inf and -Inf values in reported table (not supported by JSON) 2020-05-22 10:49:49 +03:00
allegroai
d48fa33744 Improve task reuse by completely clearing the task when reusing (more efficient than multiple edit/reload calls) 2020-05-22 10:45:46 +03:00
allegroai
93fa3bfbbd Add plotly index header 2020-05-22 10:36:36 +03:00
allegroai
0e2265a9ca Add Task.add_requirements() to force requirement package into "installed packages" 2020-05-22 10:35:27 +03:00
allegroai
072abfd6fd Add Task.get_reported_console_output() and Task.get_reported_scalars() 2020-05-22 10:34:45 +03:00
allegroai
2d22efcead Fix TaskTypes/TaskStatusEnum Enum comparison 2020-05-22 10:30:06 +03:00
allegroai
63ca84a84f Allow worker_id override using environment variable 2020-05-22 10:20:43 +03:00
allegroai
3491646ea9 Fix Task.enqueue() to use an exact queue name match 2020-05-22 10:17:17 +03:00
allegroai
5c4cfa9643 Add StorageManager to trains scope 2020-05-22 10:15:02 +03:00
Elijah Rippeth
59874b1620
address DeprecationWarning
Fixes DeprecationWarning: The usage of `cmp` is deprecated and will be removed on or after 2021-06-01.  Please use `eq` and `order` instead.
2020-05-15 21:57:42 -04:00
allegroai
cb8887da72 Task.init argument auto_connect_arg_parser can accept a dictionary disabling specific keys from the argparser 2020-05-13 20:42:33 +03:00
allegroai
d2c9523769 Fix Task.get_task raise proper error on incorrect task_id 2020-05-13 20:35:13 +03:00
allegroai
31a78d27d7 Ignore plotly custom warnings 2020-05-13 20:33:50 +03:00
allegroai
0dec7c0e9e Add Parquet framework and model type 2020-05-13 20:32:49 +03:00
allegroai
9130a06210 OutputModel with task=None should use current task, if exists 2020-05-13 20:31:58 +03:00
allegroai
99f987aafb Fix model package was not recursive when packaging a folder 2020-05-13 20:31:31 +03:00
allegroai
19a24c4b20 Fix default argument behavior to match argparse behavior (if value is the same as default, use default value and type) 2020-05-13 20:29:35 +03:00
allegroai
e1f6e5ec6a Fix typo in warning 2020-05-13 20:26:43 +03:00
allegroai
cb6ecb458a Version bump to v0.14.3 2020-05-10 09:49:14 +03:00
allegroai
7962b041c0 Fix code examples in docstrings 2020-05-08 22:16:20 +03:00
allegroai
5d04b21a79 Fix upload_object_via_stream() in Azure storage driver 2020-05-08 22:13:39 +03:00
allegroai
0298b84030 Refactor histogram code for PyTorch Ignite integration 2020-05-08 22:12:49 +03:00
allegroai
966cd6118a Add Model and Task type-annotations 2020-05-08 22:08:48 +03:00
allegroai
f90f8f06e2 Fix Task.get_tasks() return type to Task 2020-05-08 22:07:43 +03:00
allegroai
a5ff2ba9c8 Add Logger histogram mode (stack/group/relative) 2020-05-08 22:05:33 +03:00
allegroai
5a85d40fc7 Add Logger type-annotations 2020-05-08 21:34:47 +03:00
allegroai
3de95063ee Remove auto_connect_arg_parser constraint when ArgumentParser.parse_arg() is used before initializing task 2020-05-08 21:33:54 +03:00
allegroai
8b5377ee4f Add PEP610 support for new pip>=20.1 git reference feature 2020-05-08 21:31:34 +03:00
allegroai
6991e06e14 Fix RGBA upload with default JPEG files 2020-05-08 21:30:02 +03:00
allegroai
031d26a6cf Support '#' and '?' in image names 2020-05-08 21:28:48 +03:00
allegroai
3f52a6088a Version bump to v0.14.2 2020-04-28 15:32:33 +03:00
allegroai
3d9683f290 Fix logger creation guard 2020-04-28 10:44:40 +03:00
allegroai
7dc27b9a5e Rename utilities dir 2020-04-28 10:43:47 +03:00
allegroai
6746144803 Use UID in message if user name cannot be retrieved 2020-04-26 23:22:46 +03:00
allegroai
7085a2bb74 Report GPU memory usage per process (and sub-process) 2020-04-26 23:21:27 +03:00
allegroai
0a7a32f2eb Support caching of extracted zip artifacts 2020-04-26 23:19:42 +03:00
allegroai
6ff3cc0ee4 Fix duplicate suffix for artifacts 2020-04-26 23:17:51 +03:00
allegroai
302a8cbf75 Report memory usage only for the experiment process (and sub-processes) 2020-04-26 23:16:13 +03:00
allegroai
7e839204d1 Raise error in case of missing artifact directory 2020-04-26 23:14:33 +03:00
allegroai
617e5f7c1d Improve Jupyter support by ignoring IPython directives in converted python script (like % and ! lines) 2020-04-26 23:13:00 +03:00
allegroai
d9f2f01bbd Make sure trains is included in Jupyter requirements 2020-04-26 23:11:51 +03:00
allegroai
dbb3346332 Improve module requirements detection 2020-04-26 23:10:45 +03:00
allegroai
9726f782f2 Add requirement parsing optimization using entry point by default 2020-04-26 22:58:13 +03:00
allegroai
7e4fba9d10 Fix scikit-learn module detection 2020-04-26 22:54:49 +03:00
allegroai
13ce783fa3 Check for updates based on session version 2020-04-26 22:53:18 +03:00
allegroai
466b2859d9 Upload debug assets with series / title including dot or slash did not add iteration index to the filename 2020-04-16 16:49:21 +03:00
allegroai
f7b80a0da2 matplotlib with no logger should not use the last iteration reported by the resource monitor 2020-04-16 16:48:19 +03:00
allegroai
cb139f2d17 Fix Tensorflow V1/V2 audio support 2020-04-16 16:46:02 +03:00
allegroai
215aa32239 Detect exception in debugger session, set Task to Failed 2020-04-16 16:43:10 +03:00
allegroai
4eaa77dbdb Fix single log request exceeds max packet size, set limit to 1MB per request 2020-04-16 16:41:18 +03:00
allegroai
7f4b100042 Fix text encoding utf-8 and pr_curve broken in Tensorboard support 2020-04-16 16:40:14 +03:00
allegroai
4bb17ca420 Fix renaming/deleting model file right after saving will break async upload (trains/issues#123) 2020-04-13 19:03:15 +03:00
allegroai
12659307a8 Fix update_weights() to use model upload target file when passed 2020-04-13 19:00:35 +03:00
allegroai
4b9c5c235c Update docstrings 2020-04-13 18:58:39 +03:00
allegroai
648779380c Add media (audio) support for both Logger and Tensorboard bind 2020-04-09 13:14:14 +03:00
allegroai
7ac7e088a1 Add trace feature 2020-04-09 13:12:50 +03:00
allegroai
0df3d38862 Fix self references in configuration when environment variables exist 2020-04-09 13:11:21 +03:00
allegroai
3ac7dbdb49 Refactor shutdown sequence 2020-04-09 13:10:29 +03:00
allegroai
7dae058359 Optimize locking for TaskHandler, avoid lock when shutting down 2020-04-09 13:08:46 +03:00
allegroai
d9aa83380f Stop resource monitoring before signaling task stop 2020-04-09 13:07:26 +03:00
allegroai
ab263bb59f Raise ValueError if Task.get_logger() is called after task was closed 2020-04-09 13:06:06 +03:00
allegroai
3c4925d605 Fix resource monitor and check if task is valid 2020-04-09 13:00:24 +03:00
allegroai
7f00e45d6c Do not recreate logger after Task was closed/exited 2020-04-09 12:59:00 +03:00
allegroai
3f6fb5379a Revert fork patching as signal is not enough and is not called from forked processes 2020-04-09 12:57:50 +03:00
allegroai
5eb4ae6600 Use a daemon thread for the log 2020-04-09 12:56:55 +03:00
allegroai
1b901b7d13 Fix logger in case a packet was dropped before it was overwritten 2020-04-09 12:56:02 +03:00
allegroai
aa737e6b5b Make sure task is marked as started in remote execution (just in case) 2020-04-09 12:53:43 +03:00
allegroai
f61cbdeb39 Check if join was successful when waiting for std flush pool 2020-04-09 12:51:34 +03:00
allegroai
2f395cc76b Use sub-process fork signal hooks instead of os._exit patch 2020-04-09 12:50:04 +03:00
allegroai
004f925454 ThreadPool should be terminated, not closed, otherwise it might hang 2020-04-09 12:47:38 +03:00
allegroai
9916c93ce0 Add 10sec timeout for stdout/stderr flush at end of process 2020-04-09 12:46:30 +03:00
allegroai
1718aa20d4 Add thread_waited_join waited join for Thread/Process Pools 2020-04-09 12:45:06 +03:00
allegroai
23bd6097a8 Add nicer stdout log flush 2020-04-09 12:42:45 +03:00
allegroai
9a0a84a83e Do not wait for logs if we are aborting the task manually (i.e. ctrl-C) 2020-04-09 12:41:10 +03:00
allegroai
98ce0bbe43 Change TaskHandler.close() wait default to False as it should not wait for logs to flush 2020-04-09 12:39:09 +03:00
allegroai
b3c9872a3f Intercept SystemExit and do nothing so we could kill the thread 2020-04-09 12:33:16 +03:00
allegroai
5ec4d80493 Disconnect stdout/stderr logger on exit 2020-04-09 12:31:43 +03:00
allegroai
de9c88bc2d Do not try to wait for Lock 2020-04-09 12:30:42 +03:00
allegroai
337e60a376 Kill repo/package detection thread on exit 2020-04-09 12:28:57 +03:00
allegroai
b2c2002c40 Create dev task manually when constructing the Task 2020-04-09 12:27:13 +03:00
allegroai
11420adce7 Log reports at the end of the task 2020-04-09 12:24:37 +03:00
allegroai
ffedb219d5 Local modules (except trains) imported from a folder inside the git project should not be logged as "local packages", they should be ignored 2020-04-09 12:21:37 +03:00
allegroai
07daf8f5e6 Fix logger sometimes getting stuck at end of experiment 2020-04-09 12:05:56 +03:00
allegroai
e6f29428eb Add StorageManager 2020-04-09 12:03:41 +03:00
allegroai
e1fc9b3dc8 ThreadPool should be terminated, not closed, otherwise it might hang 2020-04-09 11:39:03 +03:00
allegroai
070fd8149a Store the version that matching the Session API so we do not reload every time 2020-04-09 11:35:51 +03:00
allegroai
a425a70fc6 Add api.ssl_error_count_verbosity and make sure SSL retries are taken care by the session 2020-04-09 11:33:55 +03:00
allegroai
101e5393d1 Fix TRAINS_VCS_ROOT path conversion 2020-04-01 19:06:30 +03:00
allegroai
41ca1a2e49 Fix requirements detection to make sure trains is detected even if we execute without actually being installed 2020-04-01 19:04:57 +03:00
allegroai
01772430d6 Ignore virtual-environment folder that might be inside the project's directory 2020-04-01 19:02:54 +03:00
allegroai
6de3d4b6fd Ignore local modules imported from a folder inside the git project 2020-04-01 19:01:21 +03:00
allegroai
172ed62d41 Add Task.get_tasks() filtering support 2020-04-01 18:54:16 +03:00
allegroai
581edf1098 Version bump to v0.14.1 2020-03-24 20:36:57 +02:00
allegroai
c4719f2e2f Add type annotations and fix docstrings 2020-03-23 23:26:46 +02:00
allegroai
766c8ab24f Add Task.models property 2020-03-23 23:25:55 +02:00
allegroai
0211d233d4 Deprecate Task.set_model_config(), Task.get_model_config_text() and Task.get_model_config_dict() 2020-03-23 23:25:16 +02:00
allegroai
023f1721c1 Add Task.get_models() retrieving stored models on previously executed tasks 2020-03-22 18:19:07 +02:00
allegroai
332e9e2f63 Fix Tensorflow direct V2.1 multiple FileWriters 2020-03-22 18:17:16 +02:00
allegroai
493cce443a Reuse Model objects if we are storing local files (reduce clutter) 2020-03-22 18:15:32 +02:00
allegroai
4e2564cd3a Support reusing Models. Use trains.Model as general purpose registered Model. 2020-03-22 18:13:56 +02:00
allegroai
63507c82f7 Fix Model.download_model_weights() to reuse previously downloaded file 2020-03-22 18:11:30 +02:00
allegroai
477665ee33 Fix storage_uri handling in Model.update() 2020-03-22 18:05:05 +02:00
allegroai
abc9b512f7 Fix logging typos 2020-03-22 18:03:25 +02:00
allegroai
7817ef5cda Fix joblib binding 2020-03-20 10:30:13 +02:00
allegroai
5db53ba643 Support multiple EventWriter in TensorFlow eager mode (TF 2.0+) 2020-03-20 10:29:18 +02:00
allegroai
b4050ecf25 Fix TensorFlow NaN/Inf values support 2020-03-20 10:27:52 +02:00
allegroai
babaf9f1ce Add OpenMPI/Slurm support 2020-03-20 10:23:00 +02:00
allegroai
0adbd79975 Fix StorageHelper upload on shutdown 2020-03-20 10:20:44 +02:00
allegroai
dc915d0241 Fix support for Task init/close multiple times 2020-03-20 10:20:06 +02:00
allegroai
667ddcab88 Fix import for services that do not exist in old versions 2020-03-20 10:16:48 +02:00
allegroai
3b1d2d3258 Version bump to v0.14.0 2020-03-12 19:42:48 +02:00
allegroai
afad6a42ea Add initial slurm support (multiple nodes sharing the same task id) 2020-03-12 18:12:16 +02:00
allegroai
5b29aa194c Make sure artifact temporary files names are valid file names 2020-03-12 18:10:03 +02:00
allegroai
84a34428b6 Add trains-init support for config file env override (as well as argument) 2020-03-12 18:09:03 +02:00
allegroai
b3dff9a4eb Support setting task initial iteration for continuing previous runs 2020-03-12 17:40:29 +02:00
allegroai
f3531c1af2 Allow Task.set_credentials() to override configuration file in dev mode 2020-03-12 17:22:09 +02:00
allegroai
5bc39271e3 Fix store uncommitted code configuration option 2020-03-12 17:17:39 +02:00
allegroai
461fbd9df0 Better warning messages for storage errors 2020-03-12 17:13:36 +02:00
allegroai
30cf6b4834 Fix HTTP link quoting in stored links 2020-03-12 17:04:31 +02:00
allegroai
98c9a95338 Add support for reporting tables 2020-03-10 13:30:42 +02:00
allegroai
9e0ea880ce Add missing import 2020-03-08 18:56:28 +02:00
allegroai
b7358d7fef Add portalocker for inter-process lock 2020-03-05 12:31:22 +02:00
allegroai
2e3820603a Allow argparser override values with command line even in remote execution (essential for sub-process support) 2020-03-05 12:28:36 +02:00
allegroai
1d9e70bd8b Fix signal hooking registeration (cont.) 2020-03-05 12:26:56 +02:00
allegroai
181a0be0af Remove temporary file lock at the end of the execution or in Task.close() 2020-03-05 12:25:17 +02:00
allegroai
b0c602c832 Fix signal hooking registeration 2020-03-05 12:24:14 +02:00
allegroai
bcf97afeb9 Forking processes should not pass along the original File based Lock 2020-03-05 12:22:14 +02:00
allegroai
888c53f67d Allow disabling repository detection when calling Task.init() 2020-03-05 12:19:40 +02:00
allegroai
4bca5ccf27 Always reload task section before editing parts of it 2020-03-05 12:11:55 +02:00
allegroai
a2ecb2c75d Only use file based locks for main task. Secondary tasks use traditional multiprocessing lock 2020-03-05 12:10:23 +02:00
allegroai
da804ca75f Add support for Popen subprocesses with task edit protection from multiple processes 2020-03-05 12:05:12 +02:00
allegroai
e3ae4f4e26 Optimize task refresh while pulling task status in local worker and last iteration for Resource Monitoring 2020-03-05 11:40:27 +02:00
Karthikeyan Singaravelan
a97850e5b6 Import ABC from collections.abc instead of collections for Python 3.9 compatibility. 2020-03-03 21:38:03 +05:30
allegroai
146da439e7 Integrate pigar into Trains 2020-03-01 17:12:28 +02:00
allegroai
8ee2bd1844 Retry sending console logs if session.send() fails (applicable only in local mode where we use the logging handler) 2020-02-26 17:07:07 +02:00
allegroai
cf850020fb Don't print empty line at end of process if there's no artifacts summary 2020-02-26 17:06:17 +02:00
allegroai
baf5fc9e54 version bump to v0.13.3 2020-02-23 11:20:21 +02:00
allegroai
8972c1f005 Add Task.[get/set]_parameters_as_dict() to allow interaction with non-main task parameters (no need to connect()) 2020-02-20 18:32:12 +02:00
allegroai
98e6c2004c Use standard os environment variables to obtain default credentials for AWS, Google and Azure 2020-02-20 18:29:53 +02:00
allegroai
b5168010e9 Make sure Task.connect() returns the same value it is passed 2020-02-18 11:26:52 +02:00
allegroai
14588e6dec Refactor utility function 2020-02-18 11:25:29 +02:00
allegroai
3ea570cadf Store python binary along with major.minor version in task script section 2020-02-18 11:24:04 +02:00
allegroai
9fd3b98b24 Fix session error to print the instance host and not the class host 2020-02-18 11:23:06 +02:00
allegroai
edc237dad4 Improve support for tensorboard.summarywriter.addscalars binding 2020-02-18 11:21:47 +02:00
allegroai
f5f13658c3 Add binding for tensorboard.summarywriter.addscalars as well as scalars grouping configuration option 2020-02-12 14:04:53 +02:00
allegroai
63ffc09ae0 Fix incorrect upgrade message 2020-02-12 14:03:24 +02:00
allegroai
0bc71fbcf4 Remove title/series naming restrictions (allow '$' and '.') 2020-02-10 10:30:57 +02:00
allegroai
3ee70beea2 Fix URL for uploaded files with '%' in their name to allow proper unquote during HTTP serving 2020-02-10 10:30:57 +02:00
allegroai
c6849985ea Add Task.set_base_docer() and Task.get_base_docker() 2020-02-10 10:30:57 +02:00
allegroai
8c2b36968b Prefer tensorflow_gpu over tensorflow when inspecting installed packages 2020-02-04 18:00:39 +02:00
allegroai
0c71889ca5 Fix printout during init 2020-02-04 17:59:50 +02:00
allegroai
34d9402abb version bump to v0.13.2 2020-01-27 19:45:56 +02:00
allegroai
7b9e7406ad Fix mutually_exclusive() use of at_least_one() 2020-01-27 15:41:19 +02:00
allegroai
9f8e814ca6 Support git repositories without ".git" suffix 2020-01-27 15:41:19 +02:00
allegroai
923e45bb17 Allow reporting a pre-uploaded image url in Logger.report_image using the url parameter 2020-01-26 15:29:35 +02:00
allegroai
8772bc2755 Version bump 2020-01-22 11:08:41 +02:00
allegroai
d03311764e Fix None type as default value in dictionary 2020-01-22 11:08:06 +02:00
allegroai
b50bfd5b63 Fix default argparser value handling when value is None 2020-01-22 11:06:52 +02:00
allegroai
af0b8f4c70 Fix type check in hyper-parameters argparser integration 2020-01-22 11:03:56 +02:00
allegroai
1e011e10a2 Version bump 2020-01-21 16:41:14 +02:00
allegroai
1cc0ea6cf3 Fix logs, events and jupyter flushing on exit 2020-01-21 16:41:01 +02:00
allegroai
f0a27127bf Fix matplotlib savefig patching 2020-01-21 16:37:26 +02:00
allegroai
c5dd762d9b Improve conda support 2020-01-21 16:32:57 +02:00
allegroai
9a3e130700 version bump to v0.13.1 2020-01-13 17:29:17 +02:00
allegroai
fcaff82980 Add support for pylab.savefig in matplotlib binding 2020-01-13 17:16:56 +02:00
allegroai
0ecd734fd1 Support multi-line paste of credentials in configuration wizard 2020-01-13 17:16:25 +02:00
allegroai
1c6be01e38 Add support for savefig in matplotlib binding 2020-01-13 12:06:47 +02:00
allegroai
66b251a62b Try to make sure tensorboard is available when using torch 2020-01-13 11:55:55 +02:00
Allegro AI
affd6050f6
Merge pull request #79 from danmalowany-allegro/master
Fix type hint in Logger.report_scatter3d()
2020-01-12 12:50:10 +02:00
allegroai
5fdb2398df Version bump 2020-01-10 13:42:24 +02:00
allegroai
8da8053726 Do not store keras model network design if it cannot be serialized 2020-01-10 13:41:06 +02:00
allegroai
073f4c308d Convert ndarray to histogram for axis to get rid of warning in tensorflow binding 2020-01-10 13:39:31 +02:00
allegroai
163ace8856 Display matplotlib low version warning only once 2020-01-10 13:36:27 +02:00