Commit Graph

95 Commits

Author SHA1 Message Date
allegroai
19a24c4b20 Fix default argument behavior to match argparse behavior (if value is the same as default, use default value and type) 2020-05-13 20:29:35 +03:00
allegroai
617e5f7c1d Improve Jupyter support by ignoring IPython directives in converted python script (like % and ! lines) 2020-04-26 23:13:00 +03:00
allegroai
d9f2f01bbd Make sure trains is included in Jupyter requirements 2020-04-26 23:11:51 +03:00
allegroai
9726f782f2 Add requirement parsing optimization using entry point by default 2020-04-26 22:58:13 +03:00
allegroai
7e4fba9d10 Fix scikit-learn module detection 2020-04-26 22:54:49 +03:00
allegroai
13ce783fa3 Check for updates based on session version 2020-04-26 22:53:18 +03:00
allegroai
4eaa77dbdb Fix single log request exceeds max packet size, set limit to 1MB per request 2020-04-16 16:41:18 +03:00
allegroai
4b9c5c235c Update docstrings 2020-04-13 18:58:39 +03:00
allegroai
7dae058359 Optimize locking for TaskHandler, avoid lock when shutting down 2020-04-09 13:08:46 +03:00
allegroai
5eb4ae6600 Use a daemon thread for the log 2020-04-09 12:56:55 +03:00
allegroai
1b901b7d13 Fix logger in case a packet was dropped before it was overwritten 2020-04-09 12:56:02 +03:00
allegroai
f61cbdeb39 Check if join was successful when waiting for std flush pool 2020-04-09 12:51:34 +03:00
allegroai
9916c93ce0 Add 10sec timeout for stdout/stderr flush at end of process 2020-04-09 12:46:30 +03:00
allegroai
23bd6097a8 Add nicer stdout log flush 2020-04-09 12:42:45 +03:00
allegroai
98ce0bbe43 Change TaskHandler.close() wait default to False as it should not wait for logs to flush 2020-04-09 12:39:09 +03:00
allegroai
b3c9872a3f Intercept SystemExit and do nothing so we could kill the thread 2020-04-09 12:33:16 +03:00
allegroai
b2c2002c40 Create dev task manually when constructing the Task 2020-04-09 12:27:13 +03:00
allegroai
11420adce7 Log reports at the end of the task 2020-04-09 12:24:37 +03:00
allegroai
101e5393d1 Fix TRAINS_VCS_ROOT path conversion 2020-04-01 19:06:30 +03:00
allegroai
01772430d6 Ignore virtual-environment folder that might be inside the project's directory 2020-04-01 19:02:54 +03:00
allegroai
023f1721c1 Add Task.get_models() retrieving stored models on previously executed tasks 2020-03-22 18:19:07 +02:00
allegroai
afad6a42ea Add initial slurm support (multiple nodes sharing the same task id) 2020-03-12 18:12:16 +02:00
allegroai
b3dff9a4eb Support setting task initial iteration for continuing previous runs 2020-03-12 17:40:29 +02:00
allegroai
5bc39271e3 Fix store uncommitted code configuration option 2020-03-12 17:17:39 +02:00
allegroai
9e0ea880ce Add missing import 2020-03-08 18:56:28 +02:00
allegroai
4bca5ccf27 Always reload task section before editing parts of it 2020-03-05 12:11:55 +02:00
allegroai
a2ecb2c75d Only use file based locks for main task. Secondary tasks use traditional multiprocessing lock 2020-03-05 12:10:23 +02:00
allegroai
da804ca75f Add support for Popen subprocesses with task edit protection from multiple processes 2020-03-05 12:05:12 +02:00
allegroai
e3ae4f4e26 Optimize task refresh while pulling task status in local worker and last iteration for Resource Monitoring 2020-03-05 11:40:27 +02:00
Karthikeyan Singaravelan
a97850e5b6 Import ABC from collections.abc instead of collections for Python 3.9 compatibility. 2020-03-03 21:38:03 +05:30
allegroai
146da439e7 Integrate pigar into Trains 2020-03-01 17:12:28 +02:00
allegroai
8ee2bd1844 Retry sending console logs if session.send() fails (applicable only in local mode where we use the logging handler) 2020-02-26 17:07:07 +02:00
allegroai
3ea570cadf Store python binary along with major.minor version in task script section 2020-02-18 11:24:04 +02:00
allegroai
63ffc09ae0 Fix incorrect upgrade message 2020-02-12 14:03:24 +02:00
allegroai
c6849985ea Add Task.set_base_docer() and Task.get_base_docker() 2020-02-10 10:30:57 +02:00
allegroai
8c2b36968b Prefer tensorflow_gpu over tensorflow when inspecting installed packages 2020-02-04 18:00:39 +02:00
allegroai
9f8e814ca6 Support git repositories without ".git" suffix 2020-01-27 15:41:19 +02:00
allegroai
d03311764e Fix None type as default value in dictionary 2020-01-22 11:08:06 +02:00
allegroai
b50bfd5b63 Fix default argparser value handling when value is None 2020-01-22 11:06:52 +02:00
allegroai
af0b8f4c70 Fix type check in hyper-parameters argparser integration 2020-01-22 11:03:56 +02:00
allegroai
1cc0ea6cf3 Fix logs, events and jupyter flushing on exit 2020-01-21 16:41:01 +02:00
allegroai
c5dd762d9b Improve conda support 2020-01-21 16:32:57 +02:00
allegroai
66b251a62b Try to make sure tensorboard is available when using torch 2020-01-13 11:55:55 +02:00
allegroai
f65ef3e757 Support broken Jupyter version on some conda installations (SageMaker) 2020-01-10 13:33:19 +02:00
allegroai
7820e0d14a Use an environment variable for setting a default docker image 2020-01-06 17:09:45 +02:00
allegroai
7b7b6e487e Fix argparser/subparser support and support unsynced connected hyper parameters in remote execution 2020-01-06 17:08:03 +02:00
allegroai
54ae340ccb Use source task id to determine cloned task parent 2020-01-02 12:01:03 +02:00
allegroai
4e0f711e39 Keep only the input artifacts when cloning a task 2019-12-21 18:30:24 +02:00
allegroai
a992591f3c Fix artifacts update in auxiliary task 2019-12-15 00:10:12 +02:00
allegroai
7e7329f7a0 Improve argparser automagic support 2019-11-28 00:49:19 +02:00