87 Commits

Author	SHA1	Message	Date
allegroai	7dae058359	Optimize locking for TaskHandler, avoid lock when shutting down	2020-04-09 13:08:46 +03:00
allegroai	5eb4ae6600	Use a daemon thread for the log	2020-04-09 12:56:55 +03:00
allegroai	1b901b7d13	Fix logger in case a packet was dropped before it was overwritten	2020-04-09 12:56:02 +03:00
allegroai	f61cbdeb39	Check if join was successful when waiting for std flush pool	2020-04-09 12:51:34 +03:00
allegroai	9916c93ce0	Add 10sec timeout for stdout/stderr flush at end of process	2020-04-09 12:46:30 +03:00
allegroai	23bd6097a8	Add nicer stdout log flush	2020-04-09 12:42:45 +03:00
allegroai	98ce0bbe43	Change TaskHandler.close() wait default to False as it should not wait for logs to flush	2020-04-09 12:39:09 +03:00
allegroai	b3c9872a3f	Intercept SystemExit and do nothing so we could kill the thread	2020-04-09 12:33:16 +03:00
allegroai	b2c2002c40	Create dev task manually when constructing the Task	2020-04-09 12:27:13 +03:00
allegroai	11420adce7	Log reports at the end of the task	2020-04-09 12:24:37 +03:00
allegroai	101e5393d1	Fix TRAINS_VCS_ROOT path conversion	2020-04-01 19:06:30 +03:00
allegroai	01772430d6	Ignore virtual-environment folder that might be inside the project's directory	2020-04-01 19:02:54 +03:00
allegroai	023f1721c1	Add Task.get_models() retrieving stored models on previously executed tasks	2020-03-22 18:19:07 +02:00
allegroai	afad6a42ea	Add initial slurm support (multiple nodes sharing the same task id)	2020-03-12 18:12:16 +02:00
allegroai	b3dff9a4eb	Support setting task initial iteration for continuing previous runs	2020-03-12 17:40:29 +02:00
allegroai	5bc39271e3	Fix store uncommitted code configuration option	2020-03-12 17:17:39 +02:00
allegroai	9e0ea880ce	Add missing import	2020-03-08 18:56:28 +02:00
allegroai	4bca5ccf27	Always reload task section before editing parts of it	2020-03-05 12:11:55 +02:00
allegroai	a2ecb2c75d	Only use file based locks for main task. Secondary tasks use traditional multiprocessing lock	2020-03-05 12:10:23 +02:00
allegroai	da804ca75f	Add support for Popen subprocesses with task edit protection from multiple processes	2020-03-05 12:05:12 +02:00
allegroai	e3ae4f4e26	Optimize task refresh while pulling task status in local worker and last iteration for Resource Monitoring	2020-03-05 11:40:27 +02:00
Karthikeyan Singaravelan	a97850e5b6	Import ABC from collections.abc instead of collections for Python 3.9 compatibility.	2020-03-03 21:38:03 +05:30
allegroai	146da439e7	Integrate pigar into Trains	2020-03-01 17:12:28 +02:00
allegroai	8ee2bd1844	Retry sending console logs if session.send() fails (applicable only in local mode where we use the logging handler)	2020-02-26 17:07:07 +02:00
allegroai	3ea570cadf	Store python binary along with major.minor version in task script section	2020-02-18 11:24:04 +02:00
allegroai	63ffc09ae0	Fix incorrect upgrade message	2020-02-12 14:03:24 +02:00
allegroai	c6849985ea	Add Task.set_base_docer() and Task.get_base_docker()	2020-02-10 10:30:57 +02:00
allegroai	8c2b36968b	Prefer tensorflow_gpu over tensorflow when inspecting installed packages	2020-02-04 18:00:39 +02:00
allegroai	9f8e814ca6	Support git repositories without ".git" suffix	2020-01-27 15:41:19 +02:00
allegroai	d03311764e	Fix None type as default value in dictionary	2020-01-22 11:08:06 +02:00
allegroai	b50bfd5b63	Fix default argparser value handling when value is None	2020-01-22 11:06:52 +02:00
allegroai	af0b8f4c70	Fix type check in hyper-parameters argparser integration	2020-01-22 11:03:56 +02:00
allegroai	1cc0ea6cf3	Fix logs, events and jupyter flushing on exit	2020-01-21 16:41:01 +02:00
allegroai	c5dd762d9b	Improve conda support	2020-01-21 16:32:57 +02:00
allegroai	66b251a62b	Try to make sure tensorboard is available when using torch	2020-01-13 11:55:55 +02:00
allegroai	f65ef3e757	Support broken Jupyter version on some conda installations (SageMaker)	2020-01-10 13:33:19 +02:00
allegroai	7820e0d14a	Use an environment variable for setting a default docker image	2020-01-06 17:09:45 +02:00
allegroai	7b7b6e487e	Fix argparser/subparser support and support unsynced connected hyper parameters in remote execution	2020-01-06 17:08:03 +02:00
allegroai	54ae340ccb	Use source task id to determine cloned task parent	2020-01-02 12:01:03 +02:00
allegroai	4e0f711e39	Keep only the input artifacts when cloning a task	2019-12-21 18:30:24 +02:00
allegroai	a992591f3c	Fix artifacts update in auxiliary task	2019-12-15 00:10:12 +02:00
allegroai	7e7329f7a0	Improve argparser automagic support	2019-11-28 00:49:19 +02:00
allegroai	e1a640d37a	Fix support for older git versions (<2)	2019-11-23 01:25:37 +02:00
allegroai	15683b5b43	Fix support for auto package detection for standalone scripts	2019-11-15 21:59:50 +02:00
allegroai	b37aea1839	Add Task name and comment setter	2019-11-15 21:59:24 +02:00
allegroai	9362831269	Add sdk.metric.matplotlib_untitled_history_size to limit number of untitled matplotlib plots (default: 100)	2019-11-08 22:29:36 +02:00
allegroai	1bfee56977	Improve Windows support	2019-11-08 22:28:13 +02:00
allegroai	2104c3ec6b	Add trains-server new capabilities support	2019-10-25 15:15:22 +03:00
allegroai	e0e6d9159b	Fix python 2.7 and Windows support	2019-10-10 21:10:51 +03:00
allegroai	4a42345561	Add support for tuples in hyper-parameters dict	2019-10-04 01:31:26 +03:00