allegroai
bc85ddf78d
Fix pytorch direct resolve replacing wheel link with directly installed version
2023-05-21 22:53:51 +03:00
allegroai
5b5fb0b8a6
Add agent.package_manager.pytorch_resolve configuration setting with pip or direct values. pip sets extra index based on cuda and lets pip resolve, direct is the previous parsing algorithm that does the matching and downloading (default pip)
2023-05-21 22:53:11 +03:00
allegroai
b6ca0fa6a5
Print error on resource monitor failure
2023-05-11 16:18:11 +03:00
allegroai
307ec9213e
Fix git+ssh:// links inside installed packages not being converted properly to HTTPS authenticated and vice versa
2023-05-11 16:16:51 +03:00
allegroai
154db59ce6
Add agent.package_manager.poetry_install_extra_args configuration option
2023-03-28 14:37:48 +03:00
allegroai
afffa83063
Fix git+ssh:// links inside installed packages not being converted properly to https authenticated links
2023-03-28 14:35:51 +03:00
allegroai
667c2ced3d
Fix very old pip version support (<20)
2023-03-28 14:34:19 +03:00
allegroai
8d51aed679
Protect against cache folders without permission
2023-03-05 18:05:00 +02:00
allegroai
bfc4ba38cd
Fix torch inside nvidia containers to use preinstalled version (i.e. ==x.y.z.* matching)
2023-03-05 18:05:00 +02:00
allegroai
f69e16ea9d
Fix clearml-agent build --docker stuck on certain containers
2022-12-13 15:24:32 +02:00
allegroai
ebdc215632
Remove " from pip commands in venv
2022-12-10 20:58:30 +02:00
allegroai
ca2791c65e
Fix pip support allowing multiple pip version constraints (by default, one for <PY3.10 and one for >=PY3.10)
2022-12-07 22:09:25 +02:00
allegroai
dc5e0033c8
Remove support for kubectl run
...
Allow customizing pod name prefix and limit pod label
Return deleted pods from cleanup
Some refactoring
2022-12-05 11:40:19 +02:00
allegroai
53d379205f
Support raise_error in get_bash_output()
2022-12-05 11:26:40 +02:00
allegroai
d2f3614ab0
Add support for CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV environment variable (see agent.hide_docker_command_env_vars config option)
2022-10-23 13:04:00 +03:00
allegroai
efb06891a8
Add support for PyTorch new extra_index_url repo support. We will find the correct index url based on the cuda version, and let pip do the rest.
2022-10-23 13:04:00 +03:00
allegroai
3f7a4840cc
Add support for operator != in package version (mostly for pytorch resolving)
2022-10-23 13:04:00 +03:00
allegroai
2ef5d38b32
Remove future (Python 2 is not supported for clearml-agent)
2022-10-23 13:03:59 +03:00
allegroai
0de10345f7
Moved pyhocon to internal packages
2022-10-23 13:03:59 +03:00
allegroai
f0fd62a28f
Fix docker extra args showing up in configuration printout
2022-10-23 13:03:59 +03:00
allegroai
7ff6aee20c
Add warning if venv cache is disabled
2022-10-07 15:23:10 +03:00
allegroai
67fc884895
Fix --gpus all not reporting GPU stats on worker machine
2022-10-07 15:22:13 +03:00
allegroai
594ee5842e
Allow to pverride pytorch lookup page: "agent.package_manager.torch_page / torch_nightly_page / torch_url_template_prefix"
2022-09-15 20:16:41 +03:00
allegroai
ec216198a0
Add agent.enable_git_ask_pass to improve passing user/pass to git commands
2022-08-29 18:06:26 +03:00
allegroai
fe6adbf110
Fix package @ file:// with quoted (url style) links should not be ignored
2022-08-29 18:06:09 +03:00
allegroai
ec9d027678
Add support for MIG devices, use 0:1 for GPU 0 slice 1 (or use 0.1)
2022-08-01 18:58:42 +03:00
allegroai
9acbad28f7
Fix repository URL contains credentials even when agent.force_git_ssh_protocolagent.force_git_ssh_protocol is true
2022-06-16 23:20:53 +03:00
allegroai
f66e42ddb1
Fix optional priority packaged always compare lower case package name
2022-06-16 23:18:31 +03:00
allegroai
249aa006cb
Make sure that if we have "setuptools" in the original required packages, we preserve the line in the pip freeze list
2022-05-12 23:31:32 +03:00
allegroai
335ef91d8e
Fix git unsafe directory issue (disable check on cached vcs folder)
2022-05-05 12:22:40 +03:00
allegroai
6c7a639673
Fix broken pytorch setuptools incompatibility (force setuptools < 59 if torch is below 1.11)
2022-05-05 12:22:13 +03:00
allegroai
6572023173
Fix avoid reinstall pytorch package if the same version is already installed
2022-04-09 14:18:38 +03:00
allegroai
c901bd331c
Fix git packages are installed even if commit is given and is preinstalled when using cached virtual environment
2022-03-28 18:11:46 +03:00
allegroai
341086d86a
Fix vcs packages are reinstalled when same commit version is already installed
2022-03-24 22:03:25 +02:00
allegroai
8712c5e636
Fix PyTorch aarch64 and windows support
2022-03-16 17:40:21 +02:00
allegroai
531e514003
Add custom build script support
...
Add extra configurations when starting daemon
Propagate token to docker in case credentials are not available
2022-03-15 10:04:25 +02:00
Idan Tene
e3e6a1dda8
Fix virtualenv python interpreter used ( #98 )
...
* Add virtualenv version logging
* Force using requested python interpreter
2022-02-27 11:25:25 +02:00
allegroai
1f53c4fd1b
Fix agent fails to check out code from main branch when branch/commit is not explicitly specified
2022-02-07 20:04:08 +02:00
allegroai
bfed3ccf4d
Fix agent attempts to check out code when in standalone mode
2022-02-07 20:03:08 +02:00
allegroai
53eba5658f
Fix conda package manager listed packages with local links (@ file://) should ignore the local package if it does not exist
...
Fix cuda patch version support in conda
2022-02-02 16:33:07 +02:00
allegroai
363aaeaba8
Fix symbolic links not copied from cached VCS into working copy. Windows platform will result with default copy content instead of original symbolic link (issue #89 )
2022-01-23 10:42:11 +02:00
allegroai
fa1307e62c
Add agent.poetry_version to specify poetry version (and force installation of poetry if missing)
2022-01-23 10:40:05 +02:00
allegroai
e7c9e9695b
Fix using deprecated abc support
2022-01-23 10:39:13 +02:00
allegroai
b3e8be6296
Add agent.force_git_root_python_path configuration setting to force adding the git repository root folder to the PYTHONPATH (if set working directory is not added to the PYHTONPATH)
2022-01-07 15:11:59 +02:00
allegroai
f41ed09dc1
Add support for custom docker image resolving
2021-12-21 14:29:43 +02:00
allegroai
5a6caf6399
Fix "git+git://" requirements
2021-10-29 22:58:28 +03:00
allegroai
a890e36a36
Fix PY2.7 support for pytorch
2021-10-19 10:47:09 +03:00
allegroai
26d76f52ac
Fix venv cache cannot reinstall package from git with http credentials
2021-08-13 11:00:54 +03:00
allegroai
5c5802c089
Fix python package with git+git:// links or git+ssh:// conversion
2021-08-12 13:37:10 +03:00
allegroai
0fbbe774fa
Fix support for "-r requirements.txt" in installed packages
2021-08-05 19:19:54 +03:00
allegroai
5ed47d2d2c
Add support for CLEARML_NO_DEFAULT_SERVER env var to prevent agent from using the demo server
...
Add support for FORCE_CLEARML_AGENT_REPO env var to allow installing agent from a repo url when executing a task
Implement skip venv installation on execute and allow custom binary
Fix services mode limit implementation in docker mode
2021-08-02 22:51:26 +03:00
Simon Gasse
9456e493ac
Enable rewriting SSH URLs
...
ClearML Agent allows to force git cloning via SSH and also has a
setting to force a username. The relevant settings are:
agent.force_git_ssh_protocol: true
agent.force_git_ssh_user: "git"
However, forcing a specific username or port only worked so far if the
agent translated either from https->ssh or from ssh->https. A given
ssh URL was not rewritten.
This commit adds a helper function and includes it in `_set_ssh_url`
to allow rewriting ssh URLs with the username and/or port given in the
config `agent.force_git_ssh_user`.
If neither username nor port are forced in the config, the URL is not
touched.
This is somewhat related to issue #42 .
Note that rewriting https->https is not covered in this commit.
2021-07-31 23:34:27 +03:00
allegroai
aaa6b32f9f
Fix support for "-r requirements.txt" inside "installed packages"
2021-06-24 19:26:35 +03:00
allegroai
742cbf5767
Add docker environment arguments log masking support (issue #67 )
2021-05-25 19:31:45 +03:00
allegroai
e93384b99b
Fix --stop with dynamic gpus
2021-05-20 10:58:46 +03:00
allegroai
ae3d034531
Protect against None in execution.repository
2021-05-12 15:45:31 +03:00
allegroai
e60a6f9d14
Fix --stop support for dynamic gpus
2021-04-25 10:46:43 +03:00
allegroai
82c5e55fe4
Fix usage of not_set in k8s template merge
2021-04-07 21:30:13 +03:00
allegroai
c59d268995
Fix venv cache crash on bad symbolic links
2021-04-07 18:44:11 +03:00
allegroai
9e9fcb0ba9
Add dynamic mode terminate dockers on sig_term
2021-04-07 18:43:44 +03:00
allegroai
14ac584577
Support k8s glue container env vars merging
2021-03-31 23:53:58 +03:00
allegroai
92a1e07b33
Fix local path replace back when using cache
2021-03-26 12:16:05 +03:00
allegroai
cb6bdece39
Fix cuda version from driver does not return minor version
2021-03-18 10:07:59 +02:00
allegroai
91eec99563
Add conda debug prints (--debug)
2021-03-18 10:07:58 +02:00
allegroai
f019905720
Fix venv cache support for local folders
2021-02-28 19:47:09 +02:00
allegroai
2006ab20dd
Fix conda support for git+http links
2021-02-23 12:46:06 +02:00
allegroai
0caf31719c
Fix venv caching always reinstall git repositories and local repositories
2021-02-23 12:45:34 +02:00
allegroai
58cb344ee6
Upgrade pynvml add detect CUDA version from driver level
2021-02-17 00:03:16 +02:00
allegroai
22d5892b12
Use shared git cache between multiple agents on the same machine
2021-02-14 13:49:29 +02:00
allegroai
ca242424ab
Fix service-mode support for venvs
...
Fix --services-mode with venvs
2021-02-14 13:45:17 +02:00
allegroai
407deb84e9
Fix multi instances on Windows
2021-02-14 13:44:39 +02:00
allegroai
1260e3d942
Update cache entries on conda package manager
2021-02-11 14:47:26 +02:00
allegroai
b22d926d94
Fix cache to take cuda version into account
2021-02-11 14:47:05 +02:00
allegroai
410cc8c7be
Add --dynamic-gpus and limit in --services-mode
2021-02-11 14:46:37 +02:00
allegroai
784c676f5b
Fix "from clearml" runtime diff patching (make sure we move it to after all the __future__ imports) include handling triple quotes in comments
2021-02-11 14:46:06 +02:00
allegroai
296f7970df
Fix file not found error (no 2) interpreted as aborted (i.e. ctrl-c)
2021-02-11 14:44:54 +02:00
allegroai
cd59933c9c
Remove unused packages
2021-02-11 14:44:35 +02:00
allegroai
b95d3f5300
Add venv caching with docker mode support
2021-02-11 14:44:19 +02:00
allegroai
5d8bbde434
Fix applying git diff on new added file
2021-01-24 08:46:42 +02:00
allegroai
161993f66f
Add agent.force_git_ssh_user configuration value (issue #42 )
...
Change default docker to nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
2021-01-10 12:38:45 +02:00
allegroai
428781af86
Fix support for Windows pip and Conda requirements.txt
2020-12-25 02:06:40 +02:00
allegroai
b4d143812e
initial clearml-agent v0.17.0
2020-12-22 23:00:57 +02:00
allegroai
6e1f74402e
Rename trains-agent -> clearml-agent
2020-12-22 21:21:29 +02:00
