allegroai
f1f9278928
Fix torch resolver settings applied to PytorchRequirement instance are not used
2024-03-17 18:56:47 +02:00
nfzd
2de1c926bf
Use correct Python version in Poetry init ( #179 )
...
* Use correct Python version in Poetry init
* Use interpreter override if configured
* Don't use agent.python_binary if it is empty
---------
Co-authored-by: Michael Mueller <michael.mueller@wsa.com>
2024-03-11 23:36:10 +02:00
allegroai
01e8ffd854
Improve venv cache handling:
...
- Add FileLock readonly mode, default is write mode (i.e. exclusive lock, preserving behavior)
- Add venv cache now uses readonly lock when copying folders from venv cache into target folder. This enables multiple read, single write operation
- Do not lock the cache folder if we do not need to delete old entries
2024-02-29 14:19:24 +02:00
allegroai
74edf6aa36
Fix IOError on file lock when using shared folder
2024-02-29 14:16:25 +02:00
allegroai
09c5ef99af
Fix Python 3.12 support by removing distutil imports
2024-02-29 14:12:21 +02:00
allegroai
17ae28a62f
Add agent.venvs_cache.lock_timeout to control the venv cache folder lock timeout (in seconds, default 30)
2024-02-29 14:06:06 +02:00
allegroai
9a321a410f
Add CLEARML_AGENT_FORCE_TASK_INIT to allow runtime patching of script even if no repo is specified and the code is running a preinstalled docker
2024-02-29 14:02:27 +02:00
allegroai
919013d4fe
Add CLEARML_AGENT_FORCE_POETRY to allow forcing poetry even when using pip requirements manager
2024-02-29 13:59:26 +02:00
allegroai
8d15fd8798
Fix pippip is returned as a pip version if no value exists in agent.package_manager.pip_version
2024-02-29 13:55:41 +02:00
allegroai
98fc4f0fb9
Add agent.resource_monitoring.disk_use_path configuration option to allow monitoring a different volume than the one containing the home folder
2023-12-20 17:49:33 +02:00
allegroai
111e774c21
Add extra_index_url sanitization in configuration printout
2023-12-20 17:49:04 +02:00
allegroai
3dd8d783e1
Fix agent.git_host setting will cause git@domain URLs to not be replaced by SSH URLs since furl cannot parse them to obtain host
2023-12-20 17:48:18 +02:00
allegroai
7c3e420df4
Add git clone verbosity using CLEARML_AGENT_GIT_CLONE_VERBOSE env var
2023-12-20 17:47:52 +02:00
allegroai
55b065a114
Update GPU stats and pynvml support
2023-12-20 17:47:19 +02:00
allegroai
f5861b1e4a
Change default agent.enable_git_ask_pass to True
2023-12-20 17:44:41 +02:00
allegroai
564f769ff7
Add agent.docker_args_extra_precedes_task, agent.protected_docker_extra_args
...
to prevent the same switch to be used by both `extra_docker_args` and the a Task's docker args
2023-12-20 17:42:36 +02:00
allegroai
ba4fee7b19
Fix agent.package_manager.poetry_install_extra_args are used in all Poetry commands and not just in install ( #173 )
2023-11-01 15:10:40 +02:00
allegroai
0131db8b7d
Add support for resource_applied() callback in k8s glue
...
Add support for sending log events with k8s-provided timestamps
Refactor env vars infrastructure
2023-11-01 15:10:08 +02:00
allegroai
d16825029d
Add new pytorch no resolver mode and CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE to change resolver on a Task basis, now supports "pip", "direct", "none"
2023-09-02 17:45:10 +03:00
allegroai
fb639afcb9
Fix PyTorch extra index pip resolver
2023-09-02 17:43:41 +03:00
allegroai
4c056a17b9
Add support for k8s jobs execution
...
Strip docker container obtained from task in k8s apply
2023-07-04 14:45:00 +03:00
allegroai
7115a9b9a7
Add CLEARML_EXTRA_PIP_INSTALL_FLAGS / agent.package_manager.extra_pip_install_flags to control additional pip install flags
...
Fix pip version marking in "installed packages" is now preserved for and reinstalled
2023-07-04 14:39:40 +03:00
allegroai
bc85ddf78d
Fix pytorch direct resolve replacing wheel link with directly installed version
2023-05-21 22:53:51 +03:00
allegroai
5b5fb0b8a6
Add agent.package_manager.pytorch_resolve configuration setting with pip or direct values. pip sets extra index based on cuda and lets pip resolve, direct is the previous parsing algorithm that does the matching and downloading (default pip)
2023-05-21 22:53:11 +03:00
allegroai
b6ca0fa6a5
Print error on resource monitor failure
2023-05-11 16:18:11 +03:00
allegroai
307ec9213e
Fix git+ssh:// links inside installed packages not being converted properly to HTTPS authenticated and vice versa
2023-05-11 16:16:51 +03:00
allegroai
154db59ce6
Add agent.package_manager.poetry_install_extra_args configuration option
2023-03-28 14:37:48 +03:00
allegroai
afffa83063
Fix git+ssh:// links inside installed packages not being converted properly to https authenticated links
2023-03-28 14:35:51 +03:00
allegroai
667c2ced3d
Fix very old pip version support (<20)
2023-03-28 14:34:19 +03:00
allegroai
8d51aed679
Protect against cache folders without permission
2023-03-05 18:05:00 +02:00
allegroai
bfc4ba38cd
Fix torch inside nvidia containers to use preinstalled version (i.e. ==x.y.z.* matching)
2023-03-05 18:05:00 +02:00
allegroai
f69e16ea9d
Fix clearml-agent build --docker stuck on certain containers
2022-12-13 15:24:32 +02:00
allegroai
ebdc215632
Remove " from pip commands in venv
2022-12-10 20:58:30 +02:00
allegroai
ca2791c65e
Fix pip support allowing multiple pip version constraints (by default, one for <PY3.10 and one for >=PY3.10)
2022-12-07 22:09:25 +02:00
allegroai
dc5e0033c8
Remove support for kubectl run
...
Allow customizing pod name prefix and limit pod label
Return deleted pods from cleanup
Some refactoring
2022-12-05 11:40:19 +02:00
allegroai
53d379205f
Support raise_error in get_bash_output()
2022-12-05 11:26:40 +02:00
allegroai
d2f3614ab0
Add support for CLEARML_AGENT_DOCKER_ARGS_HIDE_ENV environment variable (see agent.hide_docker_command_env_vars config option)
2022-10-23 13:04:00 +03:00
allegroai
efb06891a8
Add support for PyTorch new extra_index_url repo support. We will find the correct index url based on the cuda version, and let pip do the rest.
2022-10-23 13:04:00 +03:00
allegroai
3f7a4840cc
Add support for operator != in package version (mostly for pytorch resolving)
2022-10-23 13:04:00 +03:00
allegroai
2ef5d38b32
Remove future (Python 2 is not supported for clearml-agent)
2022-10-23 13:03:59 +03:00
allegroai
0de10345f7
Moved pyhocon to internal packages
2022-10-23 13:03:59 +03:00
allegroai
f0fd62a28f
Fix docker extra args showing up in configuration printout
2022-10-23 13:03:59 +03:00
allegroai
7ff6aee20c
Add warning if venv cache is disabled
2022-10-07 15:23:10 +03:00
allegroai
67fc884895
Fix --gpus all not reporting GPU stats on worker machine
2022-10-07 15:22:13 +03:00
allegroai
594ee5842e
Allow to pverride pytorch lookup page: "agent.package_manager.torch_page / torch_nightly_page / torch_url_template_prefix"
2022-09-15 20:16:41 +03:00
allegroai
ec216198a0
Add agent.enable_git_ask_pass to improve passing user/pass to git commands
2022-08-29 18:06:26 +03:00
allegroai
fe6adbf110
Fix package @ file:// with quoted (url style) links should not be ignored
2022-08-29 18:06:09 +03:00
allegroai
ec9d027678
Add support for MIG devices, use 0:1 for GPU 0 slice 1 (or use 0.1)
2022-08-01 18:58:42 +03:00
allegroai
9acbad28f7
Fix repository URL contains credentials even when agent.force_git_ssh_protocolagent.force_git_ssh_protocol is true
2022-06-16 23:20:53 +03:00
allegroai
f66e42ddb1
Fix optional priority packaged always compare lower case package name
2022-06-16 23:18:31 +03:00
