mads-oestergaard
6dedb4ab41
Merge f640eb6707 into 1926673951
2025-03-10 22:27:14 +01:00
clearml
1926673951
Fix Argv stores env/cwd if passed in ctor
2025-03-03 19:23:51 +02:00
clearml
3b70e1c4a0
Fix uv_replace_pip feature
...
Fix UV cache based on sync/pip-replacement
Fix if UV fails but lock file is missing, revert to UV as pip drop in replacement
Fix use UV bin instead of UV python package to avoid nested VIRTUAL_ENV issues
2025-03-03 19:23:29 +02:00
clearml
528bf314ef
Update GpuFractionsHandler GPU name to mem size
2025-02-24 13:30:30 +02:00
clearml
4f91c45d38
Fix untitled file based on binary is now py/sh based on requested binary
2025-02-24 13:29:56 +02:00
clearml
0a13fd79fc
Make sure that if we fail to kill a child processes we continue to try the rest
2025-02-24 13:26:49 +02:00
clearml
4219835aa1
Fix pip requirements print dump should be sorted
2025-02-24 13:25:23 +02:00
clearml
97cb47d48e
Add docker port mapping parsing and reassigning feature support
...
Add initial component import from clearml-sdk for easier integration
2025-02-24 13:24:35 +02:00
clearml
8f28d2882a
Fix pip freeze dump to comply with yaml fancy print
2025-02-24 13:23:50 +02:00
clearml
546ffff95d
Fix cached venv tried to reinstall priority packages even through they are preinstalled
2025-02-24 13:23:00 +02:00
clearml
a6ae6b2095
Add initial support for --break-system-packages version detection, but the reality is that we do not need it, because by the time we are running it is too late, so we do rm /usr/lib/python3.*/EXTERNALLY-MANAGED
2025-01-26 23:07:09 +02:00
clearml
28e9280a4f
Reduce required packages
2025-01-26 23:03:16 +02:00
clearml
070919973b
Fix python 3.6 compatibility, no := operator
2025-01-05 12:13:21 +02:00
clearml
a26860e79f
Fix default value handling in merge_dicts()
2024-12-26 18:58:24 +02:00
clearml
d8366dedc6
Fix UV priority
...
Fix UV cache is disabled, UV handles its own cache
Fix UV freeze
Fix make sure we do not use pip cache if poetry/uv is used (even if we reverted to pip we can't know if someone changed the repository and now in a new version, a lock file exists)
2024-12-12 23:38:42 +02:00
mads-oestergaard
cc656e2969
Add support for uv as package manager ( #218 )
...
* add uv as a package manager
* update configs
* update worker and defs
* update environ
* Update configs to highlight sync command
* rename to sync_extra_args and set UV_CACHE_DIR
2024-11-27 13:44:55 +02:00
clearml
0ff428bb96
Fix report index not advancing in resource monitoring causes more than one GPU not to be reported
2024-10-28 18:11:00 +02:00
allegroai
a88487ff25
Add support for pip legacy resolver for versions specified in the agent.package_manager.pip_legacy_resolver configuration option
...
Add skip existing packages
2024-09-22 22:36:06 +03:00
allegroai
0bb267115b
Add venvs_cache.path mount override for non-root containers (use: agent.docker_internal_mounts.venvs_cache)
2024-07-24 17:59:18 +03:00
allegroai
93df021108
Add support for .ipynb script entry files (install nbconvert in runtime, copnvert to python and execute the python script), including CLEARML_AGENT_FORCE_TASK_INIT patching of ipynb files (post python conversion)
2024-07-24 17:41:59 +03:00
allegroai
700ae85de0
Fix file mode should be optional in configuration files section
2024-07-24 17:41:06 +03:00
allegroai
f367c5a571
Fix git fetch did not update new tags #209
2024-07-24 17:39:53 +03:00
allegroai
7e8670d57f
Find the correct python version when using a pre-installed python environment
2024-07-21 14:10:38 +03:00
allegroai
77de343863
Use "venv" module if virtualenv is not supported
2024-07-19 13:18:07 +03:00
allegroai
47147e3237
Fix cached repositories were not passing user/token when pulling, agent.vcs_cache.clone_on_pull_fail now defaults to false
2024-04-19 23:50:17 +03:00
allegroai
441e5a73b2
Fix conda env should not be cached if installing into base conda or conda existing env exists
2024-04-19 23:48:10 +03:00
allegroai
4179ac5234
Fix git pulling on cached invalid git entry. On error, re-clone the entire repo again (disable using "agent.vcs_cache.clone_on_pull_fail: false")
2024-04-12 20:29:36 +03:00
allegroai
4387ed73b6
Fix None handling when no limits exist
2024-04-02 16:36:09 +03:00
allegroai
3d43240c8f
Improve conda package manager support
...
Add agent.package_manager.use_conda_base_env (CLEARML_USE_CONDA_BASE_ENV) allowing to use base conda environment (instead of installing a new one)
Fix conda support for python packages with markers and multiple specifications
Added "nvidia" conda channel and support for cuda-toolkit >= 12
2024-04-01 11:36:26 +03:00
allegroai
22672d2444
Improve GPU monitoring
2024-03-17 19:13:57 +02:00
allegroai
6a4fcda1bf
Improve resource monitor
2024-03-17 19:06:57 +02:00
allegroai
f1f9278928
Fix torch resolver settings applied to PytorchRequirement instance are not used
2024-03-17 18:56:47 +02:00
nfzd
2de1c926bf
Use correct Python version in Poetry init ( #179 )
...
* Use correct Python version in Poetry init
* Use interpreter override if configured
* Don't use agent.python_binary if it is empty
---------
Co-authored-by: Michael Mueller <michael.mueller@wsa.com>
2024-03-11 23:36:10 +02:00
allegroai
01e8ffd854
Improve venv cache handling:
...
- Add FileLock readonly mode, default is write mode (i.e. exclusive lock, preserving behavior)
- Add venv cache now uses readonly lock when copying folders from venv cache into target folder. This enables multiple read, single write operation
- Do not lock the cache folder if we do not need to delete old entries
2024-02-29 14:19:24 +02:00
allegroai
74edf6aa36
Fix IOError on file lock when using shared folder
2024-02-29 14:16:25 +02:00
allegroai
09c5ef99af
Fix Python 3.12 support by removing distutil imports
2024-02-29 14:12:21 +02:00
allegroai
17ae28a62f
Add agent.venvs_cache.lock_timeout to control the venv cache folder lock timeout (in seconds, default 30)
2024-02-29 14:06:06 +02:00
allegroai
9a321a410f
Add CLEARML_AGENT_FORCE_TASK_INIT to allow runtime patching of script even if no repo is specified and the code is running a preinstalled docker
2024-02-29 14:02:27 +02:00
allegroai
919013d4fe
Add CLEARML_AGENT_FORCE_POETRY to allow forcing poetry even when using pip requirements manager
2024-02-29 13:59:26 +02:00
allegroai
8d15fd8798
Fix pippip is returned as a pip version if no value exists in agent.package_manager.pip_version
2024-02-29 13:55:41 +02:00
Mads Oestergaard
f640eb6707
Move ssh filter into regex and use list.insert in path
2024-02-02 10:50:52 +01:00
Mads Oestergaard
04fa8a9a86
Fix ssh to http conversion for azure devops repos
2024-01-29 14:39:22 +01:00
allegroai
98fc4f0fb9
Add agent.resource_monitoring.disk_use_path configuration option to allow monitoring a different volume than the one containing the home folder
2023-12-20 17:49:33 +02:00
allegroai
111e774c21
Add extra_index_url sanitization in configuration printout
2023-12-20 17:49:04 +02:00
allegroai
3dd8d783e1
Fix agent.git_host setting will cause git@domain URLs to not be replaced by SSH URLs since furl cannot parse them to obtain host
2023-12-20 17:48:18 +02:00
allegroai
7c3e420df4
Add git clone verbosity using CLEARML_AGENT_GIT_CLONE_VERBOSE env var
2023-12-20 17:47:52 +02:00
allegroai
55b065a114
Update GPU stats and pynvml support
2023-12-20 17:47:19 +02:00
allegroai
f5861b1e4a
Change default agent.enable_git_ask_pass to True
2023-12-20 17:44:41 +02:00
allegroai
564f769ff7
Add agent.docker_args_extra_precedes_task, agent.protected_docker_extra_args
...
to prevent the same switch to be used by both `extra_docker_args` and the a Task's docker args
2023-12-20 17:42:36 +02:00
allegroai
ba4fee7b19
Fix agent.package_manager.poetry_install_extra_args are used in all Poetry commands and not just in install ( #173 )
2023-11-01 15:10:40 +02:00
