clearml
528bf314ef
Update GpuFractionsHandler GPU name to mem size
2025-02-24 13:30:30 +02:00
clearml
4f91c45d38
Fix untitled file based on binary is now py/sh based on requested binary
2025-02-24 13:29:56 +02:00
clearml
0a13fd79fc
Make sure that if we fail to kill a child processes we continue to try the rest
2025-02-24 13:26:49 +02:00
clearml
4219835aa1
Fix pip requirements print dump should be sorted
2025-02-24 13:25:23 +02:00
clearml
97cb47d48e
Add docker port mapping parsing and reassigning feature support
...
Add initial component import from clearml-sdk for easier integration
2025-02-24 13:24:35 +02:00
clearml
8f28d2882a
Fix pip freeze dump to comply with yaml fancy print
2025-02-24 13:23:50 +02:00
clearml
546ffff95d
Fix cached venv tried to reinstall priority packages even through they are preinstalled
2025-02-24 13:23:00 +02:00
clearml
a6ae6b2095
Add initial support for --break-system-packages version detection, but the reality is that we do not need it, because by the time we are running it is too late, so we do rm /usr/lib/python3.*/EXTERNALLY-MANAGED
2025-01-26 23:07:09 +02:00
clearml
28e9280a4f
Reduce required packages
2025-01-26 23:03:16 +02:00
clearml
070919973b
Fix python 3.6 compatibility, no := operator
2025-01-05 12:13:21 +02:00
clearml
a26860e79f
Fix default value handling in merge_dicts()
2024-12-26 18:58:24 +02:00
clearml
d8366dedc6
Fix UV priority
...
Fix UV cache is disabled, UV handles its own cache
Fix UV freeze
Fix make sure we do not use pip cache if poetry/uv is used (even if we reverted to pip we can't know if someone changed the repository and now in a new version, a lock file exists)
2024-12-12 23:38:42 +02:00
mads-oestergaard
cc656e2969
Add support for uv as package manager ( #218 )
...
* add uv as a package manager
* update configs
* update worker and defs
* update environ
* Update configs to highlight sync command
* rename to sync_extra_args and set UV_CACHE_DIR
2024-11-27 13:44:55 +02:00
clearml
0ff428bb96
Fix report index not advancing in resource monitoring causes more than one GPU not to be reported
2024-10-28 18:11:00 +02:00
allegroai
a88487ff25
Add support for pip legacy resolver for versions specified in the agent.package_manager.pip_legacy_resolver configuration option
...
Add skip existing packages
2024-09-22 22:36:06 +03:00
allegroai
0bb267115b
Add venvs_cache.path mount override for non-root containers (use: agent.docker_internal_mounts.venvs_cache)
2024-07-24 17:59:18 +03:00
allegroai
93df021108
Add support for .ipynb script entry files (install nbconvert in runtime, copnvert to python and execute the python script), including CLEARML_AGENT_FORCE_TASK_INIT patching of ipynb files (post python conversion)
2024-07-24 17:41:59 +03:00
allegroai
700ae85de0
Fix file mode should be optional in configuration files section
2024-07-24 17:41:06 +03:00
allegroai
f367c5a571
Fix git fetch did not update new tags #209
2024-07-24 17:39:53 +03:00
allegroai
7e8670d57f
Find the correct python version when using a pre-installed python environment
2024-07-21 14:10:38 +03:00
allegroai
77de343863
Use "venv" module if virtualenv is not supported
2024-07-19 13:18:07 +03:00
allegroai
47147e3237
Fix cached repositories were not passing user/token when pulling, agent.vcs_cache.clone_on_pull_fail now defaults to false
2024-04-19 23:50:17 +03:00
allegroai
441e5a73b2
Fix conda env should not be cached if installing into base conda or conda existing env exists
2024-04-19 23:48:10 +03:00
allegroai
4179ac5234
Fix git pulling on cached invalid git entry. On error, re-clone the entire repo again (disable using "agent.vcs_cache.clone_on_pull_fail: false")
2024-04-12 20:29:36 +03:00
allegroai
4387ed73b6
Fix None handling when no limits exist
2024-04-02 16:36:09 +03:00
allegroai
3d43240c8f
Improve conda package manager support
...
Add agent.package_manager.use_conda_base_env (CLEARML_USE_CONDA_BASE_ENV) allowing to use base conda environment (instead of installing a new one)
Fix conda support for python packages with markers and multiple specifications
Added "nvidia" conda channel and support for cuda-toolkit >= 12
2024-04-01 11:36:26 +03:00
allegroai
22672d2444
Improve GPU monitoring
2024-03-17 19:13:57 +02:00
allegroai
6a4fcda1bf
Improve resource monitor
2024-03-17 19:06:57 +02:00
allegroai
f1f9278928
Fix torch resolver settings applied to PytorchRequirement instance are not used
2024-03-17 18:56:47 +02:00
nfzd
2de1c926bf
Use correct Python version in Poetry init ( #179 )
...
* Use correct Python version in Poetry init
* Use interpreter override if configured
* Don't use agent.python_binary if it is empty
---------
Co-authored-by: Michael Mueller <michael.mueller@wsa.com>
2024-03-11 23:36:10 +02:00
allegroai
01e8ffd854
Improve venv cache handling:
...
- Add FileLock readonly mode, default is write mode (i.e. exclusive lock, preserving behavior)
- Add venv cache now uses readonly lock when copying folders from venv cache into target folder. This enables multiple read, single write operation
- Do not lock the cache folder if we do not need to delete old entries
2024-02-29 14:19:24 +02:00
allegroai
74edf6aa36
Fix IOError on file lock when using shared folder
2024-02-29 14:16:25 +02:00
allegroai
09c5ef99af
Fix Python 3.12 support by removing distutil imports
2024-02-29 14:12:21 +02:00
allegroai
17ae28a62f
Add agent.venvs_cache.lock_timeout to control the venv cache folder lock timeout (in seconds, default 30)
2024-02-29 14:06:06 +02:00
allegroai
9a321a410f
Add CLEARML_AGENT_FORCE_TASK_INIT to allow runtime patching of script even if no repo is specified and the code is running a preinstalled docker
2024-02-29 14:02:27 +02:00
allegroai
919013d4fe
Add CLEARML_AGENT_FORCE_POETRY to allow forcing poetry even when using pip requirements manager
2024-02-29 13:59:26 +02:00
allegroai
8d15fd8798
Fix pippip is returned as a pip version if no value exists in agent.package_manager.pip_version
2024-02-29 13:55:41 +02:00
allegroai
98fc4f0fb9
Add agent.resource_monitoring.disk_use_path configuration option to allow monitoring a different volume than the one containing the home folder
2023-12-20 17:49:33 +02:00
allegroai
111e774c21
Add extra_index_url sanitization in configuration printout
2023-12-20 17:49:04 +02:00
allegroai
3dd8d783e1
Fix agent.git_host setting will cause git@domain URLs to not be replaced by SSH URLs since furl cannot parse them to obtain host
2023-12-20 17:48:18 +02:00
allegroai
7c3e420df4
Add git clone verbosity using CLEARML_AGENT_GIT_CLONE_VERBOSE env var
2023-12-20 17:47:52 +02:00
allegroai
55b065a114
Update GPU stats and pynvml support
2023-12-20 17:47:19 +02:00
allegroai
f5861b1e4a
Change default agent.enable_git_ask_pass to True
2023-12-20 17:44:41 +02:00
allegroai
564f769ff7
Add agent.docker_args_extra_precedes_task, agent.protected_docker_extra_args
...
to prevent the same switch to be used by both `extra_docker_args` and the a Task's docker args
2023-12-20 17:42:36 +02:00
allegroai
ba4fee7b19
Fix agent.package_manager.poetry_install_extra_args are used in all Poetry commands and not just in install ( #173 )
2023-11-01 15:10:40 +02:00
allegroai
0131db8b7d
Add support for resource_applied() callback in k8s glue
...
Add support for sending log events with k8s-provided timestamps
Refactor env vars infrastructure
2023-11-01 15:10:08 +02:00
allegroai
d16825029d
Add new pytorch no resolver mode and CLEARML_AGENT_PACKAGE_PYTORCH_RESOLVE to change resolver on a Task basis, now supports "pip", "direct", "none"
2023-09-02 17:45:10 +03:00
allegroai
fb639afcb9
Fix PyTorch extra index pip resolver
2023-09-02 17:43:41 +03:00
allegroai
4c056a17b9
Add support for k8s jobs execution
...
Strip docker container obtained from task in k8s apply
2023-07-04 14:45:00 +03:00
allegroai
7115a9b9a7
Add CLEARML_EXTRA_PIP_INSTALL_FLAGS / agent.package_manager.extra_pip_install_flags to control additional pip install flags
...
Fix pip version marking in "installed packages" is now preserved for and reinstalled
2023-07-04 14:39:40 +03:00
