Commit Graph

294 Commits

Author SHA1 Message Date
Evan Lezar
a35236a8f6 Correct test cases for NVIDIA_VISIBLE_DEVICES=void
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-10-04 14:14:44 +02:00
Evan Lezar
f0bdfbebe4 Add support for multiple swarm resource envvars
This change allows the swarm-resource config option to specify a
comma-separated list of environment variables instead of a single
environment variable.

The first environment variable matched is considered and other
environment variables are ignored.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-10-04 14:11:10 +02:00
Evan Lezar
8c1b9b33c1 Use common code to construct ldconfig hook
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-09-29 12:12:42 +02:00
Evan Lezar
d37c17857e Add nvidia-ctk info generate-cdi command
This change adds functionality to generate CDI specifications
for all devices detected on the system. A specification containing
all GPUs and MIG devices is generated. All libraries on the host
ldcache that have an NVIDIA Driver Version suffix are included as
are the required binaries and IPC sockets.

A hook (based on the nvidia-ctk hook subcommand) to update the ldcache
in the container for the libraries being injected is also added to the
CDI specificiation.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-09-29 12:11:42 +02:00
Evan Lezar
52bb9e186b Add vulkan support through OCI spec modification
This change allows the NVIDIA Container Runtime to inject vulkan
loaders and libraries by modifying the OCI runtime specification.

This allows vulkan applications to run in containers without
additional modifications.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-09-28 16:51:52 +02:00
Evan Lezar
2b08a79206 Ensure that errors are logged
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-09-19 15:29:29 +02:00
Evan Lezar
ffd6ec3c54 Add modifier to inject Tegra platform files
This change adds a modifier to that injects the tegra platform files
* /etc/nv_tegra_release
* /sys/devices/soc0/family

allowing these files to be used for platform detection in a containerized
context such as the GPU device plugin.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-08-08 16:04:20 +02:00
Christopher Desiniotis
3a9de13f4e Apply 1 suggestion(s) to 1 file(s) 2022-07-21 08:03:39 +00:00
Evan Lezar
1161b21166 Make error message clearer
This change improves the error message when invoking the NVIDIA
Runtime Hook in non-legacy mode. This should guide users to specifying
the --runtime=nvidia flag when using docker.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-18 13:09:59 +02:00
Evan Lezar
37ee972f74 Merge branch 'CNT-2349/configure-docker' into 'main'
Add nvidia-ctk runtime configure command to update docker config

See merge request nvidia/container-toolkit/container-toolkit!166
2022-07-14 08:06:27 +00:00
Evan Lezar
9f0060f651 Add nvidia-ctk runtime configure command
This change adds a `runtime configure` command to the nvidia-ctk CLI. This
command is currently limited to configuring the docker config on the
system by modifying the daemon.json config file associated with docker.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-13 10:33:46 +02:00
Evan Lezar
f50aecb84e Rename -toolkit executable to -runtime-hook
This change renames the nvidia-container-toolkit executable
to nvidia-container-runtime-hook. Here nvidia-container-toolkit
is created as a symlink to nvidia-container-runtime-hook.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-08 12:09:11 +02:00
Evan Lezar
404e266222 Add cdi mode to NVIDIA Container Runtime
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-06 16:53:05 +02:00
Evan Lezar
925c348565 Add DevicesFromEnvvars function to CUDA image
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-01 16:12:13 +02:00
Evan Lezar
7f7bec0668 Create GDS and MOFED modifiers
This change creates GDS and MOFED modifiers and adds them to the
modifer created for the selected runtime mode if the NVIDIA_GDS
and NVIDIA_MOFED envvars are set to "enabled", respectively.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-01 14:54:05 +02:00
Evan Lezar
e8843c38f2 Move cmd/nvidia-container-runtime/modifier package to internal/modifier
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-01 14:28:40 +02:00
Evan Lezar
d66c00dd1d Use modifier list and discoverModifer
This change uses modifier compositioning and the discoverModifier to
refactor the existing CSV modifier.

This change adds a discoverModifier to the internal/modifier package and
refactors the CSV modifier to use this abstraction.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-07-01 14:25:19 +02:00
Evan Lezar
7ca0e5db60 Update NVIDIA Container Runtime readme
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-06-03 13:10:21 +02:00
Evan Lezar
c77e86137e Add version output to CLIs
This change adds version output to the nvidia-continer-runtime,
nvidia-container-toolkit, and nvidia-ctk CLIs. The same version
is used in all cases and includes a version string and a git
revision if set.

The construction of the version string mirrors what is done in runc.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-13 07:31:11 +02:00
Evan Lezar
60dacb76b6 Call logger.Reset() to ensure errors are captured
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-12 15:42:42 +02:00
Evan Lezar
19138a2110 Skip setting of log file for --version flag
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-12 14:34:37 +02:00
Evan Lezar
4c49f75365 Remove --force flag from nvidia-container-runtime-hook
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-12 10:53:52 +02:00
Evan Lezar
e591f3f26b Replace experimental and discover-mode
These changes replace the nvidia-container-runtime config options
experimental and discover-mode with a single mode config option.

Note that mode is now a string with a default value of "auto"
and a mode value of "legacy" is equivalent to experimental == false.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-12 10:53:50 +02:00
Evan Lezar
e0ad82e467 Move ResolveAutoMode to info package
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-12 10:28:56 +02:00
Evan Lezar
3a1404f2f4 Move isTegraSystem to internal info package
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-12 10:28:56 +02:00
Evan Lezar
ae57a2fc93 Merge branch 'CNT-2875/create-specific-symlinks' into 'main'
Create specific symlinks for CSV mode

See merge request nvidia/container-toolkit/container-toolkit!150
2022-05-12 05:27:43 +00:00
Evan Lezar
1eb0e3c8b3 Merge branch 'fix-executable-locator' into 'main'
Fix location of executables in PATH

See merge request nvidia/container-toolkit/container-toolkit!148
2022-05-12 05:26:22 +00:00
Evan Lezar
a524c44161 Merge branch 'CNT-2926/runc-logging' into 'main'
Support runc logging command line options

See merge request nvidia/container-toolkit/container-toolkit!144
2022-05-12 05:24:25 +00:00
Evan Lezar
675fbace01 Add hook to create specific links
This change updates the create-symlinks hook to also create symlinks for
libcuda.so, libGLX_indirect.so.0, and libnvidia-opticalflow.so

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-11 16:36:49 +02:00
Evan Lezar
eac326c5ea Add --link option to nvidia-ctk hook create-symlinks command
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-11 15:28:53 +02:00
Evan Lezar
b0f7a3809f Factor linkCreation into method
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-11 15:28:02 +02:00
Evan Lezar
126c004ee0 Improve symlink creation loop
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-11 15:17:15 +02:00
Evan Lezar
d2516cb5d5 Merge branch 'fix-container-root' into 'main'
Fix bug in update-ldcache hook when OCI spec contains a relative root

See merge request nvidia/container-toolkit/container-toolkit!147
2022-05-10 22:01:14 +00:00
Evan Lezar
4696d7ee69 Merge branch 'fix-hook-flags' into 'main'
Use singular instead of plural for hook arguments

See merge request nvidia/container-toolkit/container-toolkit!146
2022-05-10 22:00:51 +00:00
Evan Lezar
ef6f48e9f7 Use singular instead of plural for hook arguments
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-10 19:55:31 +02:00
Evan Lezar
b8ef6be6ea Use lookup.GetPath from runtime hook
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-10 14:53:26 +02:00
Evan Lezar
55efdc8765 Use state.GetContainerRoot in nvidia-ctk hook subcommands
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-10 11:48:43 +02:00
Evan Lezar
e9d929dc2f Support runc logging command line options
This change processes and supports runc logging command line arguments.
This allows for better integration into container engines such as
docker.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-09 19:32:38 +02:00
Evan Lezar
c76946cbcc Add nvidia-container-runtime.runtimes config option
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-09 09:38:12 +02:00
Evan Lezar
70c4588197 Add compute capability of first device as arch property
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-05 14:11:30 +02:00
Evan Lezar
75ce057878 Add debug log for command line arguments
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-05 13:47:39 +02:00
Evan Lezar
9d2363e12e Return low-level runtime if subcommand is not create
This also removes a test that invokes nvidia-container-runtime run --bundle
expecting an error. This test is no longer valid since this command line
is forwared to runc where the error should be detected.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-05 13:43:13 +02:00
Evan Lezar
49f4bb3198 Check requirements before creating CSV discoverer
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-05 13:43:13 +02:00
Evan Lezar
5d7b3a4a96 Return raw spec from Spec.Load
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-05 13:43:13 +02:00
Evan Lezar
50cf07e4cd Use CUDA image abstraction for runtime hook
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-05-05 13:43:13 +02:00
Evan Lezar
bdf997c761 Use semver package to parse CUDA version
This avoids the use of scanf on a user-provided string which is flagged
as a security vulnerability.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-22 14:57:52 +02:00
Evan Lezar
c224832a6d Add log-level config option for nvidia-container-runtime
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 13:56:17 +02:00
Evan Lezar
5211960fc3 Merge branch 'detect-gpus-flag' into 'master'
Detect use of --gpus flag in experimental mode

See merge request nvidia/container-toolkit/container-toolkit!125
2022-04-08 11:18:11 +00:00
Evan Lezar
45160b88a4 Remove exsiting NVIDIA Container Runtime Hooks from the spec
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 12:03:22 +02:00
Evan Lezar
a9a4704273 Raise error if hook invoked in experimental mode without force flag
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 12:03:22 +02:00
Evan Lezar
7ec3cd0b5b Fix creation of CSV parser in create-symlinks
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 11:39:18 +02:00
Evan Lezar
ab7f25500f Fix creation of CSV parser in create-symlinks
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 11:36:48 +02:00
Evan Lezar
196d5c5461 Move NVIDIA Container Runtime Hook executable name to shared constant
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 11:29:27 +02:00
Evan Lezar
f07d110e85 Use DefaultExecutableDir to determine default paths
This change adds a DefaultExecutableDir = /usr/bin constant that is used
to construct default paths for executables instead of specifying these
explicitly.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 11:28:03 +02:00
Evan Lezar
1ebd48dea6 Merge branch 'add-symlink-hook' into 'master'
Add hook create-symlinks subcommand to create symlinks in container

See merge request nvidia/container-toolkit/container-toolkit!121
2022-04-08 09:14:07 +00:00
Evan Lezar
f7c74d35cc Merge branch 'add-hooks-cli' into 'master'
Add nvidia-ctk CLI with hook command and update-ldcache subcommand to update LD cache

See merge request nvidia/container-toolkit/container-toolkit!115
2022-04-08 09:13:39 +00:00
Evan Lezar
67602b28f9 Return unmodified runtime if specModifier is nil
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-08 07:50:40 +02:00
Evan Lezar
907736b053 Inject symlinks hook for creating symlinks in a container
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 20:25:55 +02:00
Evan Lezar
ecb4ef495a Add create-symlinks subcommand to create symlinks in container for specified CSV files
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 20:25:55 +02:00
Evan Lezar
95797a8252 Move reading of container state for internal/oci package
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 20:25:55 +02:00
Evan Lezar
c87ae586d4 FIX: Rename containerSpec flag to container-spec
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 20:25:19 +02:00
Evan Lezar
d970d0a627 Add discovery for ldconfig hook that updates the LDCache
This change adds a discovered hook for updating the ldcache as a container-create
hook. The mounts from a discoverer are inspected to determine the folders that must
be added to the cache using the nvidia-ctk hook update-ldcache command.

This is added to the "csv" discovery mode for the experimental runtime.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 20:25:19 +02:00
Evan Lezar
1c892af215 Add hook command to nvidia-ctk with update-ldcache subcommand
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 16:38:01 +02:00
Evan Lezar
c945cc714d Add stub nvidia-ctk CLI
This change adds an nvidia-ctk CLI that is used as the basis for
utilities related to the NVIDIA Container Toolkit.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 16:32:25 +02:00
Evan Lezar
bb086d4b44 Add auto discover mode and use this as the default
This change adds an 'auto' discover mode that attempts to select the correct mode
for a given platform. This currently attempts to detect whether the platform is a
Tegra-based system in which case the 'csv' discover mode is used. The 'legacy'
discover mode is used as the fallback.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 15:37:03 +02:00
Evan Lezar
26d2873bb2 FIX: Rename DefaultRoot to DefaultMountSpecPath
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 14:11:52 +02:00
Evan Lezar
6682bc90b4 Add support for NVIDIA_REQUIRE_JETPACK envvar
This change ensures that by default, the CSV discovery only considers the base CSV
files (l4t.csv, drivers.csv, devices.csv) and skips the rest unless the
NVIDIA_REQUIRE_JETPACK is set to "csv-mounts=all", in which case, all CSV files in the
specified folder are considered.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 10:46:26 +02:00
Evan Lezar
1c05a463bd Add csv discovery mode to experimental runtime
This change adds support for a "csv" discovery mode to the experimental runtime.
If this is set with experimental = true, a CSV-based discovery of devices and
mounts are used to define the modifications required to the OCI spec. The edits
are expressed as CDI ContainerEdits.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-07 10:45:19 +02:00
Evan Lezar
9ce690093d FIX: Make isNVIDIAContainerRuntimeHook mode idiomatic
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-06 17:18:06 +02:00
Evan Lezar
b8dd473343 FIX: Simplify hook remover
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-06 17:15:57 +02:00
Evan Lezar
11aa1d2a7d FIX: Factor out specModifier construction into function
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-06 15:18:12 +02:00
Evan Lezar
e6730fd0f0 FIX: Don't log that hooks is being removed if it is not
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-06 15:13:32 +02:00
Evan Lezar
8db287af8b FIX: Fix typo in comment
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-06 14:46:27 +02:00
Evan Lezar
282a2c145e Fix typo in variable name
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:26 +02:00
Evan Lezar
d0608844dc Add basic README for nvidia-container-runtime
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:26 +02:00
Evan Lezar
a26d02890f Make error logging less verbose by default
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:26 +02:00
Evan Lezar
14fe35c3f4 Implement hook remover for existing nvidia-container-runtime-hooks
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:26 +02:00
Evan Lezar
d12dbd1bef Read top-level config to propagate Root to experimental runtime
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:25 +02:00
Evan Lezar
239b6d3739 Implement experimental modifier for NVIDIA Container Runtime
This change enables the experimental mode of the NVIDIA Container Runtime. If
enabled, the nvidia-container-runtime.discover-mode config option is
queried to determine how required OCI spec modifications should be defined.
If "legacy" is selected, the existing NVIDIA Container Runtime hooks is
discovered and injected into the OCI spec.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:25 +02:00
Evan Lezar
7137f4b05b Move runtime config to internal package
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:24 +02:00
Evan Lezar
0c7eb93d62 Add experimental option to NVIDIA Container Runtime config
This change adds an experimental option to the NVIDIA Container Runtime config. To
simplify the extension of this experimental mode in future an error is raised if
this is enabled.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-04-04 14:16:24 +02:00
Evan Lezar
c6dfc1027d Move modifier code for inserting nvidia-container-runtime-hook to separate package
This change moves the code defining the insertion of the nvidia-container-runtime
hook to a separate package. This allows for better distinction between the existing
and experimental modifications.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-03-29 11:05:42 +02:00
Evan Lezar
4177fddcc4 Import modifying runtime abstraction from experimental runtime
This change imports the modifying runtime abstraction from the
experimental branch. This encapsulates the checks for whether
modification is required, and forwards the loaded spec to
the specified modifier. This allows for the same code to be
reused when performing more complex modifications.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-03-29 11:05:42 +02:00
Evan Lezar
bf8c3bab72 Add test package with GetModuleRoot and PrependToPath function
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-03-29 11:05:41 +02:00
Evan Lezar
c5c2ffd68f Ensure that Exec error is also logged to file
This change removes unneeded logging and renames the return error value to rerr
to avoid it being aliased by local error values.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2022-03-29 11:05:41 +02:00
Evan Lezar
0a2db7c70e Add nvidia-container-config option to overide drivercapabilities
This change adds support for a supported-driver-capabilities config
option in the config.toml file that allows the driver capabilities
associated with the NVIDIA_DRIVER_CAPABILITIES=all environment variable.
This can be used on platforms such as Jetson to remove unsupported
capabilities such as "ngx".

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-11-17 16:53:08 +01:00
Evan Lezar
ec8a6d978d Import cmd/nvidia-container-runtime from experimental branch
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-10-29 16:49:05 +02:00
Evan Lezar
5996379fcc Add changelog entry for config.json path changes
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-10-26 14:29:28 +02:00
wenjun gao
350c8893fb skip error when bundleDir not exist 2021-10-26 17:51:12 +08:00
Evan Lezar
10cd42273e Update package references
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-09-07 13:13:03 +02:00
Evan Lezar
b6a585c77d Copy code from nvidia-container-runtime
This change copies the cmd/nvidia-container-runtime, internal, and test
folders from github.com/NVIDIA/nvidia-container-runtime@8a63b4b34f3ce3b4167f0516aa3f7207ca280dfb

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-09-07 13:13:03 +02:00
Evan Lezar
c2ac6db43b Revert "Add support for NVIDIA_FABRIC_DEVICES"
This reverts commit f828efcf64.
2021-08-18 15:17:59 +02:00
Evan Lezar
f828efcf64 Add support for NVIDIA_FABRIC_DEVICES
This change adds support for the NVIDIA_FABRIC_DEVICES envvar. The (non-empty)
value of this envvar is passed to the NVIDIA Container CLI using the --fabric-device
command line flag and allows for nvswitch and nvlink devices to be mounted
into the container.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-08-11 10:33:55 +02:00
Evan Lezar
f72b79cc2a Move pkg to cmd/nvidia-container-toolkit
This change moves the pkg folder to `cmd/nvidia-container-toolkit` to
better match go best practices. This allows, for example, for the
`cmd/nvidia-container-toolkit` to be go installed.

The only package included in `pkg` was `main`.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2021-06-08 15:20:59 +02:00