DeepEP/third-party

Install NVSHMEM

Important notices

This project is neither sponsored nor supported by NVIDIA.

Use of NVIDIA NVSHMEM is governed by the terms at NVSHMEM Software License Agreement.

Prerequisites

Hardware requirements:

• GPUs inside one node needs to be connected by NVLink
• GPUs across different nodes needs to be connected by RDMA devices, see GPUDirect RDMA Documentation
• InfiniBand GPUDirect Async (IBGDA) support, see IBGDA Overview
• For more detailed requirements, see NVSHMEM Hardware Specifications

Installation procedure

1. Acquiring NVSHMEM source code

Download NVSHMEM v3.2.5 from the NVIDIA NVSHMEM OPEN SOURCE PACKAGES.

2. Apply our custom patch

Navigate to your NVSHMEM source directory and apply our provided patch:

git apply /path/to/deep_ep/dir/third-party/nvshmem.patch

3. Configure NVIDIA driver (required by inter-node communication)

Enable IBGDA by modifying /etc/modprobe.d/nvidia.conf:

options nvidia NVreg_EnableStreamMemOPs=1 NVreg_RegistryDwords="PeerMappingOverride=1;"

Update kernel configuration:

sudo update-initramfs -u
sudo reboot

For more detailed configurations, please refer to the NVSHMEM Installation Guide.

4. Build and installation

DeepEP uses NVLink for intra-node communication and IBGDA for inter-node communication. All the other features are disabled to reduce the dependencies.

export CUDA_HOME=/path/to/cuda
# disable all features except IBGDA
export NVSHMEM_IBGDA_SUPPORT=1

export NVSHMEM_SHMEM_SUPPORT=0
export NVSHMEM_UCX_SUPPORT=0
export NVSHMEM_USE_NCCL=0
export NVSHMEM_PMIX_SUPPORT=0
export NVSHMEM_TIMEOUT_DEVICE_POLLING=0
export NVSHMEM_USE_GDRCOPY=0
export NVSHMEM_IBRC_SUPPORT=0
export NVSHMEM_BUILD_TESTS=0
export NVSHMEM_BUILD_EXAMPLES=0
export NVSHMEM_MPI_SUPPORT=0
export NVSHMEM_BUILD_HYDRA_LAUNCHER=0
export NVSHMEM_BUILD_TXZ_PACKAGE=0
export NVSHMEM_TIMEOUT_DEVICE_POLLING=0

cmake -G Ninja -S . -B build -DCMAKE_INSTALL_PREFIX=/path/to/your/dir/to/install
cmake --build build/ --target install

Post-installation configuration

Set environment variables in your shell configuration:

export NVSHMEM_DIR=/path/to/your/dir/to/install  # Use for DeepEP installation
export LD_LIBRARY_PATH="${NVSHMEM_DIR}/lib:$LD_LIBRARY_PATH"
export PATH="${NVSHMEM_DIR}/bin:$PATH"

Verification

nvshmem-info -a # Should display details of nvshmem