Chenggang Zhao
5a2e37fa28
Support statistics tensor for low-latency kernels ( #196 )
2025-06-09 15:50:56 +08:00
Chenggang Zhao
0d1a855d81
Add low-latency kernel PCIe usage flag ( #195 )
...
* Add low-latency kernel usage flag
* Update comments
2025-06-09 14:37:13 +08:00
Chenggang Zhao
564e375234
Fix < PTX ISA 8.6 compatibility ( #194 )
2025-06-09 10:48:42 +08:00
Chenggang Zhao
c8dceba110
Use TMA instead of LD/ST for intra-node normal kernels ( #191 )
...
* Update CMake files
* Use TMA instead of LD/ST for intranode dispatch
* Use TMA instead of LD/ST for intranode combine
* Adjust configs
* Test default configs as well
* More warps for combine
* Add inter-thread fence
* Enable more warps
* Do not use TMA for senders
* Update configs
* Remove useless wait
2025-06-06 15:40:17 +08:00
wzc.wuzhicheng
d0225df27d
Fix notify_dispatch: using warp 0 to issue send
...
Signed-off-by: wzc.wuzhicheng <wzc.wuzhicheng@linux.alibaba.com>
2025-06-03 20:20:02 +08:00
Shangyan Zhou
9fe9021f29
Use IBGDA only ( #177 )
2025-05-28 16:40:14 +08:00
Chenggang Zhao
92405ddf30
Code cleanup and bug fixed
2025-05-23 11:14:16 +08:00
cywork121
68ae8b3d07
Feature: LL nvlink p2p ( #173 )
2025-05-23 10:37:45 +08:00
sleepcoo
a107266a4e
support hidden size 4096
...
Co-authored-by: zhyncs <me@zhyncs.com>
Co-authored-by: yinfan98 <1106310035@qq.com>
2025-05-12 16:41:21 +08:00
wangfakang
63c29d06a0
To mitigate incast congestion, shuffle the starting index of target rank for different ranks and channels
...
Signed-off-by: wangfakang <fakangwang@gmail.com>
2025-05-10 09:55:35 +08:00
Shangyan Zhou
e255d57bef
Use put_nbi_warp.
2025-04-22 12:29:46 +08:00
Chenggang Zhao
edbb1bc3ff
Several code lints
2025-04-22 10:52:10 +08:00
Shangyan Zhou
3e54b78fd7
Normal kernels always use IBGDA mode.
2025-04-22 10:36:24 +08:00
Shangyan Zhou
20b2aaaf9e
Refactor some code.
2025-04-22 10:22:30 +08:00
Shangyan Zhou
e2c578485c
Revert ibgda_device.cuh and remove some comments.
2025-04-21 17:44:32 +08:00
moningchen
5ab80c28f3
In the Internode Normal Kernel, when using nvshmem ibrc for RDMA data transmission, a single QP is used for data transfer between two GPUs, which limits kernel performance in network card dual-port and RoCE network scenarios.
...
In our optimized Internode Normal Kernel, we implemented multiple QPs for data transmission between two GPUs, setting a different QP for each channel. Additionally, we modified the transmission method from IBRC to IBGAD.
Through these optimizations, the Internode Normal Kernel achieves optimal performance in both H800 and H20 environments, with RDMA transmission performance nearly reaching the physical network performance limit. Using the current default statistical method, in 4-node H800 and H20 environments, RDMA performance can reach 60GB/s+.
2025-04-21 15:50:39 +08:00
Chenggang Zhao
42494864ba
Remove useless control metadata for low-latency combine
2025-04-07 09:55:39 +08:00
Chenggang Zhao
c4d12b4f8f
Fix compilation
2025-03-28 16:45:10 +08:00
songhexiang
4dd1e68ac8
For the SMs which calculate metadata in notify_dispatch, each warp in the SM is used to calculate the metadata of one channel. The default configuration is 8 warps for 10 channels, which needs two rounds of loop. Maybe the number of warps can be configured to the number of the channels so that one loop is enough.
2025-03-28 06:43:29 +00:00
Chenggang Zhao
ffc39ba084
Stronger acquire scope for low-latency kernels
2025-03-27 09:30:36 +08:00
Chenggang Zhao
dcaf73e5ff
Support zero-copy for low-latency combine
2025-03-18 15:41:50 +08:00
Chenggang Zhao
82dcf48fd3
Fix bugs for intranode EP kernels
2025-03-14 16:09:23 +08:00
Shangyan Zhou
38cdaf390c
Fix style.
2025-03-14 11:22:00 +08:00
Shangyan Zhou
2d0cf41dd1
Low latency kernels use rdma atomic to support AR.
2025-03-14 11:04:57 +08:00
Chenggang Zhao
ed7487c15e
Support BF16 for low-latency kernels
2025-03-10 17:24:41 +08:00
Chenggang Zhao
1fc40d50f3
Improve AR performance
2025-03-06 21:41:19 +08:00
Chenggang Zhao
458cdcb22a
Fix AR bugs for normal kernels
2025-03-05 17:13:35 +08:00
Chenggang Zhao
680e424bdc
Bugs fixed
2025-03-05 14:27:45 +08:00
Chenggang Zhao
1553fc42bf
Improve EP2/4 performance
2025-03-04 15:34:33 +08:00
Chenggang Zhao
6cc3497df8
Remove all raw tensors for better P2P overlapping
2025-03-03 14:25:22 +08:00
Chenggang Zhao
77bb07aa20
Update some comments and docs
2025-02-27 10:27:22 +08:00
Chenggang Zhao
ebfe47e46f
Initial commit
2025-02-25 09:07:53 +08:00
