NCCL (NVIDIA Collective Comms)
The collective-communications library gluing multi-GPU training
NCCL is NVIDIA's collective-communications library that makes all-reduce/all-gather across thousands of GPUs performant; it is the invisible dependency of every large training run and reinforces the CUDA moat alongside cuDNN. Competitors must reimplement it (RCCL for AMD) to be viable.
Role
Multi-GPU all-reduce
AMD analog
RCCL
How it fits the stack
NCCL (NVIDIA Collective Comms) with what it depends on (above) and what it feeds (below). The figure renders as a crawlable diagram and upgrades to an interactive 3D graph as it scrolls into view.
NCCL (NVIDIA Collective Comms) in the AI stack. NCCL (NVIDIA Collective Comms) with its immediate upstream dependencies (top) and downstream dependents (bottom) in the AI value chain. Hover a node in 3D, or read the full relationships below.
Graph data (text) — 3 entities, 2 relationships
- NCCL (NVIDIA Collective Comms) —depends on→ CUDA / software moat
- Nvidia Data-Center GPU (Blackwell/Rubin) —depends on→ NCCL (NVIDIA Collective Comms)
Depends on ↑ · 1