As more data scientists use Python and other high-level languages, providing acceleration without code change is essential to rapidly improving development time. Networking Remote direct memory access (RDMA) in NVIDIA Mellanox® network interface controllers (NICs), NCCL2 (NVIDIA collective communication library), and OpenUCX (an open-source point-to-point communication framework) has led to tremendous improvements in training speed.