RoCEv2 is a solution for achieving swift data throughput and minimal delay in modern data centers. It incorporates features like Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) to establish a lossless network environment. PFC manages data flow at the interface level, while ECN detects and mitigates congestion before PFC activation becomes necessary. The combination of ECN and PFC, known as Data Center Quantized Congestion Notification (DCQCN), optimizes congestion management in RDMA networks. Careful tuning of queue thresholds is crucial to prevent hot spots and ensure low Job Completion Times (JCTs). The use of ECN and PFC is necessary for maintaining a lossless fabric in GPU-to-GPU communication during AI/ML training runs.

Widely available GPU-accelerated servers, combined with better hardware and popular programming languages like Python and C/C++, along with frameworks such as PyTorch, TensorFlow and JAX, simplify the development of GPU-accelerated ML applications. These applications serve diverse purposes, from medical research to self-driving vehicles, relying on large datasets and GPU clusters for training deep neural networks. Inference frameworks apply knowledge from trained models to new data with optimized clusters for performance. The learning cycles involved in AI workloads can take days or weeks, and high-latency communication between server clusters can significantly impact completion times or result in failure. AI workloads demand low-latency, lossless networks, requiring appropriate hardware, software features, and configurations. This article will explain advanced queueing solutions used by all the major OEMs in the Network Operating Systems (NOS) that support ECN and PFC.