2.1 Why distributed training



PyTorch supports several different distributed training backends that perform the All-Reduce operation