A Tour of PyTorch 2.0

Abstract: Learn about PyTorch 2.0 with a deep dive into the technology stack that powers the new torch.compile() API: TorchDynamo, AITAutograd, and TorchInductor. The new compiler stack reduces training times across a wide range of workloads while being fully backwards compatible. Bring your laptops, or connect to remote GPU powered systems and learn how to compile your code, profile and debug to improve performance.

Learning objectives

At the end of this tutorial you’ll be able to:

  1. Describe the advantages of using torch.compile()
  2. Demonstrate how to use torch.compile() to speed up PyTorch
  3. Explain what happens under the hood when you call torch.compile()
  4. List different Intermediate Representations (IR) in Pytorch
  5. Recognize the importance of operator fusion and code generation


Setup and getting started10 mins
What’s New in PyTorch 2.010 mins
Hand’s On: Simple benchmarks with torch.compile() in PyTorch 2.020 mins
Hand’s On: What happens under the hood when you call torch.compile() ?15 mins
Hand’s On: Profiling pytorch code with torch.profiler and inspecting the compiler process15 mins
Resources + Q&A10 mins