Beyond GPUs: The Power of Custom AI Accelerators, with Emily Webber

The mind-blowing A.I. capabilities of recent years are made possible by vast quantities of specialized A.I.-accelerator chips. Today, AWS's (brilliant, amusing and Zen!) Emily Webber explains how these chips work.

Emily:

• Is a Principal Solutions Architect in the elite Annapurna Labs ML service team that is part of Amazon Web Services (AWS).

• Works directly on the Trainium and Inferentia hardware accelerators (for, respectively, training and making inferences with A.I. models).

• Also works on the NKI (Neuron Kernel Interface) that acts as a bare-metal language and compiler for programming AWS instances that use Trainium and Inferentia chips.

• Wrote a book on pretraining foundation models.

• Spent six years developing distributed systems for customers on Amazon’s cloud-based ML platform SageMaker.

• Leads the Neuron Data Science community and leads the technical aspects for the “Build On Trainium” program — a $110m credit-investment program for academic researchers.

Today’s episode is on the technical side and will appeal to anyone who’s keen to understand the relationship between today’s gigantic A.I. models and the hardware they run on.

In today’s episode, Emily details:

• The little-known story of how Annapurna Labs revolutionized cloud computing.

• What it takes to design hardware that can efficiently train and deploy models with billions of parameters.

• How Tranium2 became the most powerful A.I. chip on AWS.

• Why AWS is investing $110 million worth of compute credits in academic AI research.

• How meditation and Buddhist practice can enhance your focus and problem-solving abilities in tech.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.