Beyond GPUs: The Power of Custom AI Accelerators, with Emily Webber

The mind-blowing A.I. capabilities of recent years are made possible by vast quantities of specialized A.I.-accelerator chips. Today, AWS's (brilliant, amusing and Zen!) Emily Webber explains how these chips work.

Emily:

• Is a Principal Solutions Architect in the elite Annapurna Labs ML service team that is part of Amazon Web Services (AWS).

• Works directly on the Trainium and Inferentia hardware accelerators (for, respectively, training and making inferences with A.I. models).

• Also works on the NKI (Neuron Kernel Interface) that acts as a bare-metal language and compiler for programming AWS instances that use Trainium and Inferentia chips.

• Wrote a book on pretraining foundation models.

• Spent six years developing distributed systems for customers on Amazon’s cloud-based ML platform SageMaker.

• Leads the Neuron Data Science community and leads the technical aspects for the “Build On Trainium” program — a $110m credit-investment program for academic researchers.

Today’s episode is on the technical side and will appeal to anyone who’s keen to understand the relationship between today’s gigantic A.I. models and the hardware they run on.

In today’s episode, Emily details:

• The little-known story of how Annapurna Labs revolutionized cloud computing.

• What it takes to design hardware that can efficiently train and deploy models with billions of parameters.

• How Tranium2 became the most powerful A.I. chip on AWS.

• Why AWS is investing $110 million worth of compute credits in academic AI research.

• How meditation and Buddhist practice can enhance your focus and problem-solving abilities in tech.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Serverless, Parallel, and AI-Assisted: The Future of Data Science is Here, with Zerve’s Dr. Greg Michaelson

What are "code nodes" and "RAG DAGs"? Listen to today's episode with the highly technical (but also highly hilarious) Dr. Greg Michaelson to get a glimpse into the future of data science and A.I. model development.

Greg:

• Is a Co-Founder of Zerve AI, a super-cool platform for developing and delivering A.I. products that launched to the public on this very podcast a little over a year ago.

• Previously spent 7 years as DataRobot’s Chief Customer Officer and 4 years as Senior Director of Analytics & Research for Travelers.

• Was a baptist pastor while he obtained his PhD in Applied Statistics!

Today’s episode is on the technical side and so will appeal most to hands-on practitioners like data scientists, AI/ML engineers and software developers… but Greg is such an engaging communicator that anyone interested in how the practice of data science is rapidly being revolutionized may enjoy today’s episode.

In it, Greg details:

• How Zerve's collaborative, graph-based coding environment has matured over the past year, including their revolutionary 'Fleet' feature (in beta) that allows massive parallelization of code execution without additional cost.

• How AI assistants are changing the coding experience by helping build, edit, and connect your data science projects.

• Why the rise of LLMs might spell trouble for many SaaS businesses as building in-house solutions becomes increasingly viable.

• The innovative ways companies are using retrieval-augmented generation (RAG) to create more powerful A.I. applications.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

LLMs and Agents Are Overhyped, with Dr. Andriy Burkov

Andriy Burkov's ML books are mega-bestsellers and his newsletter has a wild 900,000 subscribers. He seldom does interviews so don't miss today's episode, in which he takes compelling, contrarian views on LLMs and agents.

More on Dr. Burkov:

• His indispensable "100-Page Machine Learning Book" seems to be on *every* data scientist / ML engineer's bookshelf.

• He also wrote "ML Engineering" and his latest book, "The 100-Page Language Model Book", was released this year to rave reviews.

• His "Artificial Intelligence" newsletter is subscribed to by 900,000 people on LinkedIn.

• He's the Machine Learning Lead at TalentNeuron, a global labor-market analytics provider.

• He runs his own book-publishing company, True Positive Inc.

• Previously held data science / ML roles at Gartner, Fujitsu and more.

• Holds a PhD in Computer Science (A.I.) from Université Laval in Quebec, where his doctoral dissertation focused on multi-agent decision-making — 15 years ago!

Despite Dr. Burkov being such a technical individual, most of today’s episode should appeal to anyone interested in A.I. (although some parts here and there will be particularly appealing to hands-on machine-learning practitioners).

In today’s episode, Andriy details:

• Why he believes AI agents are destined to fail.

• How he managed to create a chatbot that never hallucinates — by deliberately avoiding LLMs.

• Why he thinks DeepSeek AI crushed Bay Area A.I. leaders like OpenAI and Anthropic.

• What makes human intelligence unique from all other animals and why A.I. researchers need to crack this in order to attain human-level intelligence in machines.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Generative AI for Business, with Kirill Eremenko and Hadelin de Ponteves

Craving an intro to building and deploying commercially successful Generative A.I. applications? In today's episode, superstar data-science instructors Kirill and Hadelin (>5 million students between them) will fill you in!

Kirill Eremenko is one of our two guests today. He's:

• Founder and CEO of SuperDataScience, an e-learning platform.

• Founded the SuperDataScience Podcast in 2016 and hosted the show until he passed me the reins four years ago.

Our second guest is Hadelin de Ponteves:

• Was a data engineer at Google before becoming a content creator.

• In 2020, took a break from Data Science content to produce and star in a Bollywood film featuring "Miss Universe" Harnaaz Sandhu.

Together, Kirill and Hadelin:

• Have created dozens of data science courses; they are the most popular data science instructors on the Udemy platform, with over five million students between them!

• They also co-founded CloudWolf, an education platform for quickly mastering Amazon Web Services (AWS) certification.

• And, in today’s episode, they announce (for the first time anywhere!) another (brand-new) venture they co-founded together.

Today’s episode is intended for anyone who’s interested in real-world, commercial applications of Generative A.I. — a technical background is not required.

In today’s episode, Kirill and Hadelin detail:

• What generative A.I. models like Large Language Models are and how they fit within the broader category of “Foundation Models”.

• The 12 crucial factors to consider when selecting a foundation model for a given application in your organization.

• The 8 steps to ensuring foundation models are deployed commercially successfully.

• Many real-world examples of how companies are customizing A.I. models quickly and at remarkably low cost.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

AI Systems as Productivity Engines, with You.com’s Bryan McCann

Today, wildly intelligent Bryan McCann describes the Agentic A.I. behind his skyrocketing startup You.com and how it will lead to scientific discoveries human scientists couldn't dream of making. Don't miss this episode!

Bryan:

• Co-Founder and CTO of You.com, a prominent Bay Area A.I. startup that has raised $99m in venture capital (including a $50m Series B in September that valued the firm at nearly a billion dollars).

• Was previously Lead Research Scientist at Salesforce and an assistant on courses at Stanford such as Andrew Ng’s wildly popular machine learning course.

• Holds a Master’s in Computer Science, a Bachelor’s in Computer Science and a Bachelor’s in Philosophy, all from Stanford University.

Today’s episode should be fascinating to anyone interested in AI. In it, extremely well-spoken Bryan details:

• The philosophical underpinnings of the breakthroughs that led to the leading A.I. models we have today as well as the ones that will emerge in the coming years.

• How a coding mistake he made serendipitously revealed fundamental insights about meaning and language model alignment.

• Why he believes humanity is entering an existential crisis due to A.I., but nevertheless remains optimistic about the future.

• The fascinating connection between language models and biological proteins.

• Why A.I. systems might soon be able to make scientific discoveries humans could never dream of making.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

PyTorch Lightning, Lit-Serve and Lightning Studios, with Dr. Luca Antiga

Lightning AI makes tons of tools that speed A.I. model dev and deployment, including the wildly popular open-source library PyTorch Lightning. Today, hear from hands-on CTO Dr. Luca Antiga how all the magic happens ⚡️

More on Luca:

• CTO of Lightning AI, which (as one of world’s hottest startups developing A.I. tools) have raised over $80m in venture capital.

• Is also CTO of OROBIX, an A.I. services company that Luca co-founded 15 years ago.

• Holds a PhD in biomedical engineering from Politecnico di Milano… and did his postdoc at the Robarts Research Institute in London, Ontario (coincidentally around the same time I was doing brain-imaging research there).

Today’s episode will probably appeal most to hands-on practitioners like data scientists, software developers and ML engineers, but any tech-savvy professional could find it valuable.

In today’s episode, Luca details:

• How Lightning AI's suite of tools (in addition to PyTorch Lightning, this includes Lightning Studios, LitServe and the Thunder Compiler) is making A.I. development faster and easier.

• The rise of small language models and their potential to rival LLMs.

• His journey from biomedical imaging to deep learning pioneer.

• How software developer’s work will be transformed by A.I. in the coming years.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

PyTorch: From Zero to Hero, with Luka Anicin

Today's episode is on Python's most popular auto-differentiation library, PyTorch, and how you can use it to design, train and deploy deep neural nets, including LLMs. Acclaimed PyTorch instructor Luka Anicin is our guide.

Luka:

• Is one of Udemy’s all-time bestselling instructors on A.I.; over 500,000 students have taken his courses.

• His latest course, available exclusively at SuperDataScience.com, is called “PyTorch: From Zero to Hero”.

• CEO of full-lifecycle A.I. consultancy Datablooz.

• Holds a Bachelor’s in Computer Science, a Master’s in Data Science and is nearing completion of his PhD in Applied A.I.

Today’s episode will probably appeal most to hands-on practitioners like data scientists, software developers and ML engineers.

In it, Luka details:

• What the popular Python library PyTorch is for.

• Why you would select PyTorch over TensorFlow or Scikit-learn.

• The tensor building blocks PyTorch provides for designing, training and deploying state-of-the-art deep neural networks, including Large Language Models (LLMs).

• His top tips for accurate and efficient deep learning.

• Guidance on PyTorch portfolio projects.

• Real-world PyTorch case-studies from his experience leading an A.I. consultancy.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Agentic AI, with Shingai Manjengwa

Today's episode is all about Agentic A.I. — perhaps the hottest topic in A.I. today. Astoundingly intelligent and articulate Shingai Manjengwa couldn't be a better guide for us on this hot topic 🔥

Shingai:

• Head of A.I. Education at ChainML, a prestigious startup focused on developing tools for a future powered by A.I. agents.

• Founder and former CEO of Fireside Analytics Inc. (developed online data-science courses that have been undertaken by 500,000 unique students).

• Previously was Director of Technical Education at the prominent global A.I. research center, the Vector Institute in Toronto.

• Holds an MSc in Business Analytics from New York University.

Today’s episode should be equally appealing to hands-on practitioners like data scientists as to folks who generally yearn to stay abreast of the most cutting-edge A.I. techniques.

In today’s episode, Shingai details:

• What A.I. agents are.

• Why agents are the most exciting, fastest-growing A.I. application today.

• How LLMs relate to agentic A.I.

• Why multi-agent systems are particularly powerful.

• How blockchain technology enables humans to better understand and trust A.I. agents.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Merged LLMs Are Smaller And More Capable, with Arcee AI’s Mark McQuade and Charles Goddard

Today's episode is seriously mind-expanding. In it, Mark and Charles detail how they're pushing the A.I. frontier through LLM merging, extremely efficient (even CPU-only!) LLM training, and *Small* Language Models.

Mark McQuade:

• Is Co-Founder and CEO of Arcee.ai.

• Previously, he held client-facing roles at Hugging Face and Roboflow as well as leading the data science and engineering practice of a Rackspace company.

• He studied electronic engineering at Fleming College in Canada.

Charles Goddard:

• Is Chief of Frontier Research at Arcee.ai

• Previously, he was a software engineer at Apple and the famed NASA Jet Propulsion Laboratory.

• Studied engineering at Olin College in Massachusetts.

Today’s episode is relatively technical so will likely appeal most to hands-on practitioners like data scientists and ML engineers. In it, Charles and Mark detail:

• How their impressive open-source model-merging approach combines the capabilities of multiple LLMs without increasing the model’s size.

• A separate open-source approach for training LLMs efficiently by targeting specific modules of the network to train while freezing others.

• The pros and cons of Mixture-of-Experts versus Mixture-of-Agents approaches.

• How to enable small language models to outcompete the big foundation LLMs like GPT-4, Gemini and Claude.

• How to leverage open-source projects to land big enterprise contracts and attract big chunks of venture capital.

On that final note, congrats to the Arcee.ai team on announcing their $24m Series A round this very day... unsurprising given their tremendously innovative tech and rapid revenue ramp-up! It's very rare to see runaway A.I. startup successes like this one.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Deep Learning Classics and Trends, with Dr. Rosanne Liu

Today's guest is the amazing Google DeepMind research scientist, Dr. Rosanne Liu!

Rosanne:

• Is a Research Scientist at Google DeepMind in California.

• Is Co-Founder and Executive Director of ML Collective, a non-profit that provides global ML research training and mentorship.

• Was a founding member of Uber AI Labs, where she served as a Senior Research Scientist.

• She has published deep learning research in top academic venues such as NeurIPS, ICLR, ICML and Science, and her work has been covered in publications like WIRED and the MIT Tech Review.

• Holds a PhD in Computer Science from Northwestern University.

Today’s episode, particularly in the second half when we dig into Rosanne’s fascinating research, is relatively technical so will probably appeal most to hands-on practitioners like data scientists and ML engineers.

In today’s episode, Rosanne details:

• The problem she founded the ML Collective to solve.

• How her work on the “intrinsic dimension” of deep learning models inspired the now-standard LoRA approach to fine-tuning LLMs.

• The thorny problems with LLM evaluation benchmarks and how they might be solved.

• The pros and cons of curiosity- vs goal-driven ML research.

• The positive impacts of diversity, equity and inclusion in the ML community.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Reinforcement Learning from Human Feedback (RLHF), with Dr. Nathan Lambert

In today's episode, the renowned RLHF thought-leader Dr. Nathan Lambert digs into the origins of RLHF, its role today in fine-tuning LLMs, emerging alternatives to RLHF... and how GenAI may democratize (human) education!

Nathan:

• Is a Research Scientist at the Allen Institute for AI (AI2) in Seattle, where he’s focused on fine-tuning Large Language Models (LLMs) based on human preferences as well as advocating for open-source AI.

• He’s renowned for his technical newsletter on AI called "Interconnects".

• Previously helped build an RLHF (reinforcement learning from human feedback) research team at Hugging Face.

• Holds a PhD from University of California, Berkeley in which he focused on reinforcement learning and robotics, and during which he worked at both Meta AI and Google DeepMind.

Today’s episode will probably appeal most to hands-on practitioners like data scientists and machine learning engineers, but anyone who’d like to hear from a talented communicator who works at the cutting edge of AI research may learn a lot by tuning in.

In today’s episode, Nathan details:

• What RLHF is and how its roots can be traced back to ancient philosophy and modern economics.

• Why RLHF is the most popular technique for fine-tuning LLMs.

• Powerful alternatives to RLHF such as RLAIF (reinforcement learning from A.I. feedback) and direct distilled preference optimization (dDPO).

• Limitations of RLHF.

• Why he considers AI to often be more alchemy than science.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Math, Quantum ML and Language Embeddings, with Dr. Luis Serrano

Today, Dr. Luis Serrano (a master at making complex math and ML topics friendly) leads a mind-expanding discussion on embeddings in LLMs, Quantum ML and what the next big trends in A.I. will be. I wouldn't miss this one 🤯

Luis:

• Is the beloved creator behind the Serrano Academy, an educational YouTube channel on math and ML with over 146,000 subscribers.

• Until this month, he worked as Head of Developer Relations at Cohere, one of the world’s few A.I. labs that is actually at the frontier of LLMs.

• Prior to that, he was a Quantum A.I. Research Scientist at Zapata Computing, Lead A.I. Educator at Apple, Head of Content for A.I. at Udacity and ML Engineer at Google.

• Holds a PhD in Math from the University of Michigan.

Today’s episode should be appealing to just about anyone! In it, Luis details:

• How supposedly complex topics like math and A.I. can be made easy to understand.
• How Cohere’s focus on enterprise use cases for LLMs has led it to specialize in embeddings, the most important component of LLMs.
• The promising application areas for Quantum Machine Learning.
• What the next big trends in A.I. will be.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Aligning Large Language Models, with Sinan Ozdemir

For today’s quick Five-Minute Friday episode, the exceptional author, speaker and entrepreneur Sinan Ozdemir provides an overview of what it actually means for an LLM to be “aligned”.

More on Sinan:
• Is Founder and CTO of LoopGenius, a generative AI startup.
• Has authored several excellent books, including, most recently, the bestselling "Quick Start Guide to Large Language Models".
• Is a serial AI entrepreneur, including founding a Y Combinator-backed generative AI startup way back in 2015 that was later acquired.

This episode was filmed live at the Open Data Science Conference (ODSC) East in Boston last month. Thanks to ODSC for providing recording space.

The Super Data Science Podcast is available on all major podcasting platforms and a video version is on YouTube. This is episode #784!

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Generative AI in Practice, with Bernard Marr

In today's episode, Bernard Marr — world-leading futurist (>4m social-media followers) and prolific author (20+ books!) — details how GenAI will revolutionize industries, enhance our lives and solve pressing global issues.

In case he isn’t already on your radar, Bernard:

• World-leading futurist who’s consulted with NVIDIA, Google, Microsoft, Amazon and many more on digital transformation and A.I. in business.

• His 20+ books have been translated into 20+ languages and earned several business and management "book of the year" awards; many have also been bestsellers.

• His writing has been featured in The Guardian, Financial Times, The Wall Street Journal, the Harvard Business Review and many other leading media outlets.

• Has over 4 million combined social media followers.

Today’s episode will be of interest to anyone who’d like to better understand Generative A.I. and how to adopt GenAI effectively at work or at home.

In this episode, Bernard details:

• The history of GenAI.

• How GenAI will pair with other industries like energy, healthcare and education to accelerate hyper-innovation across every aspect of society.

• The regulatory and ethical challenges associated with GenAI and how we can overcome them.

• How AI paradoxically makes us more human.

• How to successfully implement GenAI both professionally and personally.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.