Blackwell GPUs Are Now Available at Your Desk, with Sama Bali and Logan Lawler

Today's charming and complementary guests — Sama Bali from NVIDIA and Logan Lawler from Dell — make for an extra fun episode on the powerful new Blackwell GPUs... now available at your desk!

More on Sama:

• A.I. Solutions leader at NVIDIA that specializes in bringing A.I. products to market.

• Prior to NVIDIA, held a Machine Learning Solutions role at Amazon Web Services (AWS).

• Focused on educating data scientists and developers on A.I. innovations and implementing them effectively in enterprises.

• Holds a Masters in Engineering Management from San José State University.

More on Logan:

• Leads Dell Pro Max A.I. Solutions (if you haven’t heard of Pro Max before, we’ll cover that in this episode!)

• Over his sixteen-year tenure at Dell Technologies, has held positions across merchandising, services, marketing and e-commerce.

• Holds an MBA in management from Texas State University.

Today’s episode will be particularly appealing to hands-on data science, machine learning and A.I. practitioners but it isn’t especially technical and so can be enjoyed by anyone!

In today’s episode, Sama and Logan detail:

• Why data scientists are camping out at 6AM to attend NVIDIA's GTC conference.

• The killer specs of NVIDIA’s next-generation Blackwell GPUs.

• How Dell and Nvidia have joined forces to bring server-level AI power right to your desktop.

• How microservices are revolutionizing A.I. development and deployment.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Beyond GPUs: The Power of Custom AI Accelerators, with Emily Webber

The mind-blowing A.I. capabilities of recent years are made possible by vast quantities of specialized A.I.-accelerator chips. Today, AWS's (brilliant, amusing and Zen!) Emily Webber explains how these chips work.

Emily:

• Is a Principal Solutions Architect in the elite Annapurna Labs ML service team that is part of Amazon Web Services (AWS).

• Works directly on the Trainium and Inferentia hardware accelerators (for, respectively, training and making inferences with A.I. models).

• Also works on the NKI (Neuron Kernel Interface) that acts as a bare-metal language and compiler for programming AWS instances that use Trainium and Inferentia chips.

• Wrote a book on pretraining foundation models.

• Spent six years developing distributed systems for customers on Amazon’s cloud-based ML platform SageMaker.

• Leads the Neuron Data Science community and leads the technical aspects for the “Build On Trainium” program — a $110m credit-investment program for academic researchers.

Today’s episode is on the technical side and will appeal to anyone who’s keen to understand the relationship between today’s gigantic A.I. models and the hardware they run on.

In today’s episode, Emily details:

• The little-known story of how Annapurna Labs revolutionized cloud computing.

• What it takes to design hardware that can efficiently train and deploy models with billions of parameters.

• How Tranium2 became the most powerful A.I. chip on AWS.

• Why AWS is investing $110 million worth of compute credits in academic AI research.

• How meditation and Buddhist practice can enhance your focus and problem-solving abilities in tech.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

The Neural Processing Units Bringing AI to PCs, with Shirish Gupta

In many situations, it's impractical (or even impossible!) to have A.I. executed in the cloud. In today's episode, Shirish Gupta details when to run A.I. locally and how Neural Processing Units (NPUs) make it practical.

Today's episode is about efficiently designing and deploying AI applications that run on the edge. Our guide on that journey is SuperDataScience Podcast fan, Shirish! Here's more on him:

• Has spent more than two decades working for the global technology juggernaut, Dell Technologies, in their Austin, Texas headquarters.

• Has held senior systems engineering, quality engineering and field engineering roles.

• For the past three years, has been Director of AI Product Management for Dell’s PC Group.

• Holds a Master’s in Mechanical Engineering from the University of Maryland.

Today’s episode should appeal to anyone who is involved with or interested in real-world A.I. applications.

In this episode, Shirish details:

• What Neural Processing Units (NPUs) are and why they're transforming A.I. on edge devices.

• Four clear, compelling reasons to consider moving AI workloads from the cloud to your local device.

• The "A.I. PC" revolution that's bringing A.I. acceleration to everyday laptops and workstations.

• What kinds of Large Language Models are best-suited to local inference on AI PCs.

• How Dell's Pro A.I. Studio toolkit will drastically reduce enterprise A.I. deployment time.

• Plenty of real-life A.I. PC examples, including how a healthcare provider achieved physician-level accuracy with a custom vision model.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

NoSQL Is Ideal for AI Applications, with MongoDB’s Richmond Alake

In today's episode (#871), I'm joined by the gifted writer, speaker and ML developer Richmond Alake, who details what NoSQL databases are and why they're ideally suited for A.I. applications.

Richmond:

• Is Staff Developer Advocate for AI and Machine Learning at MongoDB, a huge publicly-listed database company with over 5000 employees and over a billion dollars in annual revenue.

• With Andrew Ng, he co-developed the DeepLearning.AI course “Prompt Compression and Query Optimization” that has been undertaken by over 13,000 people since its release last year.

• Has delivered his courses on Coursera, DataCamp, and O'Reilly.

• Authored 200+ technical articles with over a million total views, including as a writer for NVIDIA.

• Previously held roles as an ML Architect, Computer Vision Engineer and Web Developer at a range of London-based companies.

• Holds a Master’s in computer vision, machine learning and robotics from The University of Surrey in the UK.

Today's episode (filmed in-person at MongoDB's London HQ!) will appeal most to hands-on practitioners like data scientists, ML engineers and software developers, but Richmond does a stellar job of introducing technical concepts so any interested listener should enjoy the episode.

In today’s episode, Richmond details:

• How NoSQL databases like MongoDB differ from relational, SQL-style databases.

• Why NoSQL databases like MongoDB are particularly well-suited for developing modern A.I. applications, including Agentic A.I. applications.

• How Mongo incorporates a native vector database, making it particularly well-suited to RAG (retrieval-augmented generation).

• Why 2025 marks the beginning of the "multi-era" that will transform how we build A.I. systems.

• His powerful framework for building winning A.I. strategies in today's hyper-competitive landscape.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

AI Should Make Humans Wiser (But It Isn’t), with Varun Godbole

Today's trippy, brain-stimulating episode features Varun Godbole, a former Google Gemini LLM researcher who’s turned his attention to the future implications of the crazy-fast-moving exponential moment we're in.

Varun:

• Spent the past decade doing Deep Learning research at Google, across pure and applied research projects.

• For example, he was co-first author of a Nature paper where a neural network beat expert radiologists at detecting tumors.

• Also co-authored the Deep Learning Tuning Playbook (that has nearly 30,000 stars on GitHub!) and, more recently, the LLM Prompt Tuning Playbook.

• He's worked on engineering LLMs so that they generate code and most recently spent a few years as a core member of the Gemini team at Google.

• Holds a degree in Computer Science as well as in Electrical and Electronic Engineering from The University of Western Australia.

Varun mostly keeps today’s episode high-level so it should appeal to anyone who, like me, is trying to wrap their head around how vastly different society could be in a few years or decades as a result of abundant intelligence.

In today’s episode, Varun details:

• How human relationship therapy has helped him master A.I. prompt engineering.

• Why focusing on A.I. agents so much today might be the wrong approach — and what we should focus on instead.

• How the commoditization of knowledge could make wisdom the key differentiator in tomorrow's economy.

• Why the future may belong to "full-stack employees" rather than traditional specialized roles.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

BAML: The Programming Language for AI, with Vaibhav Gupta

Today's guest, Vaibhav Gupta, has developed BAML, the programming language for AI. If you are calling LLMs, you've gotta check BAML out for instant accuracy improvements and big (20-30%) cost savings.

More on charming and terrifically brilliant Vaibhav:

• Founder & CEO of Boundary (YC W23), a Y Combinator-backed startup that has developed a new programming language (BAML) that makes working with LLMs easier and more efficient for developers.

• Across his decade of experience as a software engineer, he built predictive pipelines and real-time computer vision solutions at Google, Microsoft and the renowned hedge fund The D. E. Shaw Group.

• Holds a degree in Computer Science and Electrical Engineering from The University of Texas at Austin.

This is a relatively technical episode. The majority of it will appeal to folks who interact with LLMs or other model APIs hands-on with code.

In today’s information-dense episode, Vaibhav details:

• How his company pivoted 13 times before settling upon developing a programming language for A.I.

• Why creating a programming language was "really dumb" but why it’s turning out to be brilliant, including by BAML already saving companies 20-30% on their AI costs.

• Fascinating parallels between today's A.I. tools and the early days of web development.

• His unconventional hiring process (I’ve never heard of anything remotely close to it) and the psychology behind why it works.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

AI Engineering 101, with Ed Donner

My holiday gift to you is my Nebula.io co-founder Ed Donner, one of the most brilliant, articulate people I know. In today's episode, Ed introduces the exciting, in-demand "A.I. Engineer" career — what's involved and how to become one.

After working daily alongside this world-class mind and exceptional communicator for nearly a decade, it is at long last my great pleasure to have the extraordinary Ed as my podcast guest. Ed:

• Is co-founder and CTO of Nebula, a platform that leverages generative and encoding A.I. models to source, understand, engage and manage talent.

• Previously, was co-founder and CEO of an A.I. startup called untapt that was acquired in 2020.

• Prior to becoming a tech entrepreneur, Ed had a 15-year stint leading technology teams on Wall Street, at the end of which he was a Managing Director at JPMorganChase, leading a team of 300 software engineers.

• He holds a Master’s in Physics from the University of Oxford.

Today’s episode will appeal most to hands-on practitioners, particularly those interested in becoming an A.I. Engineer or leveling up their command of A.I. Engineering skills.

In today’s episode, Ed details:

• What an A.I. Engineer (also known as an LLM Engineer) is.

• How the data indicate A.I. Engineers are in as much demand today as Data Scientists.

• What an A.I. Engineer actually does, day to day.

• How A.I. Engineers decide which LLMs to work with for a given task, including considerations like open- vs closed-source, what model size to select and what leaderboards to follow.

• Tools for efficiently training and deploying LLMs.

• LLM-related techniques including RAG and Agentic A.I.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Andrew Ng on AI Vision, Agents and Business Value

My guest today is the inimitable Andrew Ng! In his trademark, clear-spoken style, Andrew gives us a glimpse of the Agentic A.I. future, particularly how the coming Vision Agent tsunami will change the world.

I suspect pretty much everyone knows Dr. Ng already, but just in case:

• As director of Stanford University's AI Lab, his research group played a key role in the development of deep learning (which led to him to founding the influential Google Brain team) as well as educating millions on machine learning (and leading to him co-founding Coursera).

• Is Managing General Partner of AI Fund, a world-leading A.I. venture studio.

• Was CEO (is now Executive Chairman) of LandingAI, a computer-vision platform that specializes in domain-specific Large Vision Models (analogous to LLMs for language).

• Founded DeepLearning.AI, which provides excellent technical training on ML, deep learning (of course!), generative A.I. and many other associated subjects.

• Was co-CEO (as well as co-founder and chairman) of Coursera, which brought online learning from 300 leading universities to over 100 million students.

This episode was recorded live at the ScaleUp:AI conference in New York a few weeks ago. Thanks to George Mathew and Jennifer Jordan for inviting me back to the conference to interview Andrew :)

In today’s, Andrew details:

• Why a cheaper A.I. model with smart agentic A.I. workflow might outperform more expensive, more advanced models.

• The surprising truth about A.I. API costs that most businesses don't realize.• How Marvin Minsky's "Society of Mind" theory from the 1980s is making an unexpected comeback in modern A.I.

• A groundbreaking new way to process visual data that goes beyond traditional computer vision.

• Why unstructured data will be the key to A.I.'s next big revolution.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

In Case You Missed It in April 2024

Other than excessive maleness and paleness*, April 2024 was an excellent month for the podcast, packed with outstanding guests. ICYMI, today's episode highlights the most fascinating moments of my convos with them.

Specifically, conversation highlights include:

1. Iconic open-source developer Dr. Hadley Wickham putting the "R vs Python" argument to bed.

2. Aleksa Gordić, creator of a digital A.I.-learning community of 160k+ people, on the movement from formal to self-directed education.

3. World-leading futurist Bernard Marr on how we can work with A.I. as opposed to it lording over of us.

4. Educator of millions of data scientists, Kirill Eremenko, on why gradient boosting is so powerful for making informed business decisions.

5. Prof. Barrett Thomas on how drones could transform same-day delivery.

*Remedied in May!

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

NLP with ChatGPT (and other LLMs)

Over 1400 people registered for yesterday's "NLP with ChatGPT (and other LLMs)" conference that I hosted in the O'Reilly Media platform. Kudos to speakers Sinan, Melanie and Shaan for making it a smashing success 🎉

This screenshot is a taste of what it looked like from inside the broadcasting platform, captained flawlessly by producers Joan Baker and Nurul Ishak, PMP.

The presenters each spent 30 minutes presenting on their topics and then engaged in riveting Q&A with the highly engaged attendees:
• Sinan Ozdemir: The A.I. entrepreneur and author introduced the theory behind Transformer Architectures and LLMs like BERT, T5, and GPT.
• Melanie Subbiah: A first author on the original GPT-3 paper, she led interactive demos of the broad range of capabilities of LLMs like ChatGPT.
• Shaan Khosla: A data scientist on my team at Nebula.io, he detailed practical tips on training, validating, and productionizing LLMs hands-on in Python.

I've heard word that, unusually for a live event in O'Reilly, the footage of this conference will be made available as a video within the platform. Stay tuned for details!