Reinforcement Learning from Human Feedback (RLHF), with Dr. Nathan Lambert

In today's episode, the renowned RLHF thought-leader Dr. Nathan Lambert digs into the origins of RLHF, its role today in fine-tuning LLMs, emerging alternatives to RLHF... and how GenAI may democratize (human) education!

Nathan:

• Is a Research Scientist at the Allen Institute for AI (AI2) in Seattle, where he’s focused on fine-tuning Large Language Models (LLMs) based on human preferences as well as advocating for open-source AI.

• He’s renowned for his technical newsletter on AI called "Interconnects".

• Previously helped build an RLHF (reinforcement learning from human feedback) research team at Hugging Face.

• Holds a PhD from University of California, Berkeley in which he focused on reinforcement learning and robotics, and during which he worked at both Meta AI and Google DeepMind.

Today’s episode will probably appeal most to hands-on practitioners like data scientists and machine learning engineers, but anyone who’d like to hear from a talented communicator who works at the cutting edge of AI research may learn a lot by tuning in.

In today’s episode, Nathan details:

• What RLHF is and how its roots can be traced back to ancient philosophy and modern economics.

• Why RLHF is the most popular technique for fine-tuning LLMs.

• Powerful alternatives to RLHF such as RLAIF (reinforcement learning from A.I. feedback) and direct distilled preference optimization (dDPO).

• Limitations of RLHF.

• Why he considers AI to often be more alchemy than science.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Deep Reinforcement Learning — with Wah Loon Keng

For an intro to Deep Reinforcement Learning or to hear about the latest research and applications in the field (which is responsible for the most cutting-edge "A.I."), today's episode with Wah Loon Keng is for you.

Keng:
• Co-authored the exceptional book "Foundations of Deep Reinforcement Learning" alongside Laura Graesser.
• Co-created SLM-Lab, an open-source deep reinforcement learning framework written with the Python PyTorch library.
• Is a Senior A.I. Engineer at AppLovin, a marketing solutions provider.

In this episode, Keng details:
• What reinforcement learning is.
• A timeline of major breakthroughs in the history of Reinforcement Learning, including when and how Deep RL evolved.
• Modern industrial applications of Deep RL across robotics, logistics, and climate change.
• Limitations of Deep RL and how future research may overcome these limitations.
• The industrial robotics and A.I. applications Deep RL could play a leading role in in the coming decades.
• What it means to be an A.I. engineer and the software tools he uses daily in that role.

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Intro to Deep Reinforcement Learning at Columbia University

My goodness did I ever miss lecturing in-person! Was finally back in front of a live classroom on Friday, providing an intro to Deep Reinforcement Learning to engineering graduates at Columbia University in the City of New York.

Thank you Chong and Sam for having me. It's always a delight to lecture to your brilliant ELEN E6885 students, especially now that the pandemic is subsiding and I can interact with them meaningfully again.

You can see the slides from here as well as the associated GitHub repository here.

Deep Reinforcement Learning

Five-Minute Friday today is an intro to (deep) reinforcement learning, which has diverse cutting-edge applications: E.g., machines defeating humans at complex strategic games and robotic hands solving Rubik’s cubes.

You can watch or listen here.

AI Recruitment Technology & Deep Learning - Guest Appearance on the Engineered-Mind Podcast

Thanks to Jousef Murad for having me on the popular Engineered-Mind podcast.! Jousef had deeply insightful questions and I enjoyed the experience immensely :)

I spoke with Jousef back in April 2021, where we discussed:

- untapt and how AI powered recruiting works
- My background in neuroscience
- Where to get started when learning ML
- Tips for becoming a deep learning specialist
- What is I’m most excited about in terms of AI
- How I come up with the idea of writing a book

You can listen to the podcast anywhere podcasts are available including Apple Podcasts, Spotify, and Anchor.fm. You can also check out the video directly on YouTube here.