The Positron IDE, Tidy NLP and MLOps with Dr. Julia Silge

Prepare to have your brain tickled by Dr. Julia Silge. In today's episode, Julia details the IDE she's been developing for data scientists, "Tidy" NLP, and open-source libraries that make MLOps a breeze.

More on Julia:

• Engineering Manager at Posit PBC (makers of RStudio... and the company formerly known as RStudio).

• Authored the bestselling O'Reilly books “Text Mining with R” and “Tidy Modeling with R".

• Previously worked as a Data Scientist at Stack Overflow and Datassist.

• Prior to joining industry, was an academic researcher and professor at Yale University.

• Holds a PhD in Astronomy from The University of Texas at Austin.

Today’s episode will probably appeal most to hands-on practitioners like data scientists, software developers and ML engineers. In it, Julia details:

• The brand-new IDE Positron (free to use and source-available) that she’s been developing.

• Her favorite LLMs for code generation.

• The open-source software libraries that make MLOps easy.

• Her top tips for effective Natural Language Processing, including when more traditional NLP techniques should be used instead of an LLM.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

NLP with ChatGPT (and other LLMs)

Over 1400 people registered for yesterday's "NLP with ChatGPT (and other LLMs)" conference that I hosted in the O'Reilly Media platform. Kudos to speakers Sinan, Melanie and Shaan for making it a smashing success 🎉

This screenshot is a taste of what it looked like from inside the broadcasting platform, captained flawlessly by producers Joan Baker and Nurul Ishak, PMP.

The presenters each spent 30 minutes presenting on their topics and then engaged in riveting Q&A with the highly engaged attendees:
• Sinan Ozdemir: The A.I. entrepreneur and author introduced the theory behind Transformer Architectures and LLMs like BERT, T5, and GPT.
• Melanie Subbiah: A first author on the original GPT-3 paper, she led interactive demos of the broad range of capabilities of LLMs like ChatGPT.
• Shaan Khosla: A data scientist on my team at Nebula.io, he detailed practical tips on training, validating, and productionizing LLMs hands-on in Python.

I've heard word that, unusually for a live event in O'Reilly, the footage of this conference will be made available as a video within the platform. Stay tuned for details!

The State of Natural Language Processing

As the LaMDA "sentience" hubbub highlights, Natural Language Processing is perhaps the most exciting and rapidly accelerating area of Machine Learning. Hear all about NLP from the deep expert Rongyao HUANG.

(LaMDA is definitely not sentient, by the way... but it is an impressive display of state-of-the-art conversational machine capabilities.)

Rongyao:
• Is Lead Data Scientist at CB Insights, a marketing intelligence platform.
• Previously she worked as a data scientist at a number of other New York start-ups and as a quantitative research assistant at Columbia University.
• She holds a masters in research methodology and quantitative methods from Columbia University in the City of New York.

Today’s episode is more on the technical side so will appeal primarily to practicing data scientists, however the second half of the episode does contain general sage guidance for anyone seeking to navigate career options as well as to balance personal and professional obligations.

In today’s episode, Rongyao details:
• The evolution of NLP techniques over the past decade through to the large transformer models of today.
• The practical implications of this dramatic NLP evolution.
• How the “scaling law” will impact NLP model capabilities over the coming decade.
• The major limitations of today’s NLP approaches and how we might overcome them.
• Her Bauhaus-inspired model for effective data science.
• Her pathfinding model for making effective career choices.
• Her top tips for staying sane while juggling career and family.

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

DALL-E 2: Stunning Photorealism from Any Text Prompt

OpenAI just released their "DALL-E 2" multimodal model that defines "state of the art" A.I.: Provide it with (even extremely bizarre) natural-language requests for an image and it generates it! Hear about it in today's episode, and check out this interactive post from OpenAI that demonstrates DALL-E 2's mind-boggling capabilities.

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

PaLM: Google's Breakthrough Natural Language Model

This month, Google announced a large natural language model called PaLM that provides staggering results on tasks like common-sense reasoning and solving Python-coding questions. Hear all about it in today's episode!

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

GPT-3 for Natural Language Processing

With its human-level capacity on tasks as diverse as question-answering, translation, and arithmetic, GPT-3 is a game-changer for A.I. This week's brilliant guest, Melanie Subbiah, was a lead author of the GPT-3 paper.

GPT-3 is a natural language processing (NLP) model with 175 billion parameters that has demonstrated unprecedented and remarkable "few-shot learning" on the diverse tasks mentioned above (translation between languages, question-answering, performing three-digit arithmetic) as well as on many more (discussed in the episode).

Melanie's paper sent shockwaves through the mainstream media and was recognized with an Outstanding Paper Award from NeurIPS (the most prestigious machine learning conference) in 2020.

Melanie:
• Developed GPT-3 while she worked as an A.I. engineer at OpenAI, one of the world’s leading A.I. research outfits.
• Previously worked as an A.I. engineer at Apple.
• Is now pursuing a PhD at Columbia University in the City of New York specializing in NLP.
• Holds a bachelor's in computer science from Williams College.

In this episode, Melanie details:
• What GPT-3 is.
• Why applications of GPT-3 have transformed not only the field of data science but also the broader world.
• The strengths and weaknesses of GPT-3, and how these weaknesses might be addressed with future research.
• Whether transformer-based deep learning models spell doom for creative writers.
• How to address the climate change and bias issues that cloud discussions of large natural language models.
• The machine learning tools she’s most excited about.

This episode does have technical elements that will appeal primarily to practicing data scientists, but Melanie and I put an effort into explaining concepts and providing context wherever we could so hopefully much of this fun, laugh-filled episode will be engaging and informative to anyone who’s keen to learn about the start of the art in natural language processing and A.I.

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Engineering Natural Language Models — with Lauren Zhu

Zero-shot multilingual neural machine translation, how to engineer natural language models, and why you should use PCA to choose your job are topics covered this week by the fun and brilliant Lauren Zhu.

Lauren:
• Is an ML Engineer at Glean, a Silicon Valley-based natural language understanding company that has raised $55m in venture capital.
• Prior to Glean, she worked as an ML Intern at both Apple and the autonomous vehicle subsidiary of Ford Motor Company; as a software engineering intern at Qualcomm; and as an A.I. Researcher at The University of Edinburgh.
• Holds BS and MS degrees in Computer Science from Stanford
• Served as a teaching assistant for some of Stanford University’s most renowned ML courses such as "Decision Making Under Uncertainty" and "Natural Language Processing with Deep Learning".

In this episode, Lauren details:
• Where to access free lectures from Stanford courses online.
• Her research on Zero-Shot Multilingual Neural Machine Translation.
• Why you should use Principal Component Analysis to choose your job.
• The software tools she uses day-to-day at Glean to engineer natural language processing ML models into massive-scale production systems.
• Her surprisingly pleasant secret to both productivity and success.

There are parts of this episode that will appeal especially to practicing data scientists but much of the conversation will be of interest to anyone who enjoys a laugh-filled conversation on A.I., especially if you’re keen to understand the state-of-the-art in applying ML to natural language problems.

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Transformers for Natural Language Processing

This week's guest is award-winning author Denis Rothman. He details how Transformer models (like GPT-3) have revolutionized Natural Language Processing (NLP) in recent years. He also explains Explainable AI (XAI).

Denis:
• Is the author of three technical books on artificial intelligence
• His most recent book, "Transformers for NLP", led him to win this year's Data Community Content Creator Award for technical book author
• Spent 25 years as co-founder of French A.I. company Planilog
• Has been patenting A.I. algos such as those for chatbots since 1982

In this episode, Denis fills us in on:
• What Natural Language Processing is
• What Transformer architectures are (e.g., BERT, GPT-3)
• Tools we can use to explain *why* A.I. algorithms provide a particular output

We covered audience questions from Serg, Chiara, and Jean-charles during filming. For those we didn't get to ask, Denis is kindly answering via a LinkedIn post today!

The episode's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.