Anthropic’s latest publicly released model, Claude 3.5 Sonnet. This might not seem like a big deal because it’s not a “whole number” release like Claude 3 was or Claude 4 eventually will be, but in fact, it’s quite a big deal as this model now appears to actually represent the state of the art for text-in/text-out generative LLM, outcompeting the other frontier models like OpenAI’s GPT-4o and Google’s Gemini.
Read More
Filtering by Category: Five-Minute Friday
Earth’s Coming Population Collapse and How AI Can Help, with Simon Kuestenmacher
Worried about overpopulation? Excessive immigration? In today's episode, demographer Simon Kuestenmacher reveals the data on why we should be more concerned about the opposite: the coming global-population collapse.
Simon:
• Is Co-Founder and Director of The Demographics Group, a firm that provides advice on demographic data to businesses and governments.
• Writes a regular column on demographics for The Australian, the antipodean country’s most widely-read newspaper.
• He holds a Master’s in Urban Geography from the University of Melbourne.
Today’s episode should be of great interest to anyone! In it, Simon details:
• Why demography is the closest thing we have to a crystal ball.
• Why the world is at a greater risk of underpopulation than overpopulation by humans this century.
• How, in less than a decade, developed nations that depend on migrants to prevent their populations from declining will run out of immigrants.
• How A.I. and automation may solve both the coming low-migration crisis and the later global underpopulation crisis.
• The implications of vastly life-extending healthcare breakthroughs.
• What you can do in your career to prepare for the coming demographic and technological shifts.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Multi-Agent Systems: How Teams of LLMs Excel at Complex Tasks
Groundbreaking multi-agent systems (MAS, for short) are transforming the way AI models collaborate to tackle complex challenges.
Read MoreThe Six Keys to Data Scientists’ Success, with Kirill Eremenko
For today's episode, Kirill Eremenko — who has taught more than 2.8 million people data science — fills us in on his six most valuable insights about data science careers.
More on Kirill:
• Founder and CEO of SuperDataScience, an e-learning platform that is the namesake of this very podcast.
• Launched the SuperDataScience Podcast in 2016 and hosted the show until he passed me the reins four years ago.
• Has reached more than 2.8 million students through the courses he’s published on Udemy, making him Udemy’s most popular data science instructor.
At a high level, Kirill's six data science insights are:
1. Unlike many other careers, there’s no need for formal credentials to become a data scientist.
2. Mentors can be invaluable guides in a DS career, but you should also try to give back to your mentors when you can.
3. Portfolios are the key to landing the DS job of your dream because they showcase your DS abilities for all to see.
4. Hands-on labs are a fun, interactive way to develop your portfolio and are a great complement to classes.
5. Collaborations can make lots of aspects of DS career development fun, including learning new materials, completing labs and developing your portfolio.
6. Data scientists can come from any background and work from anywhere in the world with an Internet connection.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Aligning Large Language Models, with Sinan Ozdemir
For today’s quick Five-Minute Friday episode, the exceptional author, speaker and entrepreneur Sinan Ozdemir provides an overview of what it actually means for an LLM to be “aligned”.
More on Sinan:
• Is Founder and CTO of LoopGenius, a generative AI startup.
• Has authored several excellent books, including, most recently, the bestselling "Quick Start Guide to Large Language Models".
• Is a serial AI entrepreneur, including founding a Y Combinator-backed generative AI startup way back in 2015 that was later acquired.
This episode was filmed live at the Open Data Science Conference (ODSC) East in Boston last month. Thanks to ODSC for providing recording space.
The Super Data Science Podcast is available on all major podcasting platforms and a video version is on YouTube. This is episode #784!
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
In Case You Missed It in April 2024
Other than excessive maleness and paleness*, April 2024 was an excellent month for the podcast, packed with outstanding guests. ICYMI, today's episode highlights the most fascinating moments of my convos with them.
Specifically, conversation highlights include:
1. Iconic open-source developer Dr. Hadley Wickham putting the "R vs Python" argument to bed.
2. Aleksa Gordić, creator of a digital A.I.-learning community of 160k+ people, on the movement from formal to self-directed education.
3. World-leading futurist Bernard Marr on how we can work with A.I. as opposed to it lording over of us.
4. Educator of millions of data scientists, Kirill Eremenko, on why gradient boosting is so powerful for making informed business decisions.
5. Prof. Barrett Thomas on how drones could transform same-day delivery.
*Remedied in May!
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
How to Become a Data Scientist, with Dr. Adam Ross Nelson
Today's episode features Dr. Adam Ross Nelson providing his #1 most useful piece of guidance on "How to Become a Data Scientist" from his book of that very name!
This was filmed live at the Open Data Science Conference (ODSC) East in Boston last week — thanks ODSC East for providing valuable conference space for us to shoot podcast episodes.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Deep Utopia: AI Could Solve All Human Problems in Our Lifetime
Today’s episode focuses on Nick Bostrom's latest book, Deep Utopia. Published a couple of weeks ago, it delves into the possibilities of a future where artificial intelligence has solved humanity's deepest problems.
Read MoreThe Mamba Architecture: Superior to Transformers in LLMs
Modern, cutting-edge A.I. basically depends entirely on the Transformer. But now, the first serious contender to the Transformer has emerged and it’s called Mamba; we’ve got the full paper—called "Mamba: Linear-TimeSequence Modeling with Selective State Spaces" and written by researchers at Carnegie Mellon and Princeton.
Read MoreAlphaGeometry: AI is Suddenly as Capable as the Brightest Math Minds
Google DeepMind's open-sourced AlphaGeometry blends "fast thinking" (like intuition) with "slow thinking" (like careful, conscious reasoning) to enable a big leap forward in A.I. capability and match human Math Olympiad gold medalists on geometry problems.
KEY CONTEXT
• A couple weeks ago, DeepMind published on AlphaGeometry in the prestigious journal peer-reviewed Nature.
• DeepMind focused on geometry due to its demand for high-level reasoning and logical deduction, posing a unique challenge that traditional ML models struggle with.
MASSIVE RESULTS
• AlphaGeometry tackled 30 International Mathematical Olympiad problems, solving 25. This outperforms human Olympiad bronze and silver medalists' averages (who solved 19.3 and 22.9, respectively) and closely rivals gold medalists (who solved 25.9).
• This new system crushes the previous state-of-the-art A.I., which solved only 10 out of 30 problems.
• Beyond solving problems, AlphaGeometry also generates understandable proofs, making A.I.-generated solutions more accessible to humans.
HOW?
• AlphaGeometry uses a new method of generating synthetic theorems and proofs, simulating 100 million unique examples to overcome the limitations of (expensive, laborious) human-generated proofs.
• It combines a neural (deep learning) language model for intuitive guesswork with a symbolic deduction engine for logical problem-solving, mirroring "fast" and "slow thinking" processes akin to human cognition (per Daniel Kahneman's "Thinking, Fast and Slow" book).
IMPACT
• A.I. that can "think fast and slow" like AlphaGeometry could generalize across mathematical fields and potentially other scientific disciplines, pushing the boundaries of human knowledge and problem-solving capabilities.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
The Five Levels of AGI
Artificial General Intelligence (AGI) is a term thrown around a lot, but it's been poorly defined. Until now!
Read MoreA Continuous Calendar for 2024
Today's super-short episode provides a "Continuous Calendar" for 2024. In my view, far superior to the much more common Weekly or Monthly calendar formats, a Continuous Calendar can keep you on top of all your projects and commitments all year 'round.
I know I’m not the only one who Continuous Calendars because my annual blog post providing an updated continuous calendar for the new year is reliably one of my most popular blog posts. The general concept is that Continuous Calendars enable you to:
1. Overview large blocks of time at a glance (I can easily fit six months on a standard piece of paper).
2. Get a more realistic representation of how much time there is between two given dates because the dates don’t get separated by arbitrary 7-day or ~30-day cutoffs.
The way they work so effectively is that continuous calendars are a big matrix where every row corresponds to a week and every column corresponds to a day of the week.
So if you’d like to get started today with your own super-efficient Continuous Calendar in 2024, simply head to jonkrohn.com/cal24.
At that URL, you’ll find a Google Sheet with the full 52 weeks of the year, which will probably suit most people’s needs. If you print it on standard US 8.5” x 11” paper, it should get split exactly so that the first half of the year is on page one and the second half of the year is on page two.
The calendar template is simple: It’s all black except that we’ve marked U.S. Federal Holidays with red dates. If you’re in another region, or you’d like to adapt our continuous calendar for any reason at all, simply make a copy of the sheet or download it, and then customize it to your liking.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
To a Peaceful 2024
Today I reflect on the wild advances in A.I. over the past year, opine on how A.I. could make the world more peaceful, and wrap 2023 up by singing a tune. Thanks to all eight humans of the Super Data Science Podcast for their terrific work all year 'round:
• Ivana Zibert: Podcast Manager
• Natalie Ziajski: Operations & Revenue
• Mario Pombo: Media Editor
• Serg Masís: Researcher
• Sylvia Ogweng: Writer
• Dr. Zara Karschay: Writer
• Kirill Eremenko: Founder
It's these terrifically talented and diligent people that make it possible for us to create 104 high-quality podcast episodes per year for now over seven years running 🙏
I'm looking forward to the next 104 episodes with awesome guests and (no doubt!) oodles of revolutionary new machine learning breakthroughs to cover. To a wonderful and hopefully much more peaceful 2024 🥂
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Happy Holidays from All of Us
Today's podcast episode is a quick one from all eight of us humans at the SuperDataScience Podcast, wishing you the happiest of holiday seasons ☃️
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Humanoid Robot Soccer, with the Dutch RoboCup Team
In today's unique episode, robots from the Dutch Nao Team (Naos are the little humanoids shown in the photo) compete against each other at football (⚽️) while Dário Catarrinho, a developer on the team, describes the machine learning involved.
The Dutch Nao Team is one of many international teams that competes annually in RoboCup Federation tournaments. The lofty goal of the RoboCup competitions is to develop a team of humanoid robots that is able to win against the human World Cup Championship team by the year 2050. Very cool.
Dario, my human guest in today's episode is Secretary of the Dutch Nao Team as well as a software developer on the team. He's also pursuing a degree in A.I. at the University of Amsterdam.
Most of today’s episode should be accessible to anyone but occasionally Dario and I talk a bit technically about ML algorithms so those brief parts might be most meaningful to hands-on practitioners.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Data Science for Astronomy, with Dr. Daniela Huppenkothen
Our planet is a tiny little blip in a vast universe. In today's episode, the astronomical data scientist and talented simplifier of the complex, Dr. Daniela Huppenkothen, explains how we collect data from space and use ML to understand the universe.
Daniela:
• Is a Scientist at both the University of Amsterdam and the SRON Netherlands Institute for Space Research.
• Was previously an Associate Director of the Institute for Data-Intensive Research in Astronomy and Cosmology at the University of Washington, and was also a Data Science Fellow at New York University.
• Holds a PhD in Astronomy from the University of Amsterdam.
Most of today’s episode should be accessible to anyone but there is some technical content in the second half that may be of greatest interest to hands-on data science practitioners.
In today’s episode, Daniela details:
• The data earthlings collect in order to observe the universe around us.
• The three categories of ways machine learning is applied to astronomy.
• How you can become an astronomer yourself.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
How GitHub Operationalizes AI for Teamwide Collaboration and Productivity, with GitHub COO Kyle Daigle
Today's episode features the exceptionally passionate GitHub COO Kyle Daigle detailing how generative A.I. tools improve not only the way individuals work, but also dramatically transform the way people across entire firms collaborate.
Kyle was my on-stage guest for a "fireside chat" live on stage at Insight Partners' ScaleUp:AI conference in New York. It was a terrifically slick conference and a ton of fun to collaborate on stage with Kyle! He's an energizing and inspiring speaker.
Check out the episode for all of our conversation; some of the key takeaways are:
• Generative AI tools like GitHub CoPilot are most useful and efficient when they’re part of your software-development flow.
• These kinds of in-flow generative AI tools can be used for collaboration (such as speeding up code review) not just on an individual basis.
• "Innersourcing" takes open-source principles but applies them within an organization on their proprietary assets.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
Use Contrastive Search to get Human-Quality LLM Outputs
Historically, when we deploy a machine learning model into production, the parameters that the model learned during its training on data were the sole driver of the model’s outputs. With the Generative LLMs that have taken the world by storm in the past few years, however, the model parameters alone are not enough to get reliably high-quality outputs. For that, the so-called decoding method that we choose when we deploy our LLM into production is also critical.
Read MoreDecoding Speech from Raw Brain Activity, with Dr. David Moses
Dr. David Moses and his colleagues have pulled off a miracle with A.I.: allowing paralyzed patients to "speak" through a video avatar in real time — using brain waves alone. In today's episode, David details how ML makes this possible.
David:
• Is an adjunct professor at the University of California, San Francisco.
• Is the project lead on the BRAVO (Brain-Computer Interface Restoration of Arm and Voice) clinical trial.
• The success of this extraordinary BRAVO project led to an article in the prestigious journal Nature and YouTube video that already has over 3 million views.
Today’s episode does touch on specific machine learning (ML) terminology at points, but otherwise should be fascinating to anyone who’d like to hear how A.I. is facilitating real-life miracles.
In this episode, David details:
• The genesis of the BRAVO project.
• The data and the ML models they’re using on the BRAVO project in order to predict text, speech sounds and facial expressions from the brain activity of paralyzed patients.
• What’s next for this exceptional project including how long it might be before these brain-to-speech capabilities are available to anyone who needs them.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.
AI Emits Far Less Carbon Than Humans (Doing the Same Task)
There's been a lot of press about Large Language Models (LLMs), such as those behind ChatGPT, using vast amounts of energy per query. In fact, however, a person doing the same work emits 12x to 45x more carbon from their laptop alone.
Today’s "Five-Minute Friday" episode is a quick one on how “The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humans”. Everything in today’s episode is based on an ArXiV preprint paper with that title by researchers from UC Irvine, the Massachusetts Institute of Technology and other universities.
For writing a page of text, for example, the authors estimate:
• BLOOM open-source LLM (including training) produces ~1.6g CO2/query.
• OpenAI's GPT-3 (including training) produces ~2.2g CO2/query.
• Laptop usage for 0.8 hours (average time to write page) emits ~27g CO2 (that's 12x GPT-3).
• Desktop for same amount of writing time emits ~72g CO2 (32 x GPT-3).
For creating a digital illustration:
• Midjourney (including training) produces ~1.9g CO2/query.
• DALL-E 2 produces ~2.2g CO2/query.
• Human takes ~3.2 hours for the same work, emitting ~100g CO2 (45 x DALL-E 2) on a laptop or ~280g CO2 (127 x DALL-E 2) on a desktop.
There are complexities here, such as what humans do with their time instead of writing or illustrating; if it’s spent driving, for example, then the net impact would be worse. As someone who’d love to see the world at net negative carbon emissions ASAP through innovations like nuclear fusion and carbon capture, however, I have been getting antsy about how much energy state-of-the-art LLMs use, but this simple article turned that perspective upside down. I’ll continue to use A.I. to augment my work wherever I can... and hopefully get my day done earlier so I can get away from my machine and enjoy some time outdoors.
Hear more detail in today's episode or check out the video version to see figures as well.
The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.