Python Polars: The Definitive Guide, with Jeroen Janssens and Thijs Nieuwdorp

Today's episode on Polars is in equal parts hilarious and informative with Jeroen and Thijs, who co-authored the brand-new O'Reilly book "Python Polars: The Definitive Guide". Enjoy this one!

More on Dr. Jeroen Janssens:

• Senior Developer Relations Engineer at Posit PBC (iconic creators of RStudio and much more).

• Previously, was Senior Machine Learning Engineer at Xomnia.

• Wrote the invaluable O’Reilly book "Data Science at the Command Line".

• Holds a PhD in machine learning from Tilburg University.

...and on Thijs Nieuwdorp:

• Lead Data Scientist at Xomnia, the largest Dutch data and A.I. consulting company.

• Holds a degree in A.I. from Radboud University.

Today’s episode will be particularly appealing to hands-on data science, machine learning and A.I. practitioners but Jeroen and Thijs are tremendous storytellers and frankly very funny so this episode can probably be enjoyed by anyone interested in data and A.I.

In today’s episode, Jeroen and Thijs detail:

• Why pandas users are rapidly switching to Polars for dataframe operations in Python.

• The inside story of how O'Reilly rejected four book proposals on Polars before accepting the fifth.

• The moment when an innocuous GitHub pull request forced a complete rewrite of an entire book chapter.

• A previously secret collaboration with NVIDIA and Dell that revealed remarkable GPU acceleration benchmarks by Polars.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Polars: Past, Present and Future, with Polars Creator Ritchie Vink

Because of it's stunningly fast speed, Polars is an extremely popular open-source library for DataFrame operations in Python. Kinda unreal to have Ritchie Vink, Polars' creator, as today's guest!

Ritchie:

• Is CEO and Co-Founder of Polars, Inc., a startup that has raised $4m in seed funding to support his Polars open-source project.

• Previously worked as an ML Engineer, Data Scientist and Data Engineer at companies like adidas and KLM Royal Dutch Airlines.

• Holds a Master’s in Structural Engineering and worked as a civil engineer prior to catching the data-science bug.

Today’s episode will appeal most to hands-on practitioners like data scientists and ML engineers. In it, Ritchie details:

• How Polars regularly achieves 5-20x (sometimes 100x!) speed improvements over Pandas for most DataFrame operations.

• The Eager and Lazy execution APIs Polars offers and when you should use one or the other.

• Ritchie's vision for scaling Polars to handle massive distributed datasets.

• How we can continue to make data-processing efficiency gains even as Moore's Law slows down.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

DataFrame Operations 100x Faster than Pandas, with Marco Gorelli

Today's episode is all about Polars — the hot library for Python that offers up to 100x speedups for DataFrame operations relative to pandas. Marco Gorelli, a core Polars developer, is our gifted guide.

Marco is a tremendously talented communicator of complex technical topics, making him the perfect guest for this highly technical episode. He:

• Is a core developer of the popular Python libraries pandas and Polars.

• Is the creator of the Narwhals library.

• Has spoken at several major Python conferences (such as PyData), taught Polars professionally, and wrote the first complete Polars plugins tutorial.

• Currently works as Senior Software Engineer at Quansight Labs.

• Previously, worked as a data scientist and was one of the prize winners (from amongst >100,000 entrants!) of the M6 forecasting competition.

• Holds a Master’s in Mathematics and the Foundations of Computer Science from the University of Oxford.

Today’s episode will appeal primarily to hands-on technical folks like data scientists, ML engineers and software developers.

In today’s episode, Marco details:

• What the hot, fast-growing Polars library for working with DataFrames in Python is (it already has 65m downloads and 28k GitHub stars).

• How Polars offers up to 100x speed-ups relative to Pandas on DataFrame operations.

• How the lightweight, dependency-free Narwhals package he created allows for easy compatibility between different DataFrame libraries such as Polars and Pandas.

• How he got addicted to open-source development.

• The simple trick he used to be a prize-winner in super-popular forecasting competitions.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Pandas for Data Analysis and Visualization

Today's episode is jam-packed with practical tips on using the Pandas library in Python for data analysis and visualization. Super-sharp Stefanie Molin — a bestselling author and sought-after instructor on these topics — is our guide.

Stefanie:
• Is the author of the bestselling book "Hands-On Data Analysis with Pandas".
• Provides hands-on pandas and data viz tutorials at top industry conferences.
• Is a software engineer and data scientist at Bloomberg, the financial data giant, where she tackles problems revolving around data wrangling/visualization and building tools for gathering data.
• Holds a degree in operations research from Columbia University as well as a masters in computer science, with an ML specialization, from Georgia Tech.

Today’s episode is intended primarily for hands-on practitioners like data analysts, data scientists, and ML engineers — or anyone that would like to be in a technical data role like these in the future.

In this episode, Stefanie details:
• Her top tips for wrangling data in pandas.
• In what data viz circumstances you should use pandas, matplotlib, or Seaborn.
• Why everyone who codes, including data scientists, should develop expertise in Python package creation as well as contribute to open-source projects.
• The tech stack she uses in her role at Bloomberg.
• The productivity tips she honed by simultaneously working full-time, completing a masters degree and writing a bestselling book.

The SuperDataScience podcast is available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Effective Pandas

Seven-time bestselling author Matt Harrison reveals his top tips and tricks to enable you to get the most out of Pandas, the leading Python data analysis library. Enjoy!

Matt's books, all of which have been Amazon best-sellers, are:
1. Effective Pandas
2. Illustrated Guide to Learning Python 3
3. Intermediate Python
4. Learning the Pandas Library
5. Effective PyCharm
6. Machine Learning Pocket Reference
7. Pandas Cookbook (now in its second edition)

Beyond being a prolific author, Matt:
• Teaches "Exploratory Data Analysis with Python" at Stanford
• Has taught Python at big organizations like Netflix and NASA
• Has worked as a CTO and Senior Software Engineer
• Holds a degree in Computer Science from Stanford University

On top of Matt's tips for effective Pandas programming, we cover:
• How to squeeze more data into Pandas on a given machine.
• His recommended software libraries for working with tabular data once you have too many data to fit on a single machine.
• How having a computer science education and having worked as a software engineer has been helpful in his data science career.

This episode will appeal primarily to practicing data scientists who are keen to learn about Pandas or keen to become an even deeper expert on Pandas by learning from a world-leading educator on the library.

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.

Open-Source Analytical Computing (pandas, Apache Arrow)

The legend Wes McKinney is this week's guest! He details the genesis of the ubiquitous pandas library, the forthcoming edition of his bestselling book, and how Apache Arrow brings analytics into the distributed computing era.

Wes:
• Created pandas, the industry-standard Python library for data analytics
• Co-created Apache Arrow, a language-agnostic open-source library for efficient analytics on modern distributed CPUs and GPUs
• Wrote the classic O'Reilly Media desk reference "Python for Data Analysis"
• Has worked as technical expert at prestigious firms like Cloudera, RStudio PBC, Two Sigma, and AQR Capital Management
• Today serves as co-founder and CTO of Voltron Data

In this episode, Wes takes us on a technical deep-drive through:
• The creation story of his now-ubiquitous pandas library
• A sneak peek at the third edition of his international-bestselling book
• What the Apache Arrow project is and why it's poised to revolutionize the data science and software industries
• The software and hardware tools that he uses daily to be such an epically productive software developer and entrepreneur
• Responses to great questions by listeners Daniel, David, Doug, and Brett

The SuperDataScience show's available on all major podcasting platforms, YouTube, and at SuperDataScience.com.