#503: The PyArrow Revolution

Talk Python To Me

Michael Kennedy

Technology

4.8 • 635 Ratings

🗓️ 28 April 2025

⏱️ 69 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

Pandas is at a the core of virtually all data science done in Python, that is virtually all data science. Since it's beginning, Pandas has been based upon numpy. But changes are afoot to update those internals and you can now optionally use PyArrow. PyArrow comes with a ton of benefits including it's columnar format which makes answering analytical questions faster, support for a range of high performance file formats, inter-machine data streaming, faster file IO and more. Reuven Lerner is here to give us the low-down on the PyArrow revolution.

Transcript

Click on a timestamp to play from that location

0:00.0	Pandas is at the core of virtually all data science done in Python.
0:03.6	That is virtually all data science.
0:06.5	Since its beginning, Pandas has been based upon NumPy, but changes are afoot to update those
0:12.0	internals, and you can now optionally use Pi Arrow.
0:15.3	Pi Arrow comes with a ton of benefits, including its columnar format, which makes answering
0:20.4	analytical questions faster, support for a range of its calumner format, which makes answering analytical questions faster,
0:22.5	support for a range of high-performance file formats, inter-machine data streaming, faster file
0:28.0	I.O., and more. Reuven Learner is here to give us the lowdown on the Pi-Arow Revolution.
0:33.4	This is Talk PythonMe, Episode 503 recorded April 8th, 2025.
0:39.5	Are you ready for your host? There is.
0:42.3	You're listening to Michael Kennedy on Talk Python to Me.
0:45.7	Life from Portland, Oregon, and this segment was made with Python.
0:51.8	Welcome to Talk Python, a weekly podcast on Python.
0:55.4	This is your host, Michael Kennedy.
0:57.5	Follow me on Massidon, where I'm at M. Kennedy, and follow the podcast using at Talk Python,
1:03.0	both accounts over at Fostodon.org, and keep up with the show and listen to over nine years of episodes at TalkPython.fm.
1:10.7	If you want to be part of our live
1:12.3	episodes, you can find the live streams over on YouTube, subscribe to our YouTube channel over
1:17.1	at TalkPython.fm slash YouTube and get notified about upcoming shows. This episode is brought to you by
1:23.2	Nordlayer. Nordlayer is a toggle-ready network security platform built for modern businesses.
1:28.4	It combines VPN, access control, and threat protection in one easy-use platform.
1:33.2	Visit talk python.fm slash Nordlayer and remember to use the code Talk Python-10.
	...

Please login to see the full transcript.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from Michael Kennedy, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of Michael Kennedy and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.