USI - Email

Archive / INF Seminars / INF_2024_11_14_Findahl

USI - Email

	From Pandas to Polars: Achieving 50x Speedups and Scaling Beyond Memory Limits

Thursday

14.11

Room D1.13, East Campus

16:30-17:30


	Jesper Findahl

Abstract: For over a decade, Pandas has been the standard Python library for data manipulation and analysis. But recent advancements—like Rust-based performance, parallelized computations, and GPU acceleration—are reshaping the data processing landscape and paving the way for new tools. In 2024, Polars marked a major milestone with its 1.0 release in July, after steadily gaining popularity for its speed, memory efficiency, and ability to handle larger-than-memory datasets. In this talk, we’ll explore why CodeLounge decided to switch to Polars in early 2024 and how it has transformed our developer experience. By comparing Polars with Pandas, we’ll highlight how we have achieved up to 50x faster execution for specific workloads and how Polars enables seamless processing of larger than memory datasets. Finally, we’ll look at exciting upcoming features aimed at making Polars even more powerful for data science tasks.

Biography: Jesper Findahl is a Senior R&D Software Engineer at CodeLounge, the software research and development center at the Software Institute (SI), Università della Svizzera italiana (USI), Lugano Switzerland. After completing the Master’s in Informatics degree at USI he joined CodeLounge in 2018. At CodeLounge Jesper has taken on a wide array of tasks, from frontend and backend engineering to data visualization, continuous integration/deployment (CI/CD), and software analysis, recently branching into data science and machine learning. With a strong passion for software design and productivity, Jesper is always eager to embrace new technologies and deepen expertise in the field.

Chair: Alberto Martin Lopez

*************************
In February 2019, the Software Institute started its SI Seminar Series. Every Thursday afternoon, a researcher of the Institute will publicly give a short talk on a software engineering argument of their choice. Examples include, but are not limited to novel interesting papers, seminal papers, personal research overview, discussion of preliminary research ideas, tutorials, and small experiments.
On our YouTube playlist you can watch some of the past seminars. On the SI website you can find more details on the next seminar, the upcoming seminars, and an archive of the past speakers.