Polars: done the fast, now the scale
Room: Saphire B - PyData
Time: 15:00 - 15:25
DataFrame abstractions are one of the favorite data structures of many data scientists, data-engineers and programmers in general. They offer flexibility and intuitive reasoning on top of query processing. However, the implementation of DataFrame abstractions have been lacking. On the single node they have been ignoring most research available in RDBMS research. Different from RDBMS, the most known python implementations don't control their own query engines, and are therefore always compromising control, performance and memory usage. Polars is a DataFrame library that brings a very fast OLAP query engine to the DataFrame abstraction. This talk we look at what polars has achieved since it's inception and what the future will hold in store. <some extra characters because they were needed to fill the cell>
Ritchie Vink is the author of polars. He has a background in machine learning/ data engineering and software engineering.