...

falconroar

18

Karma

2021-07-26

Created

Recent Activity

  • Interesting, I wasn't aware; thanks for that. I will say, Polars' implementation is much more centered on out-of-core processing, and bypasses some of DuckDB's limitations ("DuckDB cannot yet offload some complex intermediate aggregate states to disk"). Both incredible pieces of software.

    To expand on this, Polars' `LazyFrame` implementation allows for simple addition of new backends like GPU, streaming, and now distributed computing (though it's currently locked to a vendor). The DuckDB codebase just doesn't have this flexibility, though there are ways to get it to run on GPU using external software.

  • Polars also has all of these benefits (to some degree), but also allows for larger-than-memory datasets. Also has GPU backend, distributed backend, etc. Polars is heavily underrated, even with the recent hype.

  • Polars has all of the benefits of DuckDB (to some degree), but also allows for larger-than-memory datasets.

  • Polars has all of these benefits (to some degree), but also allows for larger-than-memory datasets.

  • Polars has all of these benefits (to some degree), but also allows for larger-than-memory datasets.

HackerNews