Naive pandas is going to limit you to RAM sizes when parquet is deliberately mea... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		0cf8612b2e1e on April 4, 2023 \| parent \| context \| favorite \| on: Parquet: An efficient, binary file format for tabl... Naive pandas is going to limit you to RAM sizes when parquet is deliberately meant for huge datasets.

FridgeSeal on April 4, 2023 [–]

Anyone reading who is currently using pandas and running into this issue-polars is generally faster, and now supports (still new though) out-of-core workloads.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact