modin icon indicating copy to clipboard operation
modin copied to clipboard
verified publisher icon modin-project

read_csv(): Add heuristic for the number of partitions calculation when loading from CSV

Open AndreyPavlenko opened this issue 2 years ago • 0 comments

Currently, the number of partitions is equal to the number of CPUs, that is not efficient in case of a small dataset. Here is an example - https://github.com/modin-project/modin/issues/5533#issue-1534806633

AndreyPavlenko avatar Sep 29 '23 13:09 AndreyPavlenko