SortOfProblem icon indicating copy to clipboard operation
SortOfProblem copied to clipboard

Performance of indexed sorted data?

Open GotoFinal opened this issue 8 years ago • 1 comments

I wonder if there will be any visible difference when iterating over indexed sorted data, as instead of simple sequential reads from memory we need to jump to random places in memory to get each element. Can you maybe test if there will be any difference when iterating over all that sorted elements with some simple operation inside loop? Would be interesting if cost of accessing that data would be similar to what we saved by not sorting real data and just using indexes.

Sorry that I didn't just make a PR, but currently I don't have enough time to play with this, especially that C# isn't my language so it will take some time for me to find and learn some good benchmark tools for this language.

GotoFinal avatar Jan 21 '18 17:01 GotoFinal

Well, if you're iterating over all the data, most aggregates don't care about the sort, so you would just ignore the index and go via the data...

I understand what you're asking, and I can test it, but: in reality it isn't a huge major scenario. The main thing that might require a sorted full scan is an ordered export.

On 21 Jan 2018 5:40 pm, "Bartłomiej Mazur" [email protected] wrote:

I wonder if there will be any visible difference when iterating over indexed sorted data, as instead of simple sequential reads from memory we need to jump to random places in memory to get each element. Can you maybe test if there will be any difference when iterating over all that sorted elements with some simple operation inside loop?

Sorry that I didn't just make a PR, but currently I don't have enough time to play with this, especially that C# isn't my language so it will take some time for me to find and learn some good benchmark tools for this language.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mgravell/SortOfProblem/issues/3, or mute the thread https://github.com/notifications/unsubscribe-auth/AABDsD2GSyCzBGDAAXK3Y8BnOzkDOvO6ks5tM3Z2gaJpZM4Rl5bn .

mgravell avatar Jan 21 '18 18:01 mgravell