Why not use BSTM to calculate TopK

[xiehuc]

i think pilosa use cache and min heap to calculate TopK

but since pilosa use bsi heavly, and there are a paper introduced BSTM which means (Bit Slice Term Match) algorithm to calculate TopK

it would be very suitable for pilosa

https://www.researchgate.net/publication/221215144_Bit-Sliced_Index_Arithmetic

[kuba--]

Thanks @xiehuc for sharing. We're working to improve performance of aggregation functions. Moreover TopN implementation on integers is missed, right now. So, I think it's totally worth to consider the white-paper, which you mentioned.

[Smallhi]

@xiehuc see https://github.com/lemire/BitSliceIndex/tree/master/src/main/java/org/roaringbitmap/circuits/topk, the author of roaringBitmap write two type of algorithm for topK, one is RinfretONeil, one is simple by Tree

[xiehuc]

that's cool, i'd see it later.