doris icon indicating copy to clipboard operation
doris copied to clipboard

[improvement](vectorized) Improve count distinct performance by using fastunion

Open zenoyang opened this issue 3 years ago • 1 comments

Proposed changes

Issue Number: close #xxx Improve count distinct performance by using fastunion. Testing our user real data has a 10-40% performance improvement.

Problem summary

Describe your changes.

Checklist(Required)

  1. Does it affect the original behavior:
    • [x] Yes
    • [ ] No
    • [ ] I don't know
  2. Has unit tests been added:
    • [x] Yes
    • [ ] No
    • [ ] No Need
  3. Has document been added or modified:
    • [ ] Yes
    • [ ] No
    • [x] No Need
  4. Does it need to update dependencies:
    • [ ] Yes
    • [x] No
  5. Are there any changes that cannot be rolled back:
    • [ ] Yes (If Yes, please explain WHY)
    • [x] No

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

zenoyang avatar Aug 04 '22 10:08 zenoyang

wow

yiguolei avatar Aug 04 '22 11:08 yiguolei

hi, I see this improvement also speed up agg function of bitmap_union, why not add it?

zhangstar333 avatar Aug 12 '22 06:08 zhangstar333

LGTM, it's a great work! , And I could speed up to bitmap union using this way later

zhangstar333 avatar Aug 16 '22 02:08 zhangstar333