[Pipeline](shared_scan_opt) Support shared scan opt in pipeline exec engine

Open HappenLee opened this issue 2 years ago • 11 comments

Proposed changes

clickbench 1 tablet：

Problem summary

Describe your changes.

Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

Feb 27 '23 07:02 HappenLee

run buildall

Feb 27 '23 15:02 HappenLee

run buildall

Feb 27 '23 15:02 HappenLee

TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.13 seconds stream load tsv: 468 seconds loaded 74807831229 Bytes, about 152 MB/s stream load json: 41 seconds loaded 2358488459 Bytes, about 54 MB/s stream load orc: 75 seconds loaded 1101869774 Bytes, about 14 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230310095513_clickbench_pr_112193.html

Feb 27 '23 16:02 hello-stephen

run buildall

Feb 28 '23 11:02 HappenLee

run buildall

Mar 01 '23 15:03 HappenLee

run buildall

Mar 02 '23 11:03 HappenLee

run allbuild

Mar 03 '23 10:03 HappenLee

run allbuild

Mar 03 '23 10:03 HappenLee

run allbuild

Mar 03 '23 10:03 HappenLee

run buildall

Mar 03 '23 11:03 HappenLee

run buildall

Mar 03 '23 11:03 HappenLee

run buildall

Mar 06 '23 11:03 HappenLee

run buildall

Mar 07 '23 16:03 HappenLee

run buildall

Mar 09 '23 10:03 HappenLee

run buildall

Mar 09 '23 16:03 HappenLee

run buildall

Mar 10 '23 07:03 HappenLee

PR approved by at least one committer and no changes requested.

Mar 13 '23 02:03 github-actions[bot]

PR approved by anyone and no changes requested.

Mar 13 '23 02:03 github-actions[bot]