doris icon indicating copy to clipboard operation
doris copied to clipboard

[Improve](inverted index) improve query performance by not using output index result column

Open airborne12 opened this issue 1 year ago • 10 comments

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

airborne12 avatar Apr 29 '24 07:04 airborne12

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar Apr 29 '24 07:04 doris-robot

run buildall

airborne12 avatar Apr 29 '24 07:04 airborne12

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Apr 29 '24 07:04 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Apr 29 '24 07:04 github-actions[bot]

TeamCity be ut coverage result: Function Coverage: 35.59% (8931/25091) Line Coverage: 27.22% (73641/270583) Region Coverage: 26.41% (38046/144077) Branch Coverage: 23.17% (19383/83638) Coverage Report: http://coverage.selectdb-in.cc/coverage/6bfdb90d7987d6b86f335b9bceeec38c3ce3c226_6bfdb90d7987d6b86f335b9bceeec38c3ce3c226/report/index.html

doris-robot avatar Apr 29 '24 08:04 doris-robot

TPC-DS: Total hot run time: 182937 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6bfdb90d7987d6b86f335b9bceeec38c3ce3c226, data reload: false

query1	904	357	338	338
query2	6282	2267	2309	2267
query3	6659	215	217	215
query4	23577	21788	22060	21788
query5	3762	438	422	422
query6	261	190	184	184
query7	4562	296	290	290
query8	242	200	199	199
query9	8373	2395	2390	2390
query10	424	247	253	247
query11	11784	11080	11336	11080
query12	123	92	94	92
query13	1664	372	374	372
query14	9186	8755	7054	7054
query15	263	171	166	166
query16	8197	266	262	262
query17	1845	572	541	541
query18	2097	283	273	273
query19	342	152	152	152
query20	97	88	87	87
query21	205	135	125	125
query22	5014	4859	4872	4859
query23	33971	33189	33393	33189
query24	10555	2991	2982	2982
query25	614	362	368	362
query26	1169	153	146	146
query27	2350	312	314	312
query28	7087	2009	2016	2009
query29	855	598	590	590
query30	241	147	149	147
query31	944	718	742	718
query32	92	51	49	49
query33	740	237	246	237
query34	1044	464	465	464
query35	805	663	661	661
query36	1092	906	888	888
query37	137	67	63	63
query38	3150	3012	3054	3012
query39	1577	1509	1521	1509
query40	197	123	124	123
query41	41	37	37	37
query42	104	97	96	96
query43	553	544	554	544
query44	1263	733	734	733
query45	272	277	265	265
query46	1065	742	719	719
query47	1936	1836	1832	1832
query48	387	290	286	286
query49	835	414	386	386
query50	768	375	386	375
query51	6927	6781	6736	6736
query52	100	92	86	86
query53	348	276	288	276
query54	315	236	248	236
query55	82	73	73	73
query56	246	226	231	226
query57	1198	1108	1124	1108
query58	224	201	197	197
query59	3272	3113	2976	2976
query60	250	228	231	228
query61	86	90	87	87
query62	673	434	437	434
query63	306	280	276	276
query64	8493	7112	7080	7080
query65	3078	3071	3066	3066
query66	843	328	339	328
query67	15681	15026	14867	14867
query68	9088	555	556	555
query69	553	307	311	307
query70	1198	1096	1054	1054
query71	545	310	274	274
query72	8170	2603	2416	2416
query73	814	328	323	323
query74	6501	6141	6136	6136
query75	4963	2654	2673	2654
query76	5457	1026	919	919
query77	812	267	270	267
query78	10951	10210	10298	10210
query79	9435	523	523	523
query80	1286	439	439	439
query81	503	219	231	219
query82	250	91	99	91
query83	198	172	168	168
query84	261	84	81	81
query85	988	264	260	260
query86	353	323	305	305
query87	3335	3076	3147	3076
query88	4898	2336	2322	2322
query89	516	375	371	371
query90	2119	178	184	178
query91	125	100	96	96
query92	63	51	47	47
query93	6995	504	507	504
query94	1371	181	179	179
query95	387	300	300	300
query96	610	259	258	258
query97	3159	2915	2930	2915
query98	228	217	212	212
query99	1169	862	861	861
Total cold run time: 299130 ms
Total hot run time: 182937 ms

doris-robot avatar Apr 29 '24 08:04 doris-robot

run buildall

airborne12 avatar Apr 30 '24 03:04 airborne12

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Apr 30 '24 03:04 github-actions[bot]

TeamCity be ut coverage result: Function Coverage: 35.57% (8934/25114) Line Coverage: 27.19% (73658/270870) Region Coverage: 26.38% (38044/144203) Branch Coverage: 23.16% (19389/83708) Coverage Report: http://coverage.selectdb-in.cc/coverage/dd31227566fd952050337e0fe2ca1dec7fd77fda_dd31227566fd952050337e0fe2ca1dec7fd77fda/report/index.html

doris-robot avatar Apr 30 '24 04:04 doris-robot

TPC-DS: Total hot run time: 186390 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dd31227566fd952050337e0fe2ca1dec7fd77fda, data reload: false

query1	944	365	346	346
query2	6316	2330	2285	2285
query3	6638	211	221	211
query4	22934	21685	21819	21685
query5	3877	437	430	430
query6	269	187	175	175
query7	4682	299	299	299
query8	245	191	200	191
query9	8536	2452	2448	2448
query10	410	241	261	241
query11	15358	14856	14740	14740
query12	126	88	93	88
query13	1644	372	391	372
query14	9730	7543	6820	6820
query15	272	165	172	165
query16	8179	261	260	260
query17	1707	557	545	545
query18	2084	280	267	267
query19	329	149	149	149
query20	87	78	84	78
query21	195	130	130	130
query22	4978	4811	4865	4811
query23	33716	33251	33080	33080
query24	9489	2842	2766	2766
query25	561	354	356	354
query26	685	157	153	153
query27	2044	322	315	315
query28	5835	2067	2065	2065
query29	830	597	595	595
query30	232	148	148	148
query31	914	718	716	716
query32	90	51	52	51
query33	618	249	245	245
query34	856	492	480	480
query35	754	664	653	653
query36	1051	888	920	888
query37	102	69	65	65
query38	3136	2996	3001	2996
query39	1595	1534	1541	1534
query40	201	124	125	124
query41	42	37	38	37
query42	100	95	102	95
query43	563	542	554	542
query44	1066	723	768	723
query45	256	255	259	255
query46	1071	718	718	718
query47	1936	1855	1859	1855
query48	371	291	298	291
query49	838	395	391	391
query50	751	390	386	386
query51	6837	6879	6835	6835
query52	98	94	92	92
query53	354	279	284	279
query54	303	233	259	233
query55	80	78	70	70
query56	237	221	226	221
query57	1209	1139	1100	1100
query58	230	200	202	200
query59	3302	3096	3141	3096
query60	255	229	232	229
query61	89	86	84	84
query62	647	451	462	451
query63	306	290	285	285
query64	8285	7225	7182	7182
query65	3094	3032	3042	3032
query66	761	345	322	322
query67	15212	14981	14886	14886
query68	5290	534	554	534
query69	489	294	307	294
query70	1182	1130	1102	1102
query71	421	272	268	268
query72	7166	2513	2346	2346
query73	722	327	326	326
query74	6551	6107	6111	6107
query75	3513	2630	2640	2630
query76	3308	936	897	897
query77	405	268	272	268
query78	10897	10307	10217	10217
query79	3662	550	531	531
query80	2261	435	433	433
query81	533	216	222	216
query82	882	94	99	94
query83	277	174	173	173
query84	258	93	157	93
query85	1970	280	263	263
query86	489	297	300	297
query87	3300	3164	3079	3079
query88	4814	2423	2415	2415
query89	473	389	379	379
query90	2072	184	187	184
query91	129	97	98	97
query92	59	49	46	46
query93	5181	523	503	503
query94	1220	181	194	181
query95	396	314	299	299
query96	605	265	265	265
query97	3147	2945	2936	2936
query98	245	213	221	213
query99	1225	880	850	850
Total cold run time: 283684 ms
Total hot run time: 186390 ms

doris-robot avatar Apr 30 '24 04:04 doris-robot

run buildall

airborne12 avatar May 10 '24 01:05 airborne12

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar May 10 '24 01:05 github-actions[bot]

TeamCity be ut coverage result: Function Coverage: 35.69% (8982/25165) Line Coverage: 27.34% (74204/271399) Region Coverage: 26.57% (38357/144339) Branch Coverage: 23.40% (19565/83628) Coverage Report: http://coverage.selectdb-in.cc/coverage/962c4059415c3e828de39713127350bc374a55fb_962c4059415c3e828de39713127350bc374a55fb/report/index.html

doris-robot avatar May 10 '24 03:05 doris-robot

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar May 14 '24 08:05 github-actions[bot]