doris icon indicating copy to clipboard operation
doris copied to clipboard

[Enhance](Routine Load) enhance routine load get topic metadata

Open GoGoWen opened this issue 1 year ago • 8 comments

Proposed changes

use _k_consumer->metadata(false /* for this topic /, topic, &metadata, 5000); instead of _k_consumer->metadata(true / for this topic */, topic, &metadata, 5000); because _k_consumer->metadata(true ...) will get all topic's metadata, which will cause performance issue.

GoGoWen avatar May 30 '24 07:05 GoGoWen

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.

doris-robot avatar May 30 '24 07:05 doris-robot

run buildall

GoGoWen avatar May 30 '24 08:05 GoGoWen

TeamCity be ut coverage result: Function Coverage: 36.27% (9229/25448) Line Coverage: 27.62% (75678/274016) Region Coverage: 26.83% (39161/145957) Branch Coverage: 23.59% (19883/84280) Coverage Report: http://coverage.selectdb-in.cc/coverage/10eebb2fa6f6fcffad78eedb03113e096e534319_10eebb2fa6f6fcffad78eedb03113e096e534319/report/index.html

doris-robot avatar May 30 '24 08:05 doris-robot

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar May 30 '24 10:05 github-actions[bot]

TPC-H: Total hot run time: 40915 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10eebb2fa6f6fcffad78eedb03113e096e534319, data reload: false

------ Round 1 ----------------------------------
q1	17605	4492	4261	4261
q2	2025	196	208	196
q3	10430	1266	1231	1231
q4	10213	725	785	725
q5	7469	2695	2701	2695
q6	221	135	135	135
q7	960	635	609	609
q8	9228	2112	2070	2070
q9	9177	6639	6596	6596
q10	9303	3901	3835	3835
q11	441	230	244	230
q12	534	218	235	218
q13	17423	3207	3326	3207
q14	249	217	219	217
q15	513	485	476	476
q16	499	390	405	390
q17	986	670	737	670
q18	8415	7751	7668	7668
q19	5990	1672	1594	1594
q20	667	319	312	312
q21	5242	4094	3256	3256
q22	381	324	325	324
Total cold run time: 117971 ms
Total hot run time: 40915 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4629	4470	4383	4383
q2	400	264	269	264
q3	3098	2963	2886	2886
q4	1849	1567	1596	1567
q5	5487	5463	5464	5463
q6	214	123	122	122
q7	2184	1810	1785	1785
q8	3245	3359	3333	3333
q9	8629	8671	8658	8658
q10	3984	3730	3820	3730
q11	592	500	505	500
q12	777	624	618	618
q13	17189	3139	3163	3139
q14	305	277	258	258
q15	528	484	474	474
q16	512	421	440	421
q17	1820	1492	1477	1477
q18	7586	7562	7310	7310
q19	1666	1610	1551	1551
q20	1999	1795	1797	1795
q21	8156	4734	4766	4734
q22	609	541	531	531
Total cold run time: 75458 ms
Total hot run time: 54999 ms

doris-robot avatar May 30 '24 10:05 doris-robot

TPC-DS: Total hot run time: 169687 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10eebb2fa6f6fcffad78eedb03113e096e534319, data reload: false

query1	941	374	369	369
query2	6926	2327	2279	2279
query3	6646	220	218	218
query4	22309	17190	17351	17190
query5	4115	443	427	427
query6	247	173	154	154
query7	4589	306	288	288
query8	339	292	293	292
query9	8576	2408	2405	2405
query10	444	278	254	254
query11	10624	10096	10010	10010
query12	133	91	86	86
query13	1634	382	369	369
query14	10031	6750	7070	6750
query15	250	194	188	188
query16	7787	272	270	270
query17	1623	542	524	524
query18	1968	285	281	281
query19	200	156	158	156
query20	98	86	90	86
query21	198	140	132	132
query22	4212	3897	3952	3897
query23	33707	33103	32904	32904
query24	10550	2836	2819	2819
query25	599	359	379	359
query26	720	158	152	152
query27	2233	325	333	325
query28	6013	2099	2128	2099
query29	895	609	607	607
query30	251	157	150	150
query31	959	779	755	755
query32	86	51	61	51
query33	650	274	261	261
query34	894	471	465	465
query35	761	611	601	601
query36	1090	904	896	896
query37	102	64	67	64
query38	2890	2773	2774	2773
query39	868	807	786	786
query40	198	128	123	123
query41	51	49	50	49
query42	110	96	95	95
query43	601	566	539	539
query44	1132	745	788	745
query45	175	171	172	171
query46	1076	737	700	700
query47	1847	1787	1778	1778
query48	391	308	290	290
query49	853	387	400	387
query50	770	384	402	384
query51	6784	6670	6642	6642
query52	103	91	116	91
query53	352	287	289	287
query54	850	442	423	423
query55	77	73	71	71
query56	263	249	244	244
query57	1118	1034	1040	1034
query58	222	216	219	216
query59	3514	3249	3288	3249
query60	283	265	257	257
query61	93	87	90	87
query62	644	442	460	442
query63	317	290	288	288
query64	8562	2226	1710	1710
query65	3159	3097	3120	3097
query66	789	335	330	330
query67	15305	15138	15126	15126
query68	5349	539	559	539
query69	489	270	278	270
query70	1145	1148	1137	1137
query71	475	281	270	270
query72	8174	5138	2744	2744
query73	757	322	326	322
query74	6134	5600	5611	5600
query75	3752	2643	2603	2603
query76	3627	1007	1049	1007
query77	597	274	276	274
query78	10267	9802	9844	9802
query79	2936	519	518	518
query80	1974	442	435	435
query81	515	224	220	220
query82	1228	90	96	90
query83	312	173	175	173
query84	264	88	91	88
query85	1457	277	265	265
query86	463	296	293	293
query87	3403	3087	3091	3087
query88	3710	2350	2379	2350
query89	471	396	395	395
query90	2010	189	193	189
query91	193	99	95	95
query92	57	50	52	50
query93	2344	508	512	508
query94	1218	188	191	188
query95	397	320	310	310
query96	588	265	268	265
query97	3177	2991	3061	2991
query98	256	222	209	209
query99	1169	853	862	853
Total cold run time: 275705 ms
Total hot run time: 169687 ms

doris-robot avatar May 30 '24 10:05 doris-robot

ClickBench: Total hot run time: 30.54 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10eebb2fa6f6fcffad78eedb03113e096e534319, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.68	0.08	0.08
query5	0.50	0.50	0.50
query6	1.12	0.74	0.71
query7	0.01	0.01	0.02
query8	0.04	0.04	0.05
query9	0.56	0.50	0.49
query10	0.54	0.55	0.53
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.61
query14	0.77	0.78	0.78
query15	0.83	0.80	0.80
query16	0.36	0.35	0.37
query17	0.96	0.99	0.95
query18	0.21	0.23	0.24
query19	1.82	1.76	1.76
query20	0.02	0.01	0.01
query21	15.80	0.70	0.68
query22	4.81	7.03	2.01
query23	18.29	1.36	1.25
query24	1.86	0.25	0.20
query25	0.14	0.08	0.08
query26	0.27	0.16	0.16
query27	0.08	0.09	0.08
query28	13.34	1.01	0.99
query29	13.31	3.29	3.26
query30	0.25	0.05	0.06
query31	2.87	0.38	0.38
query32	3.30	0.46	0.47
query33	2.89	2.94	2.86
query34	17.21	4.40	4.48
query35	4.52	4.48	4.66
query36	0.66	0.45	0.46
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.13	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.03	0.04
Total cold run time: 110.99 s
Total hot run time: 30.54 s

doris-robot avatar May 30 '24 11:05 doris-robot

run regression

GoGoWen avatar May 31 '24 02:05 GoGoWen

run buildall

dataroaring avatar Jun 04 '24 02:06 dataroaring

TPC-H: Total hot run time: 40993 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a7f47be7ff730afe16ccf268fa2d60cfa0a19d94, data reload: false

------ Round 1 ----------------------------------
q1	17611	5183	4240	4240
q2	2023	194	187	187
q3	10469	1165	1200	1165
q4	10197	767	848	767
q5	7495	2730	2733	2730
q6	223	133	135	133
q7	974	616	612	612
q8	9228	2137	2112	2112
q9	9314	6765	6798	6765
q10	9726	3931	3901	3901
q11	449	243	256	243
q12	529	258	230	230
q13	17287	3154	3233	3154
q14	271	226	211	211
q15	508	480	463	463
q16	493	388	378	378
q17	993	689	589	589
q18	8478	7877	7767	7767
q19	7472	1483	1467	1467
q20	668	323	317	317
q21	5126	3228	3250	3228
q22	396	350	334	334
Total cold run time: 119930 ms
Total hot run time: 40993 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4535	4427	4457	4427
q2	377	280	269	269
q3	3175	2942	2917	2917
q4	2017	1662	1589	1589
q5	5341	5466	5485	5466
q6	223	125	124	124
q7	2237	1836	1850	1836
q8	3224	3375	3395	3375
q9	8654	8672	8749	8672
q10	4057	3897	3839	3839
q11	596	484	477	477
q12	803	608	624	608
q13	17139	3119	3122	3119
q14	294	276	269	269
q15	545	480	477	477
q16	468	428	432	428
q17	1800	1493	1504	1493
q18	8059	7601	7299	7299
q19	1773	1582	1567	1567
q20	3027	1805	1807	1805
q21	4801	4711	4616	4616
q22	634	539	565	539
Total cold run time: 73779 ms
Total hot run time: 55211 ms

doris-robot avatar Jun 04 '24 03:06 doris-robot

TPC-DS: Total hot run time: 172184 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a7f47be7ff730afe16ccf268fa2d60cfa0a19d94, data reload: false

query1	908	392	374	374
query2	6444	2478	2456	2456
query3	6647	203	202	202
query4	19234	17320	17569	17320
query5	4091	444	449	444
query6	238	153	148	148
query7	4594	292	291	291
query8	302	281	282	281
query9	8460	2383	2362	2362
query10	447	298	276	276
query11	10614	10008	10008	10008
query12	134	89	89	89
query13	1647	369	364	364
query14	10110	7560	6815	6815
query15	224	186	179	179
query16	7812	264	257	257
query17	1725	512	506	506
query18	1954	266	290	266
query19	195	151	154	151
query20	94	84	84	84
query21	208	136	127	127
query22	4291	4010	4184	4010
query23	33542	33008	33072	33008
query24	10791	2749	2755	2749
query25	623	362	355	355
query26	1180	161	153	153
query27	2653	315	312	312
query28	6850	2074	2070	2070
query29	876	620	594	594
query30	269	149	151	149
query31	929	742	750	742
query32	96	50	53	50
query33	770	275	292	275
query34	945	476	464	464
query35	762	624	629	624
query36	1123	962	970	962
query37	152	67	74	67
query38	2853	2739	2699	2699
query39	866	800	780	780
query40	208	124	121	121
query41	56	50	49	49
query42	117	92	100	92
query43	578	545	555	545
query44	1242	714	742	714
query45	193	172	168	168
query46	1079	719	697	697
query47	1844	1775	1766	1766
query48	369	298	291	291
query49	983	397	395	395
query50	772	377	383	377
query51	6794	6757	6726	6726
query52	107	87	89	87
query53	354	283	281	281
query54	873	449	436	436
query55	70	73	71	71
query56	270	290	255	255
query57	1130	1060	1050	1050
query58	242	272	262	262
query59	3442	3205	3427	3205
query60	287	269	267	267
query61	89	89	85	85
query62	608	448	439	439
query63	308	281	280	280
query64	8876	2252	1729	1729
query65	3186	3062	3117	3062
query66	801	314	335	314
query67	15329	14977	15060	14977
query68	4549	539	536	536
query69	495	371	330	330
query70	1181	1102	1092	1092
query71	416	280	277	277
query72	7030	5608	5323	5323
query73	750	317	322	317
query74	5886	5472	5495	5472
query75	3525	2650	2667	2650
query76	2577	887	922	887
query77	571	289	297	289
query78	10243	9951	9695	9695
query79	1951	509	504	504
query80	2061	459	444	444
query81	569	226	214	214
query82	1060	105	99	99
query83	294	165	162	162
query84	261	88	82	82
query85	1156	280	296	280
query86	448	311	336	311
query87	3300	3078	3061	3061
query88	3329	2355	2345	2345
query89	466	398	380	380
query90	1818	190	190	190
query91	197	93	94	93
query92	69	49	48	48
query93	1867	494	496	494
query94	1208	195	183	183
query95	414	314	313	313
query96	599	266	268	266
query97	3170	3018	2972	2972
query98	242	222	219	219
query99	1144	853	852	852
Total cold run time: 268706 ms
Total hot run time: 172184 ms

doris-robot avatar Jun 04 '24 03:06 doris-robot

ClickBench: Total hot run time: 30.94 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a7f47be7ff730afe16ccf268fa2d60cfa0a19d94, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.04	0.04
query4	1.68	0.06	0.07
query5	0.52	0.49	0.48
query6	1.12	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.04	0.05
query9	0.52	0.50	0.48
query10	0.55	0.56	0.54
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.79	0.76	0.78
query15	0.82	0.81	0.81
query16	0.38	0.37	0.38
query17	0.96	0.99	0.95
query18	0.22	0.24	0.23
query19	1.79	1.71	1.80
query20	0.02	0.02	0.01
query21	15.44	0.72	0.67
query22	3.77	6.96	2.32
query23	18.25	1.46	1.31
query24	2.07	0.22	0.20
query25	0.15	0.09	0.08
query26	0.27	0.17	0.18
query27	0.08	0.09	0.07
query28	13.34	1.00	1.00
query29	13.12	3.40	3.30
query30	0.24	0.05	0.06
query31	2.89	0.38	0.38
query32	3.28	0.47	0.47
query33	2.90	2.84	2.82
query34	17.21	4.43	4.44
query35	4.55	4.53	4.63
query36	0.68	0.47	0.46
query37	0.17	0.15	0.16
query38	0.16	0.14	0.14
query39	0.04	0.03	0.03
query40	0.16	0.15	0.14
query41	0.08	0.05	0.04
query42	0.05	0.05	0.05
query43	0.03	0.04	0.03
Total cold run time: 109.59 s
Total hot run time: 30.94 s

doris-robot avatar Jun 04 '24 03:06 doris-robot

TeamCity be ut coverage result: Function Coverage: 36.31% (9268/25528) Line Coverage: 27.61% (76017/275306) Region Coverage: 26.87% (39383/146581) Branch Coverage: 23.59% (19948/84558) Coverage Report: http://coverage.selectdb-in.cc/coverage/a7f47be7ff730afe16ccf268fa2d60cfa0a19d94_a7f47be7ff730afe16ccf268fa2d60cfa0a19d94/report/index.html

doris-robot avatar Jun 04 '24 04:06 doris-robot

PR approved by anyone and no changes requested.

github-actions[bot] avatar Jun 04 '24 06:06 github-actions[bot]

run buildall

dataroaring avatar Jun 05 '24 01:06 dataroaring

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Jun 05 '24 01:06 github-actions[bot]

TeamCity be ut coverage result: Function Coverage: 36.41% (8985/24677) Line Coverage: 27.92% (73375/262836) Region Coverage: 27.36% (37998/138899) Branch Coverage: 23.94% (19282/80534) Coverage Report: http://coverage.selectdb-in.cc/coverage/2ed7ecf22eae0639f948e589f2151ac74b73c956_2ed7ecf22eae0639f948e589f2151ac74b73c956/report/index.html

doris-robot avatar Jun 05 '24 01:06 doris-robot

TPC-H: Total hot run time: 41016 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2ed7ecf22eae0639f948e589f2151ac74b73c956, data reload: false

------ Round 1 ----------------------------------
q1	17628	4348	4255	4255
q2	2038	202	195	195
q3	10420	1254	1221	1221
q4	10188	847	853	847
q5	7577	2714	2717	2714
q6	223	135	141	135
q7	944	598	607	598
q8	9225	2101	2090	2090
q9	9378	6696	6603	6603
q10	9166	3886	3867	3867
q11	488	258	247	247
q12	476	229	246	229
q13	17517	3218	3183	3183
q14	287	227	241	227
q15	534	471	503	471
q16	499	396	409	396
q17	978	724	659	659
q18	8419	7879	7889	7879
q19	7982	1257	1310	1257
q20	678	329	317	317
q21	5028	3950	3282	3282
q22	413	344	354	344
Total cold run time: 120086 ms
Total hot run time: 41016 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4545	4440	4443	4440
q2	372	267	281	267
q3	3138	2947	2862	2862
q4	2010	1705	1594	1594
q5	5337	5524	5500	5500
q6	221	124	130	124
q7	2196	1839	1826	1826
q8	3213	3398	3400	3398
q9	8681	8594	8726	8594
q10	4126	3896	3802	3802
q11	569	484	502	484
q12	796	623	636	623
q13	16113	3094	3164	3094
q14	321	279	268	268
q15	542	461	488	461
q16	479	426	421	421
q17	1838	1515	1497	1497
q18	7962	7612	7231	7231
q19	1718	1624	1609	1609
q20	3280	1792	1754	1754
q21	10457	4734	4651	4651
q22	796	555	573	555
Total cold run time: 78710 ms
Total hot run time: 55055 ms

doris-robot avatar Jun 05 '24 01:06 doris-robot

TPC-DS: Total hot run time: 170799 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2ed7ecf22eae0639f948e589f2151ac74b73c956, data reload: false

query1	931	381	370	370
query2	6458	2436	2536	2436
query3	6646	210	210	210
query4	19662	17172	17310	17172
query5	4127	447	458	447
query6	249	160	156	156
query7	4596	310	300	300
query8	348	295	291	291
query9	8569	2435	2437	2435
query10	442	308	286	286
query11	10538	9968	9957	9957
query12	144	91	87	87
query13	1637	371	366	366
query14	9325	7745	7660	7660
query15	237	188	190	188
query16	7410	265	269	265
query17	1343	523	532	523
query18	1919	269	267	267
query19	193	159	148	148
query20	89	84	82	82
query21	204	130	132	130
query22	4538	4067	3967	3967
query23	33549	33067	33039	33039
query24	11323	2815	2819	2815
query25	618	357	362	357
query26	1164	155	154	154
query27	2330	323	319	319
query28	6959	2104	2099	2099
query29	900	617	591	591
query30	270	151	151	151
query31	952	745	785	745
query32	91	54	53	53
query33	760	278	278	278
query34	997	474	486	474
query35	741	611	617	611
query36	1106	922	947	922
query37	140	70	77	70
query38	2857	2718	2720	2718
query39	867	780	790	780
query40	212	126	123	123
query41	56	51	53	51
query42	126	99	97	97
query43	598	570	570	570
query44	1232	726	740	726
query45	196	169	160	160
query46	1078	715	712	712
query47	1849	1771	1772	1771
query48	382	298	309	298
query49	1100	399	415	399
query50	807	384	396	384
query51	6846	6771	6751	6751
query52	103	93	93	93
query53	355	291	289	289
query54	892	464	440	440
query55	75	71	75	71
query56	284	256	267	256
query57	1146	1031	1068	1031
query58	239	247	242	242
query59	3405	3353	3335	3335
query60	293	277	272	272
query61	95	92	91	91
query62	647	452	451	451
query63	311	287	285	285
query64	9038	2198	1702	1702
query65	3174	3100	3089	3089
query66	902	323	325	323
query67	15237	14913	14961	14913
query68	4530	551	538	538
query69	454	303	299	299
query70	1057	1151	1126	1126
query71	385	279	288	279
query72	7299	5222	2752	2752
query73	742	333	328	328
query74	5926	5456	5502	5456
query75	3325	2603	2671	2603
query76	2335	888	939	888
query77	422	290	293	290
query78	10177	9783	9745	9745
query79	1309	517	517	517
query80	823	457	457	457
query81	597	217	218	217
query82	599	104	100	100
query83	264	169	164	164
query84	245	85	85	85
query85	1193	265	259	259
query86	438	317	315	315
query87	3260	3084	3069	3069
query88	4073	2472	2450	2450
query89	470	387	380	380
query90	1732	190	190	190
query91	198	99	97	97
query92	60	48	52	48
query93	2006	507	497	497
query94	1142	188	186	186
query95	401	322	319	319
query96	596	272	276	272
query97	3181	2998	3032	2998
query98	249	222	215	215
query99	1374	830	816	816
Total cold run time: 266769 ms
Total hot run time: 170799 ms

doris-robot avatar Jun 05 '24 01:06 doris-robot

ClickBench: Total hot run time: 31.09 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2ed7ecf22eae0639f948e589f2151ac74b73c956, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.23	0.05	0.04
query4	1.68	0.07	0.07
query5	0.50	0.50	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.03	0.04
query9	0.54	0.49	0.49
query10	0.54	0.54	0.52
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.78	0.81	0.77
query15	0.81	0.80	0.81
query16	0.37	0.34	0.36
query17	1.00	0.98	0.99
query18	0.23	0.26	0.25
query19	1.80	1.74	1.81
query20	0.01	0.01	0.01
query21	15.61	0.67	0.67
query22	4.03	7.05	2.36
query23	18.65	1.37	1.36
query24	1.64	0.33	0.21
query25	0.14	0.09	0.08
query26	0.26	0.17	0.16
query27	0.07	0.07	0.08
query28	13.27	1.03	0.99
query29	13.24	3.42	3.35
query30	0.24	0.06	0.05
query31	2.87	0.39	0.38
query32	3.31	0.46	0.48
query33	2.93	2.88	2.92
query34	17.13	4.50	4.39
query35	4.51	4.50	4.59
query36	0.70	0.52	0.45
query37	0.17	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.17	0.16	0.14
query41	0.09	0.04	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.03
Total cold run time: 110.03 s
Total hot run time: 31.09 s

doris-robot avatar Jun 05 '24 02:06 doris-robot

run buildall

GoGoWen avatar Jun 11 '24 07:06 GoGoWen

clang-tidy review says "All clean, LGTM! :+1:"

github-actions[bot] avatar Jun 11 '24 07:06 github-actions[bot]

TeamCity be ut coverage result: Function Coverage: 36.42% (8982/24660) Line Coverage: 28.00% (73511/262559) Region Coverage: 27.39% (38069/139010) Branch Coverage: 24.01% (19333/80526) Coverage Report: http://coverage.selectdb-in.cc/coverage/048275fe38ec9e2278938a182fd29d372c407391_048275fe38ec9e2278938a182fd29d372c407391/report/index.html

doris-robot avatar Jun 11 '24 07:06 doris-robot

TPC-H: Total hot run time: 39290 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 048275fe38ec9e2278938a182fd29d372c407391, data reload: false

------ Round 1 ----------------------------------
q1	17594	4301	4253	4253
q2	2020	189	188	188
q3	10471	1136	978	978
q4	10198	739	784	739
q5	7481	2675	2660	2660
q6	224	137	137	137
q7	939	605	593	593
q8	9221	2051	2026	2026
q9	8795	6482	6451	6451
q10	8931	3740	3723	3723
q11	463	233	236	233
q12	440	232	223	223
q13	18825	2978	2948	2948
q14	274	212	215	212
q15	513	487	467	467
q16	519	369	377	369
q17	956	678	661	661
q18	8042	7433	7297	7297
q19	6190	1384	1378	1378
q20	659	309	299	299
q21	4881	3133	3832	3133
q22	380	322	332	322
Total cold run time: 118016 ms
Total hot run time: 39290 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4378	4263	4236	4236
q2	362	256	267	256
q3	2942	2910	2874	2874
q4	2011	1677	1714	1677
q5	5498	5406	5430	5406
q6	223	125	133	125
q7	2150	1935	1827	1827
q8	3288	3377	3436	3377
q9	8633	8667	8773	8667
q10	4081	3913	3652	3652
q11	577	499	506	499
q12	801	651	616	616
q13	17210	3163	3158	3158
q14	299	277	254	254
q15	529	480	490	480
q16	489	427	440	427
q17	1771	1508	1491	1491
q18	8127	7951	7848	7848
q19	1781	1611	1568	1568
q20	3152	1914	1844	1844
q21	6913	4864	4728	4728
q22	643	524	551	524
Total cold run time: 75858 ms
Total hot run time: 55534 ms

doris-robot avatar Jun 11 '24 07:06 doris-robot

TPC-DS: Total hot run time: 174964 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 048275fe38ec9e2278938a182fd29d372c407391, data reload: false

query1	912	382	369	369
query2	6449	2429	2343	2343
query3	6643	201	204	201
query4	19150	17330	17360	17330
query5	3611	463	448	448
query6	236	168	156	156
query7	4573	308	293	293
query8	310	288	289	288
query9	8528	2388	2370	2370
query10	564	306	268	268
query11	10827	10301	10224	10224
query12	126	86	86	86
query13	1637	358	358	358
query14	9664	7639	7660	7639
query15	231	196	187	187
query16	7788	267	258	258
query17	1493	527	510	510
query18	1953	270	263	263
query19	190	145	145	145
query20	87	83	81	81
query21	212	129	125	125
query22	4564	4251	4462	4251
query23	33624	33626	33630	33626
query24	10124	2940	2874	2874
query25	593	388	366	366
query26	722	153	154	153
query27	2347	319	322	319
query28	5880	2127	2117	2117
query29	883	636	658	636
query30	241	155	154	154
query31	974	765	788	765
query32	97	57	81	57
query33	646	280	274	274
query34	897	490	470	470
query35	715	650	621	621
query36	1116	990	967	967
query37	151	72	67	67
query38	2992	2835	2879	2835
query39	882	842	833	833
query40	208	127	121	121
query41	53	52	57	52
query42	112	108	115	108
query43	595	565	559	559
query44	1089	723	747	723
query45	191	157	159	157
query46	1088	691	689	689
query47	1863	1799	1827	1799
query48	372	302	291	291
query49	833	405	414	405
query50	753	377	374	374
query51	6825	6648	6693	6648
query52	95	96	88	88
query53	355	286	286	286
query54	855	430	434	430
query55	73	70	70	70
query56	266	250	262	250
query57	1148	1064	1068	1064
query58	249	251	260	251
query59	3506	3147	3384	3147
query60	294	262	267	262
query61	86	85	83	83
query62	606	445	449	445
query63	308	284	286	284
query64	8485	2173	1801	1801
query65	3198	3089	3090	3089
query66	754	334	336	334
query67	15236	15148	14749	14749
query68	4546	534	535	534
query69	593	457	370	370
query70	1172	1081	1115	1081
query71	389	280	269	269
query72	7553	5660	5649	5649
query73	758	322	324	322
query74	5932	5565	5585	5565
query75	3343	2655	2648	2648
query76	2180	992	951	951
query77	464	297	291	291
query78	10432	9968	9777	9777
query79	2499	513	505	505
query80	2696	484	470	470
query81	573	223	237	223
query82	791	109	103	103
query83	329	168	174	168
query84	264	90	88	88
query85	1112	323	311	311
query86	450	276	315	276
query87	3280	3123	3091	3091
query88	3578	2406	2366	2366
query89	486	376	371	371
query90	1648	185	186	185
query91	137	108	109	108
query92	66	49	51	49
query93	2105	496	488	488
query94	1106	192	188	188
query95	405	316	308	308
query96	606	269	267	267
query97	3317	3071	3020	3020
query98	231	198	206	198
query99	1180	838	838	838
Total cold run time: 266349 ms
Total hot run time: 174964 ms

doris-robot avatar Jun 11 '24 07:06 doris-robot

ClickBench: Total hot run time: 30.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 048275fe38ec9e2278938a182fd29d372c407391, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.05
query4	1.68	0.07	0.06
query5	0.48	0.50	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.04
query9	0.54	0.49	0.49
query10	0.54	0.56	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.58	0.61
query14	0.77	0.78	0.78
query15	0.83	0.84	0.81
query16	0.34	0.35	0.37
query17	1.05	1.04	0.96
query18	0.22	0.26	0.23
query19	1.90	1.81	1.71
query20	0.01	0.01	0.01
query21	15.41	0.64	0.63
query22	4.14	7.74	1.78
query23	18.29	1.39	1.28
query24	2.10	0.25	0.21
query25	0.16	0.09	0.09
query26	0.27	0.17	0.17
query27	0.08	0.08	0.07
query28	13.21	1.02	1.00
query29	12.62	3.33	3.25
query30	0.26	0.06	0.06
query31	2.86	0.40	0.39
query32	3.25	0.47	0.46
query33	2.89	2.94	2.86
query34	17.14	4.43	4.41
query35	4.49	4.55	4.46
query36	0.64	0.44	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.17	0.14	0.15
query41	0.09	0.05	0.04
query42	0.05	0.04	0.05
query43	0.04	0.03	0.03
Total cold run time: 109.34 s
Total hot run time: 30.28 s

doris-robot avatar Jun 11 '24 08:06 doris-robot

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Jun 26 '24 12:06 github-actions[bot]