doris icon indicating copy to clipboard operation
doris copied to clipboard

[fix](group commit) add bvar for group commit block by memory counter

Open mymeiyi opened this issue 7 months ago • 8 comments

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • [ ] Regression test
    • [ ] Unit Test
    • [ ] Manual test (add detailed scripts or steps below)
    • [ ] No need to test or manual test. Explain why:
      • [ ] This is a refactor/code format and no logic has been changed.
      • [ ] Previous test can cover this change.
      • [ ] No code files have been changed.
      • [ ] Other reason
  • Behavior changed:

    • [ ] No.
    • [ ] Yes.
  • Does this need documentation?

    • [ ] No.
    • [ ] Yes.

Check List (For Reviewer who merge this PR)

  • [ ] Confirm the release note
  • [ ] Confirm test cases
  • [ ] Confirm document
  • [ ] Add branch pick label

mymeiyi avatar Jun 18 '25 04:06 mymeiyi

Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Thearas avatar Jun 18 '25 04:06 Thearas

run buildall

mymeiyi avatar Jun 18 '25 06:06 mymeiyi

TPC-H: Total hot run time: 33582 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5a5cc47b00315c468ba711db331e25ddc8a2172, data reload: false

------ Round 1 ----------------------------------
q1	17598	5122	4978	4978
q2	1943	281	170	170
q3	10313	1259	754	754
q4	10196	1027	506	506
q5	7512	2284	2349	2284
q6	173	167	131	131
q7	910	751	623	623
q8	9328	1272	1035	1035
q9	6770	5054	5090	5054
q10	6891	2387	1964	1964
q11	484	288	286	286
q12	354	344	212	212
q13	17766	3669	3031	3031
q14	225	251	213	213
q15	568	485	475	475
q16	433	429	388	388
q17	600	847	377	377
q18	7608	7064	7126	7064
q19	1239	962	547	547
q20	332	355	225	225
q21	3672	2563	2327	2327
q22	1009	982	938	938
Total cold run time: 105924 ms
Total hot run time: 33582 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5004	5025	5029	5025
q2	235	317	223	223
q3	2160	2653	2272	2272
q4	1371	1764	1349	1349
q5	4240	4125	4248	4125
q6	228	177	144	144
q7	2051	1996	1781	1781
q8	2601	2616	2552	2552
q9	7337	7293	7234	7234
q10	3062	3260	2790	2790
q11	574	495	517	495
q12	718	778	602	602
q13	3494	3838	3245	3245
q14	289	293	269	269
q15	524	492	477	477
q16	454	488	433	433
q17	1152	1548	1362	1362
q18	7757	7460	7362	7362
q19	784	780	835	780
q20	1996	2085	1935	1935
q21	4926	4505	4317	4317
q22	1092	1048	1012	1012
Total cold run time: 52049 ms
Total hot run time: 49784 ms

doris-robot avatar Jun 18 '25 06:06 doris-robot

TPC-DS: Total hot run time: 192613 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5a5cc47b00315c468ba711db331e25ddc8a2172, data reload: false

query1	1383	1023	983	983
query2	6110	1775	1810	1775
query3	11001	4635	4444	4444
query4	53428	24450	23045	23045
query5	5134	517	455	455
query6	359	204	193	193
query7	4882	505	276	276
query8	272	225	208	208
query9	5533	2705	2687	2687
query10	444	342	275	275
query11	15019	15115	15174	15115
query12	158	111	111	111
query13	1038	530	423	423
query14	10196	6463	6604	6463
query15	199	197	181	181
query16	7108	665	512	512
query17	1094	741	610	610
query18	1590	444	357	357
query19	212	204	177	177
query20	135	128	119	119
query21	211	124	109	109
query22	4312	4414	4265	4265
query23	34450	33485	33397	33397
query24	6656	2469	2469	2469
query25	466	465	416	416
query26	710	273	156	156
query27	2295	527	356	356
query28	2973	2180	2177	2177
query29	582	580	430	430
query30	280	232	193	193
query31	881	879	777	777
query32	74	65	63	63
query33	439	358	349	349
query34	792	915	538	538
query35	771	844	741	741
query36	930	1009	884	884
query37	112	107	81	81
query38	4244	4264	4252	4252
query39	1513	1451	1456	1451
query40	206	121	108	108
query41	63	62	60	60
query42	128	117	115	115
query43	497	526	456	456
query44	1320	835	844	835
query45	180	174	170	170
query46	858	1125	666	666
query47	1840	1854	1767	1767
query48	391	438	328	328
query49	643	480	406	406
query50	659	693	409	409
query51	4285	4315	4302	4302
query52	118	114	107	107
query53	229	258	192	192
query54	587	604	524	524
query55	86	86	84	84
query56	312	320	286	286
query57	1199	1236	1190	1190
query58	298	289	262	262
query59	2699	2829	2662	2662
query60	335	337	303	303
query61	144	127	123	123
query62	736	790	698	698
query63	229	195	190	190
query64	1818	1067	695	695
query65	4256	4128	4138	4128
query66	726	389	331	331
query67	15941	15830	15420	15420
query68	7486	937	539	539
query69	543	302	279	279
query70	1249	1118	1107	1107
query71	522	320	299	299
query72	6129	4834	4877	4834
query73	1441	673	353	353
query74	9254	9092	8648	8648
query75	3838	3198	2709	2709
query76	4243	1203	758	758
query77	715	388	292	292
query78	10069	10003	9336	9336
query79	3243	851	579	579
query80	642	529	452	452
query81	507	260	221	221
query82	487	128	94	94
query83	366	254	240	240
query84	294	108	82	82
query85	807	358	356	356
query86	387	295	295	295
query87	4449	4488	4305	4305
query88	3534	2281	2277	2277
query89	422	330	284	284
query90	1915	215	220	215
query91	146	145	114	114
query92	80	61	58	58
query93	2469	982	593	593
query94	672	405	285	285
query95	374	301	286	286
query96	492	577	285	285
query97	2723	2746	2624	2624
query98	241	204	209	204
query99	1410	1399	1280	1280
Total cold run time: 299918 ms
Total hot run time: 192613 ms

doris-robot avatar Jun 18 '25 06:06 doris-robot

ClickBench: Total hot run time: 29.83 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5a5cc47b00315c468ba711db331e25ddc8a2172, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.06	0.07
query4	1.61	0.10	0.10
query5	0.42	0.42	0.42
query6	1.16	0.65	0.66
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.58	0.52	0.51
query10	0.58	0.59	0.57
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.63	0.61	0.61
query14	0.80	0.83	0.81
query15	0.88	0.86	0.89
query16	0.38	0.39	0.38
query17	1.07	1.08	1.09
query18	0.24	0.21	0.21
query19	1.91	1.86	1.88
query20	0.02	0.02	0.01
query21	15.40	0.89	0.58
query22	0.76	1.27	0.62
query23	14.90	1.34	0.63
query24	6.78	1.94	1.08
query25	0.47	0.22	0.15
query26	0.62	0.16	0.14
query27	0.07	0.05	0.06
query28	9.88	0.83	0.43
query29	12.59	4.01	3.28
query30	0.25	0.09	0.07
query31	2.83	0.60	0.39
query32	3.24	0.56	0.47
query33	3.02	3.16	3.09
query34	16.18	5.36	4.80
query35	4.86	4.78	4.80
query36	0.70	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.14
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 104.01 s
Total hot run time: 29.83 s

doris-robot avatar Jun 18 '25 06:06 doris-robot

BE UT Coverage Report

Increment line coverage 0.00% (0/1) :tada:

Increment coverage report Complete coverage report

Category Coverage
Function Coverage 56.34% (15050/26712)
Line Coverage 45.11% (134599/298370)
Region Coverage 44.25% (67693/152982)
Branch Coverage 38.83% (34728/89440)

hello-stephen avatar Jun 18 '25 08:06 hello-stephen

PR approved by at least one committer and no changes requested.

github-actions[bot] avatar Jun 18 '25 12:06 github-actions[bot]

PR approved by anyone and no changes requested.

github-actions[bot] avatar Jun 18 '25 12:06 github-actions[bot]