doris
doris copied to clipboard
apache
[Feature](iceberg-writer) Implements iceberg sink basic functionality for inserting into table.
Proposed changes
#31442
[Feature] (iceberg-writer) Implements iceberg sink basic functionality for inserting into table.
Currently only insert into and CTAS are supported. Among them, insert into supports unpartitioned tables and partitioned tables, but partitioned tables only support identity transform partitioned tables.
Future work:
- Complete partitioned table transform functionality.
- Sorting function.
- Insert overwrite function.
- More features like update, delete and merge.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TeamCity be ut coverage result: Function Coverage: 35.83% (9124/25463) Line Coverage: 27.35% (74876/273809) Region Coverage: 26.62% (38754/145595) Branch Coverage: 23.41% (19732/84280) Coverage Report: http://coverage.selectdb-in.cc/coverage/b59d26848f5c67e38796fb4f24b75a5fe91a3788_b59d26848f5c67e38796fb4f24b75a5fe91a3788/report/index.html
TPC-H: Total hot run time: 42040 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 63e6b031f9a60e878533ab8a052b64f337185a8d, data reload: false
------ Round 1 ----------------------------------
q1 17604 4422 4257 4257
q2 2035 186 191 186
q3 10459 1285 1219 1219
q4 10192 813 837 813
q5 7479 2735 2775 2735
q6 226 132 130 130
q7 1000 615 605 605
q8 9222 2148 2136 2136
q9 9260 6722 6763 6722
q10 9486 3967 3900 3900
q11 451 237 242 237
q12 450 224 225 224
q13 17214 3149 3297 3149
q14 276 242 231 231
q15 513 464 476 464
q16 480 377 387 377
q17 977 725 722 722
q18 8586 8018 7747 7747
q19 5028 1600 1558 1558
q20 634 322 303 303
q21 5189 4047 4093 4047
q22 355 291 278 278
Total cold run time: 117116 ms
Total hot run time: 42040 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4666 4381 4437 4381
q2 369 266 276 266
q3 3101 2981 2903 2903
q4 1896 1601 1585 1585
q5 5476 5506 5487 5487
q6 216 126 124 124
q7 2185 1832 1824 1824
q8 3235 3402 3395 3395
q9 8702 8691 8681 8681
q10 3964 3742 3771 3742
q11 582 496 497 496
q12 828 621 632 621
q13 15903 3173 3164 3164
q14 303 286 267 267
q15 524 485 486 485
q16 496 431 432 431
q17 1799 1543 1462 1462
q18 7848 7790 7495 7495
q19 1745 1580 1559 1559
q20 2062 1788 1822 1788
q21 15229 4709 4659 4659
q22 555 495 488 488
Total cold run time: 81684 ms
Total hot run time: 55303 ms
TeamCity be ut coverage result: Function Coverage: 35.79% (9129/25509) Line Coverage: 27.32% (74947/274371) Region Coverage: 26.57% (38831/146129) Branch Coverage: 23.39% (19763/84488) Coverage Report: http://coverage.selectdb-in.cc/coverage/63e6b031f9a60e878533ab8a052b64f337185a8d_63e6b031f9a60e878533ab8a052b64f337185a8d/report/index.html
TPC-DS: Total hot run time: 168709 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 63e6b031f9a60e878533ab8a052b64f337185a8d, data reload: false
query1 903 377 362 362
query2 6431 2345 2264 2264
query3 6634 217 210 210
query4 19163 17064 17458 17064
query5 4105 416 419 416
query6 257 153 156 153
query7 4584 296 290 290
query8 233 188 185 185
query9 8530 2361 2358 2358
query10 470 275 262 262
query11 10597 10100 10011 10011
query12 134 91 84 84
query13 1636 355 355 355
query14 9462 6791 6942 6791
query15 212 165 168 165
query16 7125 261 263 261
query17 1023 524 518 518
query18 1903 266 269 266
query19 203 160 151 151
query20 89 90 85 85
query21 190 139 133 133
query22 4276 3964 3759 3759
query23 33641 33182 32938 32938
query24 6996 2849 2865 2849
query25 532 352 359 352
query26 709 175 155 155
query27 1929 325 323 323
query28 3847 2052 2042 2042
query29 854 600 610 600
query30 244 175 178 175
query31 961 772 726 726
query32 80 50 54 50
query33 536 262 256 256
query34 855 484 463 463
query35 707 621 618 618
query36 1012 910 909 909
query37 108 69 71 69
query38 2891 2790 2761 2761
query39 828 795 784 784
query40 194 127 122 122
query41 45 44 43 43
query42 100 96 93 93
query43 581 557 536 536
query44 1076 710 737 710
query45 176 157 161 157
query46 1061 711 711 711
query47 1847 1768 1774 1768
query48 368 295 290 290
query49 793 380 375 375
query50 767 379 388 379
query51 6960 6658 6854 6658
query52 100 92 90 90
query53 352 281 282 281
query54 578 430 449 430
query55 76 70 71 70
query56 260 231 243 231
query57 1101 1058 1044 1044
query58 219 214 202 202
query59 3369 3337 3174 3174
query60 272 250 255 250
query61 86 87 89 87
query62 564 461 440 440
query63 306 286 297 286
query64 8390 2313 1809 1809
query65 3186 3077 3104 3077
query66 914 332 323 323
query67 14989 14961 15129 14961
query68 4612 535 537 535
query69 445 269 268 268
query70 1125 1089 1094 1089
query71 395 272 271 271
query72 7845 5463 2734 2734
query73 710 320 321 320
query74 5996 5644 5540 5540
query75 3284 2614 2639 2614
query76 2328 914 1028 914
query77 394 263 266 263
query78 10238 9778 9807 9778
query79 2139 512 511 511
query80 1015 433 432 432
query81 525 243 238 238
query82 958 94 93 93
query83 251 170 171 170
query84 266 85 92 85
query85 1025 260 261 260
query86 498 303 301 301
query87 3299 3143 3100 3100
query88 3619 2343 2335 2335
query89 475 373 370 370
query90 2004 196 192 192
query91 128 99 157 99
query92 63 50 48 48
query93 2046 503 497 497
query94 1148 184 187 184
query95 409 313 305 305
query96 582 261 264 261
query97 3160 3022 3024 3022
query98 247 216 213 213
query99 1189 857 857 857
Total cold run time: 256038 ms
Total hot run time: 168709 ms
ClickBench: Total hot run time: 30.58 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 63e6b031f9a60e878533ab8a052b64f337185a8d, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.05
query4 1.67 0.06 0.07
query5 0.50 0.49 0.50
query6 1.12 0.72 0.72
query7 0.01 0.01 0.02
query8 0.05 0.04 0.04
query9 0.53 0.49 0.49
query10 0.54 0.56 0.54
query11 0.15 0.12 0.11
query12 0.14 0.12 0.12
query13 0.60 0.59 0.60
query14 0.77 0.77 0.78
query15 0.83 0.80 0.82
query16 0.37 0.37 0.37
query17 0.91 1.04 0.99
query18 0.22 0.24 0.23
query19 1.82 1.64 1.68
query20 0.01 0.01 0.01
query21 15.44 0.69 0.68
query22 4.70 6.17 2.04
query23 18.23 1.37 1.26
query24 1.53 0.29 0.21
query25 0.16 0.09 0.08
query26 0.25 0.16 0.16
query27 0.08 0.08 0.08
query28 13.47 1.02 0.99
query29 13.30 3.26 3.27
query30 0.24 0.06 0.06
query31 2.89 0.38 0.38
query32 3.27 0.47 0.46
query33 2.83 2.88 2.88
query34 17.17 4.42 4.35
query35 4.55 4.55 4.71
query36 0.65 0.45 0.49
query37 0.17 0.17 0.16
query38 0.15 0.16 0.14
query39 0.05 0.03 0.03
query40 0.16 0.14 0.16
query41 0.09 0.05 0.04
query42 0.06 0.04 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.07 s
Total hot run time: 30.58 s
TPC-H: Total hot run time: 40827 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 087ef6b7feea6879e851a181cd0dee020bfc93d4, data reload: false
------ Round 1 ----------------------------------
q1 17597 4492 4243 4243
q2 2023 196 198 196
q3 10472 1240 1159 1159
q4 10197 819 843 819
q5 7494 2722 2695 2695
q6 226 130 128 128
q7 968 598 615 598
q8 9222 2134 2086 2086
q9 9308 6645 6700 6645
q10 9201 3823 3872 3823
q11 466 244 245 244
q12 487 231 222 222
q13 17862 3236 3187 3187
q14 270 220 217 217
q15 524 479 479 479
q16 493 404 385 385
q17 964 723 724 723
q18 8384 8018 7668 7668
q19 6299 1576 1527 1527
q20 646 311 311 311
q21 5258 3201 4068 3201
q22 348 271 285 271
Total cold run time: 118709 ms
Total hot run time: 40827 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4452 4408 4327 4327
q2 387 273 262 262
q3 3180 2895 2950 2895
q4 2004 1680 1587 1587
q5 5306 5498 5515 5498
q6 218 123 122 122
q7 2200 1874 1811 1811
q8 3220 3386 3380 3380
q9 8632 8641 8741 8641
q10 4094 3877 3700 3700
q11 594 487 512 487
q12 786 617 628 617
q13 16440 3142 3176 3142
q14 314 265 279 265
q15 507 487 473 473
q16 479 439 437 437
q17 1817 1529 1536 1529
q18 7853 7660 7392 7392
q19 1661 1552 1558 1552
q20 1998 1777 1768 1768
q21 8531 4700 4736 4700
q22 565 466 491 466
Total cold run time: 75238 ms
Total hot run time: 55051 ms
TeamCity be ut coverage result: Function Coverage: 35.91% (9122/25399) Line Coverage: 27.38% (74927/273622) Region Coverage: 26.62% (38793/145713) Branch Coverage: 23.46% (19741/84148) Coverage Report: http://coverage.selectdb-in.cc/coverage/087ef6b7feea6879e851a181cd0dee020bfc93d4_087ef6b7feea6879e851a181cd0dee020bfc93d4/report/index.html
TPC-DS: Total hot run time: 172807 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 087ef6b7feea6879e851a181cd0dee020bfc93d4, data reload: false
query1 936 371 367 367
query2 6456 2509 2418 2418
query3 6639 213 214 213
query4 19475 17257 17148 17148
query5 4154 422 415 415
query6 253 156 150 150
query7 4586 295 284 284
query8 235 175 178 175
query9 8590 2398 2392 2392
query10 462 304 270 270
query11 10603 10053 10028 10028
query12 139 92 86 86
query13 1661 389 363 363
query14 9256 7519 7585 7519
query15 205 168 171 168
query16 7813 258 255 255
query17 1739 517 512 512
query18 1953 274 271 271
query19 193 153 149 149
query20 93 87 82 82
query21 194 133 129 129
query22 4294 3962 3847 3847
query23 33947 33235 33118 33118
query24 10603 2853 2919 2853
query25 562 351 367 351
query26 703 150 150 150
query27 2254 307 346 307
query28 5873 2048 2049 2048
query29 854 597 599 597
query30 220 145 153 145
query31 974 759 769 759
query32 94 57 54 54
query33 663 266 263 263
query34 850 472 477 472
query35 714 630 592 592
query36 1099 929 944 929
query37 108 64 70 64
query38 2893 2788 2747 2747
query39 828 783 811 783
query40 206 125 127 125
query41 48 45 50 45
query42 103 119 96 96
query43 583 562 543 543
query44 1054 718 746 718
query45 181 162 158 158
query46 1059 710 705 705
query47 1840 1770 1788 1770
query48 365 295 306 295
query49 848 378 380 378
query50 772 377 372 372
query51 6926 6882 6776 6776
query52 113 86 88 86
query53 345 279 276 276
query54 819 423 422 422
query55 78 74 73 73
query56 254 235 237 235
query57 1101 1039 1056 1039
query58 231 210 219 210
query59 3388 3137 3154 3137
query60 273 254 279 254
query61 90 84 85 84
query62 617 446 449 446
query63 312 279 281 279
query64 8514 2243 1722 1722
query65 3206 3111 3113 3111
query66 783 336 319 319
query67 15140 14811 15012 14811
query68 4530 537 535 535
query69 434 259 268 259
query70 1174 1144 1111 1111
query71 364 270 256 256
query72 7586 5356 5332 5332
query73 716 323 318 318
query74 6134 5566 5595 5566
query75 3316 2631 2602 2602
query76 2268 1029 1061 1029
query77 386 267 265 265
query78 10311 9961 10033 9961
query79 2381 519 516 516
query80 1021 430 434 430
query81 543 218 225 218
query82 688 95 90 90
query83 242 173 171 171
query84 250 86 85 85
query85 1534 290 282 282
query86 487 308 293 293
query87 3312 3131 3121 3121
query88 4211 2370 2414 2370
query89 475 405 391 391
query90 2027 192 184 184
query91 124 97 98 97
query92 62 47 48 47
query93 1915 489 490 489
query94 1234 187 186 186
query95 397 315 307 307
query96 583 262 266 262
query97 3211 2998 3011 2998
query98 246 222 217 217
query99 1157 861 881 861
Total cold run time: 265736 ms
Total hot run time: 172807 ms
ClickBench: Total hot run time: 30.15 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 087ef6b7feea6879e851a181cd0dee020bfc93d4, data reload: false
query1 0.04 0.04 0.03
query2 0.09 0.05 0.04
query3 0.23 0.05 0.05
query4 1.66 0.06 0.07
query5 0.50 0.48 0.50
query6 1.13 0.72 0.74
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.54 0.49 0.50
query10 0.55 0.56 0.55
query11 0.15 0.11 0.11
query12 0.14 0.12 0.12
query13 0.60 0.59 0.61
query14 0.79 0.79 0.76
query15 0.83 0.82 0.81
query16 0.37 0.36 0.37
query17 1.03 0.97 0.96
query18 0.23 0.23 0.25
query19 1.80 1.69 1.74
query20 0.01 0.01 0.01
query21 15.49 0.71 0.69
query22 4.32 8.11 1.45
query23 18.28 1.38 1.31
query24 1.55 0.39 0.21
query25 0.15 0.09 0.08
query26 0.28 0.16 0.17
query27 0.08 0.08 0.08
query28 13.29 1.02 1.01
query29 13.24 3.32 3.24
query30 0.24 0.06 0.06
query31 2.88 0.38 0.38
query32 3.26 0.46 0.47
query33 2.83 2.92 2.94
query34 17.09 4.41 4.43
query35 4.52 4.51 4.68
query36 0.65 0.46 0.46
query37 0.17 0.16 0.16
query38 0.16 0.15 0.14
query39 0.04 0.04 0.03
query40 0.16 0.14 0.15
query41 0.09 0.05 0.05
query42 0.07 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 109.64 s
Total hot run time: 30.15 s
TeamCity be ut coverage result: Function Coverage: 35.91% (9122/25399) Line Coverage: 27.38% (74922/273626) Region Coverage: 26.63% (38797/145716) Branch Coverage: 23.46% (19738/84150) Coverage Report: http://coverage.selectdb-in.cc/coverage/35b487dda933e3366e1c0f2d246511c5830c14e6_35b487dda933e3366e1c0f2d246511c5830c14e6/report/index.html
TPC-H: Total hot run time: 41964 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 35b487dda933e3366e1c0f2d246511c5830c14e6, data reload: false
------ Round 1 ----------------------------------
q1 17605 4432 4231 4231
q2 2026 198 193 193
q3 10427 1243 1221 1221
q4 10210 894 896 894
q5 7493 2694 2699 2694
q6 232 134 137 134
q7 951 617 612 612
q8 9221 2123 2095 2095
q9 9329 6694 6685 6685
q10 9078 3853 3857 3853
q11 456 246 241 241
q12 408 235 228 228
q13 17382 3220 3225 3220
q14 270 219 240 219
q15 537 473 479 473
q16 518 395 407 395
q17 992 719 714 714
q18 8324 7827 7728 7728
q19 6246 1571 1532 1532
q20 659 321 314 314
q21 5123 4013 4045 4013
q22 360 275 281 275
Total cold run time: 117847 ms
Total hot run time: 41964 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4489 4472 4482 4472
q2 383 274 266 266
q3 3170 2936 2922 2922
q4 1995 1711 1596 1596
q5 5295 5469 5495 5469
q6 217 129 128 128
q7 2193 1862 1857 1857
q8 3267 3373 3401 3373
q9 8673 8639 8625 8625
q10 4075 3916 3686 3686
q11 589 470 475 470
q12 795 613 630 613
q13 16958 3082 3154 3082
q14 293 279 273 273
q15 527 493 484 484
q16 507 448 449 448
q17 1842 1527 1484 1484
q18 7948 7564 7345 7345
q19 1678 1569 1566 1566
q20 2014 1812 1794 1794
q21 12239 4634 4777 4634
q22 559 480 478 478
Total cold run time: 79706 ms
Total hot run time: 55065 ms
TPC-DS: Total hot run time: 169261 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 35b487dda933e3366e1c0f2d246511c5830c14e6, data reload: false
query1 942 391 368 368
query2 6965 2460 2355 2355
query3 6643 217 221 217
query4 19809 17476 17390 17390
query5 4126 430 429 429
query6 254 154 175 154
query7 4581 299 297 297
query8 234 184 186 184
query9 8495 2390 2384 2384
query10 447 295 265 265
query11 10603 10153 10022 10022
query12 135 94 91 91
query13 1639 395 358 358
query14 10093 7556 6661 6661
query15 225 173 169 169
query16 7725 260 252 252
query17 1796 527 504 504
query18 1949 269 268 268
query19 196 158 152 152
query20 93 91 84 84
query21 192 145 123 123
query22 4202 4037 3977 3977
query23 33797 32901 32968 32901
query24 10459 2900 2804 2804
query25 547 355 357 355
query26 705 156 154 154
query27 2232 322 329 322
query28 5918 2036 2062 2036
query29 872 619 597 597
query30 261 150 149 149
query31 946 763 740 740
query32 90 51 51 51
query33 643 266 255 255
query34 852 483 479 479
query35 766 613 604 604
query36 1067 903 893 893
query37 103 70 65 65
query38 2857 2794 2770 2770
query39 834 819 774 774
query40 200 130 131 130
query41 49 47 49 47
query42 105 100 100 100
query43 588 567 555 555
query44 1096 735 752 735
query45 179 162 160 160
query46 1076 727 720 720
query47 1872 1780 1790 1780
query48 374 282 299 282
query49 858 383 383 383
query50 784 378 382 378
query51 6934 6796 6854 6796
query52 111 90 91 90
query53 353 289 284 284
query54 845 424 438 424
query55 75 74 76 74
query56 259 241 249 241
query57 1135 1051 1054 1051
query58 240 214 211 211
query59 3329 3308 3163 3163
query60 274 258 257 257
query61 93 91 89 89
query62 618 442 452 442
query63 310 280 286 280
query64 8497 2219 1720 1720
query65 3156 3073 3128 3073
query66 786 343 322 322
query67 15343 14860 14965 14860
query68 4578 544 520 520
query69 447 268 267 267
query70 1137 1149 1148 1148
query71 391 278 274 274
query72 7747 5649 2699 2699
query73 710 324 322 322
query74 6145 5623 5558 5558
query75 3317 2647 2688 2647
query76 2596 998 873 873
query77 472 267 266 266
query78 10178 10031 9750 9750
query79 1384 517 500 500
query80 846 464 457 457
query81 537 226 222 222
query82 351 95 93 93
query83 243 187 170 170
query84 236 89 86 86
query85 1109 270 261 261
query86 432 302 297 297
query87 3359 3124 3157 3124
query88 3560 2353 2333 2333
query89 470 397 371 371
query90 2019 191 196 191
query91 138 109 165 109
query92 62 49 54 49
query93 1774 504 491 491
query94 1224 197 189 189
query95 397 313 312 312
query96 589 264 269 264
query97 3150 3076 3054 3054
query98 237 215 224 215
query99 1147 852 849 849
Total cold run time: 264814 ms
Total hot run time: 169261 ms
ClickBench: Total hot run time: 31.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 35b487dda933e3366e1c0f2d246511c5830c14e6, data reload: false
query1 0.04 0.04 0.02
query2 0.08 0.04 0.04
query3 0.24 0.05 0.05
query4 1.66 0.09 0.08
query5 0.55 0.49 0.51
query6 1.13 0.72 0.72
query7 0.02 0.01 0.01
query8 0.04 0.04 0.05
query9 0.54 0.49 0.50
query10 0.53 0.55 0.55
query11 0.15 0.12 0.11
query12 0.14 0.12 0.11
query13 0.59 0.60 0.61
query14 0.77 0.78 0.80
query15 0.82 0.82 0.81
query16 0.35 0.34 0.37
query17 1.01 1.00 0.93
query18 0.20 0.24 0.25
query19 1.77 1.71 1.75
query20 0.01 0.00 0.01
query21 15.57 0.68 0.66
query22 4.47 6.26 2.51
query23 18.32 1.40 1.30
query24 1.63 0.38 0.21
query25 0.13 0.08 0.08
query26 0.26 0.17 0.17
query27 0.09 0.08 0.08
query28 13.30 1.02 1.00
query29 13.30 3.35 3.28
query30 0.24 0.05 0.05
query31 2.86 0.38 0.38
query32 3.27 0.48 0.47
query33 2.87 2.95 2.90
query34 17.24 4.46 4.39
query35 4.54 4.48 4.61
query36 0.69 0.46 0.46
query37 0.17 0.16 0.15
query38 0.15 0.14 0.14
query39 0.05 0.03 0.03
query40 0.16 0.15 0.14
query41 0.09 0.05 0.04
query42 0.05 0.04 0.05
query43 0.05 0.03 0.03
Total cold run time: 110.14 s
Total hot run time: 31.11 s
TeamCity be ut coverage result: Function Coverage: 35.91% (9120/25399) Line Coverage: 27.38% (74913/273626) Region Coverage: 26.62% (38785/145716) Branch Coverage: 23.45% (19735/84150) Coverage Report: http://coverage.selectdb-in.cc/coverage/e77394de9a215d692ce3c797a033c9038c2cb45c_e77394de9a215d692ce3c797a033c9038c2cb45c/report/index.html
TPC-H: Total hot run time: 41274 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e77394de9a215d692ce3c797a033c9038c2cb45c, data reload: false
------ Round 1 ----------------------------------
q1 17620 4676 4256 4256
q2 2014 201 193 193
q3 10433 1244 1203 1203
q4 10207 928 819 819
q5 7483 2703 2700 2700
q6 216 135 130 130
q7 952 606 614 606
q8 9225 2128 2099 2099
q9 9432 6716 6747 6716
q10 9410 3904 3862 3862
q11 444 252 240 240
q12 423 244 242 242
q13 17548 3257 3093 3093
q14 269 202 222 202
q15 505 490 474 474
q16 504 413 389 389
q17 965 716 671 671
q18 8360 7958 7970 7958
q19 6310 1595 1560 1560
q20 637 317 314 314
q21 5251 4036 3264 3264
q22 349 283 303 283
Total cold run time: 118557 ms
Total hot run time: 41274 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4604 4500 4414 4414
q2 380 275 275 275
q3 3169 2988 2896 2896
q4 1884 1665 1617 1617
q5 5527 5521 5500 5500
q6 207 126 126 126
q7 2131 1834 1810 1810
q8 3278 3390 3384 3384
q9 8629 8836 8933 8836
q10 4244 3885 3888 3885
q11 619 512 510 510
q12 823 679 674 674
q13 16160 3273 3191 3191
q14 299 278 267 267
q15 549 493 512 493
q16 485 416 460 416
q17 1814 1562 1517 1517
q18 8237 7526 7445 7445
q19 1642 1555 1531 1531
q20 2018 1769 1797 1769
q21 9859 4866 4620 4620
q22 573 493 494 493
Total cold run time: 77131 ms
Total hot run time: 55669 ms
TPC-DS: Total hot run time: 168985 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e77394de9a215d692ce3c797a033c9038c2cb45c, data reload: false
query1 911 386 367 367
query2 6441 2223 2322 2223
query3 6649 216 216 216
query4 19678 17487 17364 17364
query5 4203 435 429 429
query6 250 158 153 153
query7 4591 303 291 291
query8 233 184 178 178
query9 8532 2405 2369 2369
query10 456 280 250 250
query11 10616 9943 9970 9943
query12 138 93 91 91
query13 1642 352 356 352
query14 10106 6762 6739 6739
query15 218 165 166 165
query16 7790 264 276 264
query17 1404 541 519 519
query18 1949 282 278 278
query19 239 150 147 147
query20 90 84 87 84
query21 197 134 131 131
query22 4258 3940 3965 3940
query23 33678 33133 32888 32888
query24 11134 2864 2918 2864
query25 608 352 362 352
query26 1167 153 154 153
query27 2738 319 325 319
query28 7232 2061 2019 2019
query29 873 610 602 602
query30 214 149 152 149
query31 994 736 712 712
query32 86 52 54 52
query33 746 266 259 259
query34 976 487 476 476
query35 729 617 594 594
query36 1093 917 892 892
query37 124 64 64 64
query38 2918 2745 2813 2745
query39 860 782 769 769
query40 197 126 126 126
query41 45 44 46 44
query42 101 99 96 96
query43 589 517 559 517
query44 1140 725 742 725
query45 174 166 189 166
query46 1066 727 727 727
query47 1841 1771 1766 1766
query48 367 295 292 292
query49 864 374 376 374
query50 769 380 399 380
query51 6878 6723 6768 6723
query52 97 95 88 88
query53 354 282 279 279
query54 833 423 420 420
query55 74 74 73 73
query56 253 243 245 243
query57 1115 1006 1011 1006
query58 236 205 204 204
query59 3382 3213 3195 3195
query60 272 246 256 246
query61 91 86 89 86
query62 622 455 449 449
query63 306 280 275 275
query64 8806 2223 1761 1761
query65 3212 3119 3153 3119
query66 792 340 333 333
query67 15627 14860 15077 14860
query68 4528 536 528 528
query69 434 268 267 267
query70 1159 1132 1093 1093
query71 423 267 267 267
query72 7498 5535 2713 2713
query73 707 320 317 317
query74 5949 5650 5601 5601
query75 3319 2666 2648 2648
query76 2723 972 1013 972
query77 393 264 266 264
query78 10219 9948 9812 9812
query79 2198 508 510 508
query80 1292 443 424 424
query81 547 220 217 217
query82 739 93 90 90
query83 267 170 170 170
query84 245 86 82 82
query85 2060 264 267 264
query86 510 310 313 310
query87 3288 3137 3104 3104
query88 4209 2327 2325 2325
query89 478 394 385 385
query90 2037 190 183 183
query91 171 97 96 96
query92 58 49 48 48
query93 2450 498 480 480
query94 1287 183 186 183
query95 400 318 310 310
query96 584 267 261 261
query97 3229 3033 3046 3033
query98 233 227 212 212
query99 1131 858 843 843
Total cold run time: 271629 ms
Total hot run time: 168985 ms
ClickBench: Total hot run time: 31.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e77394de9a215d692ce3c797a033c9038c2cb45c, data reload: false
query1 0.04 0.03 0.03
query2 0.09 0.03 0.04
query3 0.24 0.06 0.05
query4 1.67 0.09 0.08
query5 0.52 0.49 0.50
query6 1.12 0.73 0.73
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.53 0.49 0.51
query10 0.54 0.54 0.55
query11 0.16 0.12 0.11
query12 0.14 0.11 0.11
query13 0.59 0.59 0.60
query14 0.77 0.77 0.77
query15 0.81 0.81 0.81
query16 0.35 0.38 0.37
query17 0.94 0.98 1.02
query18 0.20 0.26 0.23
query19 1.79 1.67 1.65
query20 0.02 0.01 0.00
query21 15.68 0.67 0.65
query22 3.76 6.80 3.18
query23 18.32 1.37 1.21
query24 2.06 0.20 0.22
query25 0.15 0.09 0.08
query26 0.26 0.17 0.17
query27 0.08 0.08 0.07
query28 13.25 1.02 0.99
query29 13.28 3.30 3.26
query30 0.24 0.06 0.05
query31 2.91 0.37 0.38
query32 3.24 0.47 0.46
query33 2.90 2.89 2.90
query34 17.17 4.42 4.44
query35 4.48 4.46 4.54
query36 0.72 0.51 0.46
query37 0.17 0.16 0.14
query38 0.15 0.14 0.15
query39 0.04 0.03 0.04
query40 0.16 0.14 0.14
query41 0.09 0.05 0.05
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.8 s
Total hot run time: 31.62 s