apache
[Fix](hive-writer) Fix partition column orders issue when the partition fields inserted into the target table are inconsistent with the field order of the query source table and the schema field order of the query source table.
Proposed changes
Fix partition column orders issue when the partition fields inserted into the target table are inconsistent with the field order of the query source table and the schema field order of the query source table.
Please look at regression-test/suites/external_table_p0/hive/ddl/test_hive_write_type.groovy in the PR.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
PR approved by at least one committer and no changes requested.
TeamCity be ut coverage result: Function Coverage: 35.65% (9016/25293) Line Coverage: 27.32% (74567/272971) Region Coverage: 26.52% (38568/145420) Branch Coverage: 23.39% (19665/84088) Coverage Report: http://coverage.selectdb-in.cc/coverage/dcfabac080e7edfee0e232f9b52df7bcb25745e3_dcfabac080e7edfee0e232f9b52df7bcb25745e3/report/index.html
TPC-H: Total hot run time: 41896 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dcfabac080e7edfee0e232f9b52df7bcb25745e3, data reload: false
------ Round 1 ----------------------------------
q1 17593 4301 4258 4258
q2 2021 194 192 192
q3 10512 1160 1222 1160
q4 10194 797 830 797
q5 7480 2754 2735 2735
q6 218 134 143 134
q7 1086 610 609 609
q8 9225 2134 2089 2089
q9 9279 6697 6682 6682
q10 9753 3958 3904 3904
q11 439 241 240 240
q12 479 237 238 237
q13 17716 3198 3330 3198
q14 256 212 207 207
q15 512 471 469 469
q16 475 402 415 402
q17 967 692 772 692
q18 8394 7862 7783 7783
q19 6681 1605 1538 1538
q20 655 325 305 305
q21 5058 3990 4348 3990
q22 348 280 275 275
Total cold run time: 119341 ms
Total hot run time: 41896 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4555 4472 4386 4386
q2 369 273 272 272
q3 3138 3018 2824 2824
q4 1905 1583 1658 1583
q5 5490 5484 5501 5484
q6 214 132 129 129
q7 2228 1851 1833 1833
q8 3271 3375 3447 3375
q9 8777 8833 8730 8730
q10 3893 3847 3883 3847
q11 615 511 500 500
q12 787 620 630 620
q13 17042 3197 3187 3187
q14 291 253 257 253
q15 517 486 465 465
q16 469 412 419 412
q17 1789 1485 1489 1485
q18 7691 7689 7427 7427
q19 1660 1535 1575 1535
q20 2009 1819 1766 1766
q21 7620 4883 4802 4802
q22 572 480 493 480
Total cold run time: 74902 ms
Total hot run time: 55395 ms
TPC-DS: Total hot run time: 172159 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dcfabac080e7edfee0e232f9b52df7bcb25745e3, data reload: false
query1 920 389 385 385
query2 6464 2451 2391 2391
query3 6632 212 213 212
query4 19871 17404 17420 17404
query5 4103 424 432 424
query6 249 156 147 147
query7 4586 296 305 296
query8 232 193 182 182
query9 8650 2369 2356 2356
query10 454 284 264 264
query11 10480 10061 10059 10059
query12 136 91 89 89
query13 1641 372 389 372
query14 8426 6904 6974 6904
query15 215 173 172 172
query16 7750 265 262 262
query17 1629 544 510 510
query18 1960 265 270 265
query19 198 155 165 155
query20 92 86 84 84
query21 190 130 127 127
query22 4144 4134 3958 3958
query23 33627 33021 32847 32847
query24 10604 2820 2793 2793
query25 563 352 362 352
query26 709 157 160 157
query27 2231 325 322 322
query28 5892 2033 2041 2033
query29 853 609 610 609
query30 255 172 177 172
query31 951 776 763 763
query32 91 53 52 52
query33 663 266 263 263
query34 879 474 477 474
query35 723 585 588 585
query36 1030 924 910 910
query37 105 69 69 69
query38 2901 2804 2768 2768
query39 840 794 816 794
query40 192 126 122 122
query41 43 43 44 43
query42 102 95 96 95
query43 608 577 595 577
query44 1064 718 748 718
query45 182 163 166 163
query46 1075 738 690 690
query47 1848 1801 1805 1801
query48 364 293 298 293
query49 840 381 378 378
query50 769 389 389 389
query51 6926 6895 6759 6759
query52 107 95 91 91
query53 349 294 284 284
query54 862 430 423 423
query55 80 72 73 72
query56 258 238 276 238
query57 1118 1067 1044 1044
query58 230 204 216 204
query59 3480 3562 3280 3280
query60 282 250 247 247
query61 88 90 92 90
query62 601 462 477 462
query63 310 292 286 286
query64 8498 2319 1864 1864
query65 3167 3062 3090 3062
query66 774 325 338 325
query67 15422 14886 14701 14701
query68 4500 530 544 530
query69 443 266 262 262
query70 1139 1143 1071 1071
query71 375 263 268 263
query72 7712 5506 5259 5259
query73 730 328 326 326
query74 5988 5553 5721 5553
query75 3295 2637 2628 2628
query76 2296 1033 957 957
query77 396 269 271 269
query78 10250 10021 9766 9766
query79 2076 505 506 505
query80 1199 435 428 428
query81 552 241 236 236
query82 666 98 95 95
query83 248 169 171 169
query84 245 83 83 83
query85 1978 268 262 262
query86 511 333 312 312
query87 3270 3121 3128 3121
query88 4195 2334 2336 2334
query89 488 390 389 389
query90 2038 188 187 187
query91 128 153 97 97
query92 65 50 48 48
query93 2284 504 493 493
query94 1265 184 188 184
query95 402 319 321 319
query96 591 268 267 267
query97 3191 2996 3004 2996
query98 248 219 217 217
query99 1131 876 851 851
Total cold run time: 265473 ms
Total hot run time: 172159 ms
ClickBench: Total hot run time: 30.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dcfabac080e7edfee0e232f9b52df7bcb25745e3, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.06
query4 1.67 0.10 0.07
query5 0.49 0.49 0.52
query6 1.12 0.73 0.72
query7 0.02 0.02 0.01
query8 0.04 0.04 0.05
query9 0.54 0.49 0.50
query10 0.54 0.56 0.55
query11 0.15 0.11 0.11
query12 0.15 0.12 0.12
query13 0.60 0.59 0.58
query14 0.78 0.78 0.79
query15 0.83 0.80 0.80
query16 0.38 0.38 0.37
query17 0.94 1.03 1.04
query18 0.22 0.24 0.22
query19 1.78 1.69 1.67
query20 0.02 0.01 0.01
query21 15.45 0.72 0.68
query22 4.11 7.11 2.05
query23 18.31 1.40 1.28
query24 1.58 0.35 0.22
query25 0.15 0.09 0.08
query26 0.26 0.18 0.17
query27 0.08 0.08 0.08
query28 13.33 1.01 0.99
query29 13.21 3.30 3.24
query30 0.23 0.06 0.05
query31 2.87 0.38 0.38
query32 3.30 0.48 0.46
query33 2.86 2.88 2.93
query34 17.10 4.43 4.52
query35 4.52 4.58 4.58
query36 0.67 0.48 0.47
query37 0.17 0.15 0.15
query38 0.15 0.14 0.14
query39 0.04 0.04 0.04
query40 0.16 0.14 0.14
query41 0.10 0.05 0.05
query42 0.05 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 109.37 s
Total hot run time: 30.82 s
PR approved by at least one committer and no changes requested.