[Fix](Stmt)Move reset insert timeout in handleInsert
Proposed changes
Issue Number: close #xxx
Problem summary
PR #16343 seprate the query_timeout and insert_timeout.
But it only change the timeout in ConnectProcessor::handleQuery function which only handle mysql client sql.
The CTAS and MTMV insert job will miss these sesssion variable.
So I move it into StmtExecutor::handleInsert funtion since all these three feature will go through these code.
Checklist(Required)
- [ ] Does it affect the original behavior
- [ ] Has unit tests been added
- [ ] Has document been added or modified
- [ ] Does it need to update dependencies
- [ ] Is this PR support rollback (If NO, please explain WHY)
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
run fe-ut
run buildall
TeamCity pipeline, clickbench performance test result: the sum of best hot time: 33.38 seconds stream load tsv: 474 seconds loaded 74807831229 Bytes, about 150 MB/s stream load json: 36 seconds loaded 2358488459 Bytes, about 62 MB/s stream load orc: 68 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230302061932_clickbench_pr_106929.html
Hi @SaintBacchus , just for disscussion, how about replace the ConnectContext#resetExecTimeout by
public void setExecTimeout(int timeout) {
executionTimeoutS = timeout;
}
And in StmtExecutor#handleInsertStmt, use
context.setExecTimeout(Math.max(context.getSessionVariable().getInsertTimeoutS(), context.getExecTimeoutS()));
I think it may seem clearer.
@TangSiyang2001 thanks for suggestions. I had modified it.
And I place the Math.max in the function setExecTimeout, how do you think about these?
public void setExecTimeout(int timeout) {
executionTimeoutS = Math.max(timeout, executionTimeoutS);
}
run buildall
run p1
run nereids
run p1
@TangSiyang2001 thanks for suggestions. I had modified it.
And I place the
Math.maxin the functionsetExecTimeout, how do you think about these?public void setExecTimeout(int timeout) { executionTimeoutS = Math.max(timeout, executionTimeoutS); }
Thank u for adopting my idea. I just curious about that if we support a finer-grained timeout in the future, and some stmt will require a shorter timeout than the default exec_timeout(currently query_timeout), in that case max policy in setExecTimeout may not be suitable anymore, and dragging max to the certain stmt, just keep setExecTimeout a simple set operation will be more flexible and extensible for supporting different timeout policy. How do you think :)
fine @TangSiyang2001
run buildall
PR approved by anyone and no changes requested.
@TangSiyang2001 can you reviewer again? I find that the open txn also need set this timeout
run p1
I think use resetExecTimeForInsert to do these job is better than setExecTime since it had called three times.
run buildall
run p1
I think use
resetExecTimeForInsertto do these job is better thansetExecTimesince it had called three times.
Sorry for late reply, you may look at my comment above. Will it be clearer than the former ways?
run buildall
run arm
run arm
PR approved by at least one committer and no changes requested.