apache
HDDS-10316. Speed up TestReconTasks
What changes were proposed in this pull request?
Speed up TestReconTasks Creating cluster and initial setup is done once for all methods and modification accordingly. Speed is improved from 140.549 seconds to 98.482 seconds.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10316
How was this patch tested?
Works locally it works with num of datanodes as 1 but in CI it worked with number of datanodes as 3 and hence kept 3.
I ran TestReconTasks with your changes locally and faced the same result - testEmptyMissingContainerDownNode fails:
2024-03-01 23:50:24,505 [IPC Server handler 19 on default port 15002] DEBUG server.SCMDatanodeHeartbeatDispatcher (SCMDatanodeHeartbeatDispatcher.java:dispatch(157)) - Dispatching ICRs. 2024-03-01 23:50:24,505 [IPC Server handler 50 on default port 15009] DEBUG server.SCMDatanodeHeartbeatDispatcher (SCMDatanodeHeartbeatDispatcher.java:dispatch(157)) - Dispatching ICRs. 2024-03-01 23:50:24,510 [Recon-FixedThreadPoolWithAffinityExecutor-0-0] INFO scm.ReconContainerManager (ReconContainerManager.java:addNewContainer(246)) - Successfully added container #2 to Recon. 23:50:24.532 [8cc60fff-ccbe-46e4-9c74-b718081d73d5-ChunkReader-8] ERROR DNAudit - user=null | ip=null | op=UPDATE_CONTAINER {containerID=112022403450798084, forceUpdate=false} | ret=FAILURE org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: ContainerID 112022403450798084 does not exist at org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatchRequest(HddsDispatcher.java:305) ~[classes/:?] at org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.lambda$dispatch$0(HddsDispatcher.java:183) ~[classes/:?]
Can you please attach the stracktrack log to understand the flow because I didn't see this error. Thanks.
Changing the maven runner's JRE to java11 fixes these tests for me locally (this is weird).
Thanks @raju-balpande for the patch, @devmadhuu, @myskov for the review.
@raju-balpande can you take a look at HDDS-10654? I recently faced some flakiness in TestReconTasks, can you check if this change caused it?
