iotdb icon indicating copy to clipboard operation
iotdb copied to clipboard

fix test in InnerUnseqCompactionWithReadPointPerformerTest

Open kabo87777 opened this issue 2 months ago • 0 comments

Fix Non-Deterministic Behavior in InnerUnseqCompactionWithReadPointPerformerTest

Problem

InnerUnseqCompactionWithReadPointPerformerTest.test was failing non-deterministically under NonDex with 100% failure rate (10/10 runs) due to reliance on HashMap/HashSet iteration order in test setup and file generation.

Way to Reproduce

cd iotdb-core/datanode
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex \
  -Dtest=InnerUnseqCompactionWithReadPointPerformerTest#test \
  -DnondexRuns=10 -Drat.skip=true
# Result: 10/10 failures with varying expected/actual values
# Example: expected:<1149> but was:<2949>

Root Cause

Multiple HashMap/HashSet usages caused non-deterministic iteration order:

  1. Test expectations map (chunkPagePointsNumMerged): HashMap iteration affected assertion order
  2. Measurement paths set (fullPath): HashSet iteration determined TsFile write order
  3. Modification maps (toDeleteTimeseriesAndTime): HashMap affected deletion ordering
  4. Device-measurement grouping in CompactionFileGeneratorUtils: HashMap caused non-deterministic device ordering in generated TsFiles

When NonDex shuffled collection order, measurements/devices were written in different sequences, causing expected chunk/page point counts to mismatch actual values.

Solution

InnerUnseqCompactionWithReadPointPerformerTest.java:

  • Changed Set<String> fullPath from HashSet to LinkedHashSet (line 130)
  • Changed toDeleteTimeseriesAndTime maps from HashMap to LinkedHashMap (lines 313, 356)
  • Changed chunkPagePointsNumMerged from HashMap to LinkedHashMap (line 383)

CompactionFileGeneratorUtils.java:

  • Changed deviceMeasurementMap from HashMap to LinkedHashMap in writeChunkToTsFileWithTimeRange() method

Verification

cd iotdb-core/datanode
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex \
  -Dtest=InnerUnseqCompactionWithReadPointPerformerTest#test \
  -DnondexRuns=20 -Drat.skip=true
# Result: 20/20 passes (100% success rate)

Key changed classes:

  • InnerUnseqCompactionWithReadPointPerformerTest (test changes only)
  • CompactionFileGeneratorUtils (test utility changes only)

kabo87777 avatar Nov 11 '25 22:11 kabo87777