iotdb
iotdb copied to clipboard
fix tests in BatchedAlignedSeriesReadChunkCompactionTest
Fix flaky tests in BatchedAlignedSeriesReadChunkCompactionTest
Description
This PR fixes 12 flaky tests in BatchedAlignedSeriesReadChunkCompactionTest that were failing intermittently due to non-deterministic iteration order of collections.
Root Cause
The getPaths() method in AbstractCompactionTest.java had two sources of non-determinism:
-
HashSetusage: The method usedHashSet<IFullPath>which does not guarantee iteration order -
Unsorted
schemaMap.values(): ThemeasurementSchemaslist was created directly fromschemaMap.values()without sorting, and sinceschemaMapis backed by aConcurrentHashMap, the iteration order is non-deterministic
This caused AlignedFullPath objects to be created with measurements in different orders across test runs, leading to comparison failures when validating compaction results.
Fix
- Changed
HashSettoLinkedHashSetto preserve insertion order - Sorted
measurementSchemasby measurement name before creating the list
Changes
File: iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/compaction/AbstractCompactionTest.java
// Before (non-deterministic):
Set<IFullPath> paths = new HashSet<>();
// ...
List<IMeasurementSchema> measurementSchemas = new ArrayList<>(schemaMap.values());
// After (deterministic):
Set<IFullPath> paths = new LinkedHashSet<>();
// ...
List<IMeasurementSchema> measurementSchemas =
schemaMap.values().stream()
.sorted(Comparator.comparing(IMeasurementSchema::getMeasurementName))
.collect(Collectors.toList());
Tests Fixed
-
BatchedAlignedSeriesReadChunkCompactionTest#testCompactionWithDifferentCompressionTypeOrEncoding -
BatchedAlignedSeriesReadChunkCompactionTest#testCompactionWithEmptyBatch -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionByFlushChunk -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionByFlushPage -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionByWritePoint -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithAllDeletedColumnByFlushChunk -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithAllDeletedPageByFlushPage -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithNotExistColumnByFlushChunk -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithNullColumn -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithNullColumnByFlushChunk -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithPartialDeletedColumnByFlushChunk -
BatchedAlignedSeriesReadChunkCompactionTest#testSimpleCompactionWithPartialDeletedPageByWritePoint
Verification
Verified using NonDex with 3 different random seeds - all tests pass consistently.