microstream-one
when machine is shutdown accidentally, microstream storage is corrupted
Environment Details
- MicroStream Version:08.01.00-MS-GA
- JDK version: 17.0.4
- OS:Linux CentOS Linux release 7.7.1908 (Core)
- Spring 2.6.8
Describe the bug
I used the LazyArrayList to store my entity, the virtual machine is shutdown accidentally（not shutdown by man or plan, it is caused by some air condition issue ） , then I start my program by ‘systemctl start xxx’, it reports the following error
2023-06-15 19:32:28,040 ERROR o.m.s.types.StorageChannelTask.addProblem [MicroStream-StorageChannel-0] Error occurred in storage channel#0
one.microstream.storage.exceptions.StorageException: null
at one.microstream.storage.types.StorageFileManager$Default.readTransactionsFile(StorageFileManager.java:722)
at one.microstream.storage.types.StorageFileManager$Default.readStorage(StorageFileManager.java:688)
at one.microstream.storage.types.StorageChannel$Default.readStorage(StorageChannel.java:728)
at one.microstream.storage.types.StorageChannelTaskInitialize$Default.internalProcessBy(StorageChannelTaskInitialize.java:193)
at one.microstream.storage.types.StorageChannelTaskInitialize$Default.internalProcessBy(StorageChannelTaskInitialize.java:36)
at one.microstream.storage.types.StorageChannelTask$Abstract.processBy(StorageChannelTask.java:252)
at one.microstream.storage.types.StorageChannel$Default.work(StorageChannel.java:409)
at one.microstream.storage.types.StorageChannel$Default.run(StorageChannel.java:492)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: one.microstream.storage.exceptions.StorageException: Zero length transactions entry.
at one.microstream.storage.types.StorageTransactionsAnalysis$Logic.processBufferedEntities(StorageTransactionsAnalysis.java:377)
at one.microstream.storage.types.StorageTransactionsAnalysis$Logic.processInputFile(StorageTransactionsAnalysis.java:343)
at one.microstream.storage.types.StorageTransactionsAnalysis$Logic.processInputFile(StorageTransactionsAnalysis.java:301)
at one.microstream.storage.types.StorageLiveTransactionsFile$Default.processBy(StorageLiveTransactionsFile.java:93)
at one.microstream.storage.types.StorageFileManager$Default.readTransactionsFile(StorageFileManager.java:716)
... 8 common frames omitted
2023-06-15 19:32:28,047 WARN o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext.refresh [main] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'xxx': Unsatisfied dependency expressed through field 'connDB'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cellConnDB' defined in ai.momenta.osmdb.cellconn.CellConnConfig: Unsatisfied dependency expressed through method 'cellConnDB' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageService' defined in ai.momenta.osmdb.cellconn.CellConnConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [xxx.StorageService]: Factory method 'storageService' threw exception; nested exception is one.microstream.storage.exceptions.StorageException: Problem in channel #0
To Reproduce
Expected behavior
when the machine is shutdown accidentally, microstream storage is not corrupted
Screenshots
Additional context
my storage is opened by the following way.
StorageEntityCache.Default.setGarbageCollectionEnabled(enableGC);//todo change from default to each instance
var builder = EmbeddedStorageConfiguration.Builder();
var foundation = builder
.setStorageDirectory(graphDbPath)
.setChannelCount(1)
.setDataFileMaximumSize(ByteSize.New(2, ByteUnit.GB))
.createEmbeddedStorageFoundation();
foundation.onConnectionFoundation(BinaryHandlersJDK8::registerJDK8TypeHandlers);
return foundation.createEmbeddedStorageManager().start();
The exception indicates that the transaction file has an invalid entry written. The “zero length” relates to the transaction entry header and contains the header length that is a constant value (see comment in one.microstream.storage.types.StorageTransactionsAnalysis.java). As the error occurred after the VM has been shut down accidentally I assume it is caused by an incomplete io operation while writing the transaction entry to the transaction log.
Nevertheless we must ensure that such erroneous transaction entries don’t prevent the storage from starting up if possible.
@yqbjtu: The starting code you provided is ok, that is most likely not causing the problem.
Did you try to delete the transaction log before restarting the storage? (The transaction logs are the “.sft” files in the channel directories) if no transaction log is found the storage will try to start without checking the transactions.
Do you still have the corrupted transaction log files? Or did you check that files if they only contain zeros?
The exception indicates that the transaction file has an invalid entry written. The “zero length” relates to the transaction entry header and contains the header length that is a constant value (see comment in one.microstream.storage.types.StorageTransactionsAnalysis.java). As the error occurred after the VM has been shut down accidentally I assume it is caused by an incomplete io operation while writing the transaction entry to the transaction log.
Nevertheless we must ensure that such erroneous transaction entries don’t prevent the storage from starting up if possible.
@yqbjtu: The starting code you provided is ok, that is most likely not causing the problem.
Did you try to delete the transaction log before restarting the storage? (The transaction logs are the “.sft” files in the channel directories) if no transaction log is found the storage will try to start without checking the transactions.
Do you still have the corrupted transaction log files? Or did you check that files if they only contain zeros?
the corrupted microstream files info : ericyang@ericyangs-MacBook-Pro channel_0 % ls -al total 3285120 drwxr-xr-x 4 ericyang staff 128 6 15 01:11 . drwxr-xr-x 4 ericyang staff 128 6 3 21:53 .. -rw-r--r-- 1 ericyang staff 1508540416 6 15 18:25 channel_0_16.dat -rw-r--r-- 1 ericyang staff 163581952 6 15 18:25 transactions_0.sft ericyang@ericyangs-MacBook-Pro channel_0 % du -sh channel_0_16.dat 1.4G channel_0_16.dat ericyang@ericyangs-MacBook-Pro channel_0 % du -sh transactions_0.sft 156M transactions_0.sft
the transactions_0.sft size is 156M, When I delete transactions_0.sft, program fails to start
