apache
Fix the concurrency problem of the compaction process and the regular…
… write process
Descriptions of the changes in this PR: add a synchronized createNewLog method in EntryLogManagerForSingleEntryLog.
Motivation
we use SortedLedgerStorage+EntryLogManagerForSingleEntryLog.
When the entrylog file managed by EntryLogManagerForSingleEntryLog reaches the upper limit, the createNewLog method is executed. This operation is not locked, which will conflict with the addEntry executed in the compaction process, resulting in disordered entryLog file data.
There is a concurrency problem with the two operations shown above.
In our case we found this code execution order:
1.logChannel.flush();
2.logChannel.write(sizeBuffer);
3.logChannel.appendLedgersMap();
4.logChannel.write(entry);
The correct entryLog file format should be like this： header entrySize1+entryData1 entrySize2+entryData2 ... entrySizen+entryDatan ledgersMap
The wrong entrylog is as follows: header entrySize1+entryData1 entrySize2+entryData2 ... entrySizen ledgersMap entryDatan
Changes
add a synchronized createNewLog method in EntryLogManagerForSingleEntryLog.
Master Issue: #3604
I used the scanEntryLog method in EntryLogger to parse an error file. The phenomenon is as follows:
I don't think it's nice to lock the method, the createNewLog should be a separate process and should eliminate the point of thread insecurity
Do you have any concerns about this modification? Could you be more specific? @StevenLuMT
getCurrentLogForLedgerForAddEntry
The createNewLog does not lock the activeLogChannel
Close this since it is open for a long time without any updates. Feel free to reopen it if you want to continue
