LinearTapeFileSystem
Enhancement for creating pretty extent at flush
Summary of changes
Change the implementation of flush() request to enqueue the remaining blocks to the writer queue. And make them processed sequentially.
Description
Previously, our flush operation might flip the final block when we get a small time window below.
- LTFS receives a write and the unified scheduler completes one block and send it to the Q
- LTFS receives another write
- LTFS receives a flush
- LTFS processes the block created in step2 within flush operation (because previous flush stops the writer thread and write blocks directly)
- The writer thread wake up because of step1 and process the block queued in step1
In this change, all blocks are queued in flush operation and shall be processed by writer thread.
- Do not wait the writer thread processes the queued block when flush is called against individual files
- Wait the writer thread processes the queued block when flush is called against all files (like periodical sync request)
I believe this strategy improves the LTFS performance in normal condition (little bit make LTFS danger but enough safe, I believe). And LTFS makes a pretty extent in any time.
Type of change
- Enhancement with long term test
Checklist:
- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my own code
- [X] I have commented my code, particularly in hard-to-understand areas
- [X] I have made corresponding changes to the documentation
- [X] My changes generate no new warnings
- [X] I have confirmed my fix is effective or that my feature works
@piste-jp-ibm, I'm now looking into reproducing the original behavior. Would you know if the file with the flipped extent needs to be a candidate for writing to the Index Partition? Also, I'm wondering if the periodic sync thread may be playing a role here..
@lucasvr
It's a good point. The answer is yes. But I believe size of placement rules are small, like 1MiB, in most cases. So I can't see any problem around here.
But from logic point of view, we need to tape care about this.
That's really good to know, thanks! I will take a closer look at the interactions between the queues so I can understand the conditions which can lead to flipped extent entries.