r4victor
Change allocation strategy of the distance matrix to avoid overcommit
The DTWBD() function that performs sequence alignment allocates the whole distance matrix at once. This allocation may request a lot of memory. On Linux and Mac OS this is not a problem because they overcommit:
Overcommit refers to the practice of giving out virtual memory with no guarantee that physical storage for it exists).
And in practice, when the matrix is large, only elements in window are actually used. Windows simply won't allocate memory and return NULL. This caused some bugs before (Issue 1).
The solution is to change the code so that only elements in window are allocated. Like this, for example.
@ravigupta-art did you try turning on overcommit first? Run:
echo 1 | sudo tee /proc/sys/vm/overcommit_memory
In your terminal first.
I had the same issue in Linux and running the code suggested by @bhattarai333 made it work for me.
$ echo 1 | sudo tee /proc/sys/vm/overcommit_memory
This fixes it for me initially, but then I get a "Killed". Error message.. How much ram is needed for a sync? Is 8gb not enough?
@WoxWik, the RAM needed depends on the length of audio files. For example, when syncing a book with chapters up to an hour long (https://librivox.org/the-picture-of-dorian-gray-by-oscar-wilde/), the used memory peaks at about 1GB (tested on M1 MacBook). The used memory grows linearly with the audio length and the allocated memory grows quadratically. 8GB should be enough unless you sync much longer audio files.
is there any workaround to this issue? i have an 8gb ram laptop and i really needed to sync a 6hour audiobook with this setup. do you happen to know of any software i could use to merge these epub files perhaps, or do you know of any way that perhaps the task could be split into various subtasks within the audiofile ram limits?
