libass
Browser freezing under 'heavy' subs
I recently got JavascriptSubtitlesOctopus up and running and testest a few videos, most which work great, but I noticed a couple videos with heavy intro subs (animation and effects) overloads my browser and freezes. From my understanding it should be able to handle it without lag. Perhaps I need to adjust some configurations? Below is a screenshot of the console output log.
@Yay295 Thanks for replying. Here is the ASS file along with the fonts used: https://drive.google.com/drive/folders/10li-vh6EO-0YfvK3GZqTW0Tqm1jfNI31?usp=sharing
A bit, but I don't have a conclusion. It is a big file, but it doesn't actually have anything too complex in it, and you can see from this demo that it can handle similar files (mostly) fine. I think it's probably just the sheer number of subtitle "lines" in the file. For the OP, some of the individual letters are composed of 50 or more lines to make it look how it does.
In my experience, at least in chrome, freeze happens when cache grows too large (or too fast?). Seems like memory reallocations in emscripten are becoming too expensive. But I don't really know much about it. And after freeze video will actually play ok. One more thing. Usually only 1080p are affected, while 720p are perfectly fine - another evidence of memory hog.
Scratch my previous comment, through maybe its a different issue.
Today I played with some heavy subs, and I noticed that they have large number of bitmaps for each frame.
So I tried to rewrite rendering part using createImageBitmap and got pretty good results.
Keep in mind, I'm not very familiar with this stuff.
I'm not sure if async breaks something, and there is no exception handling.
subtitles-octopus.js
function renderFrames() {
var data = self.renderFramesData;
var beforeDrawTime = performance.now();
self.ctx.clearRect(0, 0, self.canvas.width, self.canvas.height);
for (var i = 0; i < data.bitmaps.length; i++) {
var image = data.bitmaps[i];
self.ctx.drawImage(image.bitmap, image.x, image.y);
}
if (self.debug) {
var drawTime = Math.round(performance.now() - beforeDrawTime);
console.log(data.bitmaps.length + ' bitmaps, libass: ' + Math.round(data.libassTime) + 'ms, decode: ' + Math.round(data.decodeTime) + 'ms, draw: ' + drawTime + 'ms');
self.renderStart = performance.now();
}
}
subtitles-octopus-worker.js
self.render = function (force) {
self.rafId = 0;
self.renderPending = false;
var startTime = performance.now();
var renderResult = self._render(self.getCurrentTime() + self.delay, self.changed);
var changed = Module.getValue(self.changed, "i32");
if (changed != 0 || force) {
var result = self.buildResult(renderResult);
var newTime = performance.now();
var libassTime = newTime - startTime;
var promises = [];
for (var i = 0; i < result[0].length; i++) {
var image = result[0][i];
var imageBuffer = new Uint8ClampedArray(image.buffer);
var imageData = new ImageData(imageBuffer, image.w, image.h);
promises[i] = createImageBitmap(imageData, 0, 0, image.w, image.h);
}
Promise.all(promises).then(function (imgs) {
var decodeTime = performance.now() - newTime;
var bitmaps = [];
for (var i = 0; i < imgs.length; i++) {
var image = result[0][i];
bitmaps[i] = { x: image.x, y: image.y, bitmap: imgs[i] };
}
postMessage({
target: "canvas",
op: "renderMultiple",
time: Date.now(),
libassTime: libassTime,
decodeTime: decodeTime,
bitmaps: bitmaps
}, imgs);
});
}
if (!self._isPaused) {
self.rafId = self.requestAnimationFrame(self.render);
}
};
Here is some stats. Before:
...
19:54:49.747 subtitles-octopus.min.js:1 176 canvases, 7 ms (+ 51 ms draw)
19:54:49.788 subtitles-octopus.min.js:1 176 canvases, 7 ms (+ 40 ms draw)
19:54:50.086 subtitles-octopus.min.js:1 184 canvases, 7 ms (+ 20 ms draw)
19:54:50.107 subtitles-octopus.min.js:1 184 canvases, 7 ms (+ 21 ms draw)
19:54:50.126 subtitles-octopus.min.js:1 184 canvases, 7 ms (+ 18 ms draw)
...
After:
...
20:13:15.469 subtitles-octopus.min.js:1 176 bitmaps, libass: 6ms, decode: 3ms, draw: 1ms
20:13:15.519 subtitles-octopus.min.js:1 176 bitmaps, libass: 11ms, decode: 10ms, draw: 1ms
20:13:15.536 subtitles-octopus.min.js:1 176 bitmaps, libass: 4ms, decode: 3ms, draw: 1ms
20:13:15.569 subtitles-octopus.min.js:1 180 bitmaps, libass: 6ms, decode: 3ms, draw: 1ms
20:13:15.603 subtitles-octopus.min.js:1 180 bitmaps, libass: 4ms, decode: 3ms, draw: 0ms
...
@no1d Test code: https://dador.github.io/JavascriptSubtitlesOctopus/experimental/no1d-render/stress.html
It is process intensive but it doesn't freeze much if at all, though at one point in a random test the subtitles just decided to not render at all when it became a burden, but I just like the fact that it doesn't freeze (as often as the previous one):
