xiaozhuai
I am adding a parallelism feature
I was not happy with how slow this was for making vey large resolution gifs, and my generator is capable of using threading and this gif encoder became the bottleneck. So I'm adding a parallelism functionality to it and when I'm finished I'll share it here. Some points:
- My paralellism doesn't support global color maps, so I had to cut that out, I might still include it in the legacy single-threaded option.
- I will also add a support for out of order frame pushes. Because my generator creates the images in paralel, it's not guaranteed they are finished in order, so it would just waste time to wait for the nedxt sequential one.
- I'm also adding support for a cancellation token to quickly cancel and restart the process.
- I'm also letting it take the picture bytes directly without copying them, that however doesn't support any non-BGR format. I might still keep that option in a separate overload.
So basically I will be restructuring the inner workings, but I'll keep the original calls for compatibility, and to use the new features, I'll add overloaded versions of those calls.
How to use the parallelism:
- Use open_parallel and push_parallel instead of the classic ones, the push_parallel makes its own threads and puts them into a reference into you pass that thread reference into the push argument, otherwise if you use an overload without a thread reference, then it assumes you are already callid the push within multiple parallel threads.
- Calling close() in parallel mode will just flag the last pushed frame as the last.
- 3.Calling tryWrite will try to write the next sequential frame into the file. You should keep callign tryWrite until it either fails or returns "Finished". At that point your file will be complete.
- Do NOT call tryWrite in parallel without giving it a true parrallel argument (that sets a monitor lock), the parallelism is only for the CPU intensive push_parallel.
Example:
GifEncoder* e = new GifEncoder();
e->open_parallel(ConvertToStdString(gifTempPath), w, h, 1, 0, &cancel->Token)
for(int i = 0; i < frames; ++i) task[i] = std::thread(&Example::ParallelThread, this, e, i);
for(int i = 0; i < frames; ++i) if(task[i].joinable()) task[i].join();
e->close();
while(true){ int u = e->getFinishedFrame();
switch(result = e->tryWrite(&cancel->Token))
case GifEncoderTryWriteResult::Failed: return "failed";
case GifEncoderTryWriteResult::FinishedFrame: b[u]->UnlockBits(d[u]);break;
case GifEncoderTryWriteResult::FinishedAnimation: return "success";
}}
void Example::ParallelThread(GifEncoder* e, int i){
r = Rectangle(0, 0, w, h);
b[i] = gcnew Bitmap(w,h)
d[i] = b[i]->LockBits(r, ImageLockMode::WriteOnly, PixelFormat::Format24bppRgb);
uint8_t* p, mp = p = (uint8_t*)(void*)d[i]->Scan0;
for(int y = 0; y<h; ++y)for(int x = 0; x<w; ++x){p[2]=blue;p[1]=green;p[0]=red;p+=3;}
e->push_parallel(mp, w, h, delay, i, &cancel->Token);
}
On my vacation now. Please submit a PR. I'll review this as soon as I come back to office.
I've never done PR before, but I guess it can't be that hard, I'll try to test everything to see if there aren't any serious flaws that i might still fix on my own, and document it a little more and then I'll do it. But I have a question - your code suggests that local color map can get pushed framesof differing resolution as you only block that with the global one, and explicitly say that global colormap doesn't support it. Is that really true? should i try to test to push such a scenario and account for that possibility in my expansion?
Update: i've tried to do a pull requested, but ended up forking instead: https://github.com/SkrFractals/GifEncoder That seemed to be the first step to make a PR, and now I think I've managed to submit the PR.
Update: I've kept documenting it further and fixing more bugs, and I thought that aborting and resubmitting the PR would be ok, and it probably somewhat is, but then I found the closed PR spam I caused and can't remove these aborts. I probably should've have just tried to commit the updates to the one PR, and will try to do that from now on. Right now I've discovered that I've implemented the global color map slightly incorrectly - only learning from the first frame instead of all of them. I'm fixing it now and will push a fix in following days. And I might keep this bug in as an optional feature because I can actually find it useful.