xiph
Varying quality within the frame
When encoding AVIF images with an alpha channel, I know which areas of the image are transparent, and therefore their YUV channels are meaningless (or proportionally less important, in case of semi-transparency).
I would like to pass that information to the encoder, so that it can avoid wasting bits on invisible pixels.
Another pitfall of decoupled alpha, I suppose. I'm not sure how high ROI encoding is on anyone's list of things to do... or how it would interact with all of our current RDO, etc.
If their YUV channels are zero, zero bits will be spent, and there's even a fast path in RDO for handling the zero-distortion case (which should usually be true for a zero channel). So forcing YUV channel values to zero in completely transparent areas is a good way to get this behavior. It's not ideal of course, but also won't hurt even if we add some extra sideband information later.
The problem is, I shouldn't have 0 values near the edges of opaque pixels. Black color will bleed into visible pixels, and a sharp edges will waste bits.
Even without the transparency, sometimes center of the picture is more important than the borders.
Can QP be set on per-tile basis, so that central tiles have lower QP?
So far I've worked around that by setting solid color under transparent pixels and adding blur/bloom around the edges with opaque pixels, so that the color channel can have hopefully-cheaper-to-encode soft edges when alpha gives it sharp edges.
https://github.com/kornelski/cavif-rs/blob/ff0e85bb67c6bb193fd496e195fa92ae6e93fb83/ravif/src/dirtyalpha.rs
However, it's all still very speculative and costs some bits. Since I can't know how the image will be split into tiles, I can't clear transparent colors to zero completely (I'd like to clear to zero only rectangular areas exactly matching tiles).