EquirectangularSeamCorrection icon indicating copy to clipboard operation
EquirectangularSeamCorrection copied to clipboard

Consider less generic solutions

Open rdoeffinger opened this issue 4 years ago • 11 comments

The proposed solutions all are based on explicitly smoothing out the derivatives. However I would suggest a different interpretation of the issue: the function to calculate the coordinates is discontinuous, and the solution to switch between 2 equivalent variants that have the discontinuity in different locations. Instead of selecting whichever outputs that minimize the delta (where btw you could imagine cases where it would be wrong), it would also be possible to simply select the correct function from the start, leaving the remaining challenge to agree on which across the quad (assuming it's not possible to stitch the functions up in such a way to completely remove the discontinuity). To give an example, in ShaderToy the approach looks like this:

        // Calculate if we are near a bad part of atan and should switch,
        // then distribute that information.
        // Where possible, it would be better to calculate "bad" in the vertex
        // shader and store it into a flat attribute
        float bad = (2.0*pos.y < pos.x) ? 1.0 : 0.0;
        float badx1 = dFdx(bad);
        float bady1 = dFdy(bad);
        bad = (badx1 >= bad || bady1 >= bad) ? 1.0 : 0.0;
        badx1 = dFdx(bad);
        bady1 = dFdy(bad);
        if (badx1 > 0.0 || bady1 > 0.0) bad = 1.0;
        uv = vec2(bad > 0.0 ? phi_frac : phi, theta);

Note that this is particularly cheap in cases like the sphere with a simple texture mapped onto it since you can a priori know which triangles need to use which function and set things up either in a vertex shader or even as fixed vertex attributes (in which case it can work even without flat attributes, though requires duplicating a few vertices). I call this less generic because you need to be able to figure out if a quad (or triangle) would be affected by a discontinuity, so needs adjustment to your specific coordinate generation function.

rdoeffinger avatar Apr 13 '21 18:04 rdoeffinger

However I would suggest a different interpretation of the issue: the function to calculate the coordinates is discontinuous, and the solution to switch between 2 equivalent variants that have the discontinuity in different locations.

That's explicitly what the Tarini, Coarse Emulation, and Least Worst Quad Derivatives methods do. Only the explicit LOD and explicit gradient methods attempt to "smooth out the derivatives". Though they all do it by comparing two matching UVs with their discontinuities in different locations.

Note that this is particularly cheap in cases like the sphere with a simple texture mapped onto it since you can a priori know which triangles need to use which function and set things up either in a vertex shader or even as fixed vertex attributes (in which case it can work even without flat attributes, though requires duplicating a few vertices).

Precalculating which UV is best per triangle requires knowing the whole triangle before hand, which you don't know from the vertex shader. That means it'd have to be calculated in c# before hand which partially defeats the purpose of being able to calculate the UVs entirely in the shader. It also assumes the mesh in question doesn't have a pole in the middle of a triangle where neither UV set is "best" across the entire surface. Or that you have a "mesh" at all.

I should also point out this project originally came out of the context of procedural UVs on non-mesh based objects. Specifically equirectangular UVs on raytraced spheres. I'm using a mesh in this project to focus it more on the seam correction techniques and less on the raytracing. I would ask you to look at the Medium article this project is paired with, and the article it's a follow up to: https://bgolus.medium.com/distinctive-derivative-differences-cce38d36797b https://bgolus.medium.com/rendering-a-sphere-on-a-quad-13c92025570c

To give an example, in ShaderToy the approach looks like this:

That's basically the Tarini method written out in a more complicated way. Though you're selecting the best UV set at the mid point rather than at the seam. Though I do concede that approach will have fewer problems with derivative mismatches between the shader derivatives and the mip map calculation.

bgolus avatar Apr 13 '21 19:04 bgolus

FWIW, testing the ShaderToy on my Apple M1, the proposed method looks good whereas the Tarini's fwidth approach does not (has a seam). :)

cforfang avatar Apr 13 '21 19:04 cforfang

Interestingly, none of the approaches (original 4 + this one) look good on my Mali-G77 device... EDIT: Here's a direct link to a fork of the shadertoy with viewport 3 replaced with Reimar's approach for reference.

cforfang avatar Apr 13 '21 20:04 cforfang

It doesn't work on Mali? Super fascinating! What do you see here: https://www.shadertoy.com/view/WdccRS

Do you see "3 stipes" with the middle stripe showing diagonal lines, or 6 boxes with each box showing a different pattern?

bgolus avatar Apr 13 '21 20:04 bgolus

That one gives me the 3 stripes.

cforfang avatar Apr 13 '21 20:04 cforfang

Noooo! That means Mali defaults to coarse shader derivatives, and uses full quad derivatives for the mip level calculation! That means only the explicit LOD / gradients methods will work on Mali.

bgolus avatar Apr 13 '21 20:04 bgolus

Hm, my code was actually buggy. This one does actually seem to work for me on Mali (Note 10 Lite), though it's always possible my eyes are not good enough. But note it assumes that it doesn't matter that it cannot detect the "bad" case in one of the pixels because the changes are small enough, so it can't guarantee to catch all cases in case of coarse derivatives...

        int magic = 1 << (pixel_quad_pos.x+2*pixel_quad_pos.y);
        float bad = (2.0*pos.x < pos.y) ? float(magic) : 0.0;
        float badx1 = dFdx(bad);
        float bady1 = dFdy(bad);
        // ignore our own value of "bad", so that all pixels use the same function even in case of coarse derivatives
        // In that case we have to assume that pos.x/pos.y change slowly enough that checking 3 out of 4 pixels is good enough
        bad = (badx1 != 0.0 || bady1 != 0.0) ? float(magic) : 0.0;
        // for fine derivatives do an extra step to make sure a non-zero value was broadcast to all 4 pixels
        badx1 = dFdx(bad);
        bady1 = dFdy(bad);
        if (badx1 != 0.0 || bady1 != 0.0) bad = 1.0;
        uv = vec2(bad > 0.0 ? phi_frac : phi, theta);

rdoeffinger avatar Apr 13 '21 21:04 rdoeffinger

Note the reason why I think mine should usually work while the others would not is the difference on what is checked. The other solutions check the actual delta, and for that the delta available by dFdx etc needs to align with the texturing derivatives. However my proposal just checks "are we close to a problematic case" - in my specific code via the input values, but I think it should also be possible to do via the output values from atan (checking (phi < -0.4 || phi > 0.4) seemed to work in a quick test). For many of the interesting cases, just checking "are we close to the problem" means it is fine if we are able to check only 3 out of 4 pixels.

rdoeffinger avatar Apr 13 '21 21:04 rdoeffinger

Updated with the v2; https://www.shadertoy.com/view/7dsSRN Unless my eyes deceive me, this looks good on Mali-G77. Can test the M1 as well tomorrow..

cforfang avatar Apr 13 '21 22:04 cforfang

Updated with the v2; https://www.shadertoy.com/view/7dsSRN

Confirm that 3 looks fine on:

  • Mali-G76 (Bifrost Gen 2 texture mapper)
  • Mali-G78 (Valhall)
  • Apple M1

solidpixel avatar Apr 13 '21 22:04 solidpixel

Not well tested, but here is a version that has a few more comments and is optimized (the magic value can be done a bit cheaper and one derivative was completely unnecessary, making it potentially cheaper than the original version as well).

        // arbitrary non-zero value that is unique in each quad so the delta is never 0
        float magic = 1.0 + float(fragCoord.x) + 2.0 * float(fragCoord.y);
        float bad = (2.0*pos.x < pos.y) ? magic : 0.0;
        float badx1 = dFdx(bad);
        float bady1 = dFdy(bad);
        // Distribute "bad" value horizontally and vertically.
        // In case of coarse derivatives this also eliminates the value
        // calculated by the non-participating pixel, which is important
        bad = (badx1 != 0.0 || bady1 != 0.0) ? magic : 0.0;
        // For fine derivatives we need an extra step to distribute the value
        // to the diagonally opposite side.
        badx1 = dFdx(bad);
        if (badx1 != 0.0) bad = 1.0;
        uv = vec2(bad != 0.0 ? phi_frac : phi, theta);

EDIT: also replaced a last left-over > 0 with != 0 which is more consistent and correct.

rdoeffinger avatar Apr 14 '21 07:04 rdoeffinger