Understanding 2.5D Motion Vectors & Reprojection

[Passeridae]

Hi!

1. I'm using ReBLUR and ReLAX as learning resources to improve my own denoiser, and after looking into the temporal accumulation stage, I got interested in 2.5D motion vectors. However, this seems to be a rather uncommon concept — I haven’t found anything similar besides regular 2D MVs, neither in Unity (including HDRP, which adopted ReBLUR for ray-traced reflections), nor in UE5, AMD denoisers, or most other public repositories and engines. Where can I find more information about 2.5D MVs and, more specifically, how to compute them? I noticed an example of per-object 2.5D MV computation in this issue thread, in the first post. Is that code correct?

If I understand it right, I’d need to modify the MV shader pass and the camera’s MV output to include the ViewZ delta in the 3rd component. For now, I’m using a per-pixel mask to mark moving objects and relax disocclusion weights for their pixels to avoid harsh history rejection, but your approach looks like a more appropriate and elegant solution.

2. In TemporalAccumulation in ReBLUR (line 229) and ReLAX (line 135), I noticed that after loading history normals, you transform them as: smbNavg = Geometry::RotateVector( gWorldPrevToWorld, smbNavg );. What’s the reason for this transformation? Aren’t both current and previous normals stored in absolute world space and therefore directly comparable?

3. Looking at the original presentation (slide 30), I see that temporal disocclusion was initially computed based on tangent plane distance. But now, it seems to rely mostly on the angle between the view vector and the surface normal (along with several lerps and adjustments):

    float3 V = GetViewVector( X );
    float NoV = abs( dot( N, V ) );
    float NoVstrict = lerp( NoV, 1.0, saturate( smbParallaxInPixelsMax / 30.0 ) );
    float4 smbDisocclusionThreshold = GetDisocclusionThreshold( disocclusionThreshold, frustumSize, NoVstrict );
    smbDisocclusionThreshold *= float( dot( smbNavg, Navg ) > thresholdAngle ); // good for smb
    smbDisocclusionThreshold *= IsInScreenBilinear( smbBilinearFilter.origin, gRectSizePrev );
    smbDisocclusionThreshold -= NRD_EPS;. What was the reason behind this change or am I misentepreting the code 

What was the reason behind this change, or am I misinterpreting the code? I’m asking because most plane-based rejection functions require the world positions of both the sample and center pixels (and sometimes even the normal of the sample pixel). You seem to get away with using only the ViewZ of sample pixels, which is much cheaper, but it no longer looks like a pure plane weight.

Thank you!

[dzhdanNV]

Hi!

1: 2.5D motion

I have just added the link into README to 2.5D motion calculation (this link)

2D motion:

• represents only motion in the screen plane (pixel motion)

3D motion:

• represents world-space motion (point motion), but may suffer from imprecision problems if FP16 storage is used (we definitely want to stick with no more than FP16)

2.5D motion:

• Eureka - let's combine 2D and 3D!
• 2.5D motion = 2D motion + "viewZ delta"
• 2D motion = 2.5D motion with .z = 0 (yes, forced)
• 2.5D is a natural extension of 2D
• 2.5D offers better precision than 3D when used with low-precision formats (even FP16)

2: "previous world" to "current world" transformation

Special thanks for this question! In all cases except one this is not needed, because the matrices are the same. But in one case, which is "Portal RTX" (NRD version) it's needed to avoid a history reset at the moment when the camera is crossing the portal plane, because normals get flipped :)

3: changes in disocclusion calculations?

This is absolutely valid:

I’m asking because most plane-based rejection functions require the world positions of both the sample and center pixels
(and sometimes even the normal of the sample pixel). You seem to get away with using only the ViewZ of sample pixels,
which is much cheaper...

This is not:

...but it no longer looks like a pure plane weight

In short - the current code is a highly optimized version of naive 3D math (as you noted, with some adjustments).

3a: smbDisocclusionThreshold -= NRD_EPS; - why?

step(y, x) does (x >= y) ? 1 : 0, this addition makes to fail step(0, 0) - what is important.