NVIDIAGameWorks
Recommendations for LocalLighting Shadows in Pathtracing and NRD
Hello Dzdhan,
I am experimenting with moving all local lights using Path tracing and denoising using NRD. I am following the NRD sample of using Importance sampling using the Light TLAS and selecting that ray, this approach is very simple and it gives very good results, there are few problems I am facing however.
-
I am trying to get slightly sharper shadows so decreasing light source radius and increasing light intensity based on square law helps. Note that these were analytical light sources, so trying to find a good balance, between intensity and sharpness as large intensities increases variance. So I am playing around with these at the moment.
-
Shadow ghosting and lag, this is happening as the surface receiving the shadow is stationary and shadow is being disocclued during Path tracing. So I am wondering how best to handle this.
Also I have been studying Restir DI. What's your opinion on Restir DI ? because as NRD relies on hit-distance this may mess things up and I see similar documentation on DLSS-RR as well, i,e to have uncorrelated sampling.
Regards, Ganesh
I think I have an idea, I know if BRDF ray was traced to a light source during importance sampling, and I already know if the object hit on the path was a skinned instance, Can use this information(motion vector of occluder) to pass confidence inputs to NRD. Will try and let you know.
- Noted
- Do you mean that "not-so-soft" static shadows become very blurry if they come from a disoccluded region?
RESTIR itself is good. Algorithms based on top of it may be unreasonably slow. So, there are good and bad usages. The reality shows that it may be make sense to separate lighting and have 2 denoising instances with different settings:
- direct lighting - RELAX or maybe REBLUR
- indirect lighting - this where REBLUR shines
because as NRD relies on hit-distance
It's worth clarifying that only REBLUR uses hit distances everywhere for diffuse and specular. RELAX uses hit distances only for specular tracking (motion) and in PrePass for both diffuse and specular if enabled. This is why RELAX (with turned off pre-pass) can be used to denoise "abstract" signals.
As a side note, hit distances produced by local lights importance sampling (LightBVH or RESTIR) can be used with NRD.
P.S. Not commenting your idea, since it might be unrelated :)
Do you mean that "not-so-soft" static shadows become very blurry if they come from a disoccluded region?
Since by default NRD is designed to work with noisy indirect lighting, the result right after a disocclsuion maybe very blurry. You can do 2 experiments with 0 blur radius and 0 history fix frames to understand if it's the case or not.
Thanks for the reply Dzhdan, I am sticking to Reblur for everything(Running another instance of Relax, the overhead maybe too much atm),
Here is what I have so far
First of to clarify, Static shadows, with moving camera this is perfectly fine atm(even with Local lights also Path traced), I have fixed some extreme light values and now the noise/fireflies are almost non-existing.
-
Sun/LocalLights Direct lighting, with shadows rendered from Shadowmap, Compute shader for directlighting, Path tracing for direct lighting, REBLUR denoises the indirect GI, as you have mentioned its great. There is no need to denoise the direct lighting as everything is per-pixel with no stochastic sampling.
-
I decided to give local lights using path tracing a shot, well the results are absolutely amazing. You don't really notice that specular reflections are wrong with raster lighting or the fact that the light radius is used to contain the specular lighting to sensible performant ranges.
For LocalLights PT mode, I didn't use Restir or anything complicated just used the Sampling mechanism you have in the NRD sample, I realize Reblur denoises this so well as the HitDistance to the light is directly passed to Reblur. The primary issue I have is "Shading change" as the character is moving on top of a surface the shadows will get ghosting. This is where I was thinking of passing confidence inputs to the NRD Reblur denoise. The performance is also very good as you are shooting multiple rays into LightOnly BVH.
I am totally new to Path tracing as I have mentioned before and for me only looking to solve the problem rather than focus on specific algorithm like Restir feels right to me.
Questions and Problems I am trying to Solve.
-
You mentioned passing HitDistance to LightBVH. But after you select the Ray , assuming it was blocked in WorldTLAS, do I still pass the LightBVH hit distance, I assume this may make sense for DiffuseHitT , but keep the SpecularHitT to real value from WorldTLAS.
-
Shadow Sharpness, This is more specific to small objects, or characters/enemies. Its the spatial passes which are making them soft. i.e Looking to Preserve the Silhoutte of the shadows.
So right now I am thinking of a solution to pass in "Confidence inputs", This is another topic in itself xD. I read about it from ASVGF, as you don't want to falsely disocclude. Just "Rate of change", NRD - Reblur is still able to pull the history forward quite a bit, without too much Noise (AccumFrameNum - 21, FastHistoryNum - 2/3).
Sampleshot :). This is so cool. Seeing the reflection or "Lighting" of the central lamp on ship here.
Performance : DLSS Quality 1440p, 3060ti, 16ms Frame time for whole thing. Adding another denoiser instance will be difficult for me ATM. I am ok with a "Good enough solution for now for the shadows so specifically focusing on that problem
Regards, Ganesh
Do you mean that "not-so-soft" static shadows become very blurry if they come from a disoccluded region?
Since by default NRD is designed to work with noisy indirect lighting, the result right after a disocclsuion maybe very blurry. You can do 2 experiments with 0 blur radius and 0 history fix frames to understand if it's the case or not.
Yeah its the spatial passes which are blurring the shadows, I experimented with reducing Prepass Blur radius, When the character is fast moving, even if slightly blurry is ok I think. Anyways this is all a bit new to me as I never planned to do direct lighting with Path tracing, but is no harm to work on it and arrive at a meaningful/performant/reasonable solution.
Another Idea was running a SIGMA denoiser since that is very fast just for the shadows(I believe it can denoise all shadows together at once for locallights) and then somehow I combine that back in Composite or just do one "NEE" light , SIGMA denoise and combine back. You mentioned I create a "DarkSouls PT" thread would that be ok? Hope you guessed by now this is Mod work xD and not official etc, This is only fun/Hobby project. So I have been hesitant to post more questions.
Reducing Prepass Radius - > Very bad, Introduces a lot of boiling, It makes sense as you are blurring the outliers, I went through your paper on Reblur to understand the importance of each pass, Temporal Stabilization is another pass I understood why it exists, I made the stabilization strength low and lots of instability, going through the code it is mimic TAA. Every pass is important, The only breathing room left feels like the Confidence input parameter, not sure of Discocclusion threshold alternate will help much. So I am looking in Confidence input direction. Again want to clarify all this is in context of direct emission from lights. Indirect lighting no issues at all. NRD-Reblur is doing exactly as it is designed to do.
Thank you again for your responses.
OK, super short:
- yes, to mitigate temporal lags, coming from dynamic lighting, "confidence inputs" is the best approach. It's easy for me since I just accelerate the accumulation internally (with some tweaks). And it's not as easy for you since you have to calculate a confidence somehow (we can discuss it)
- SIGMA is per light only, unfortunately. I tried, I failed. But it's very fast and it's just a must have for sun/moon shadows (Fashion Souls series always have the primary directional light source + local light sources). SIGMA can handle the primary one
Another Idea was running a SIGMA denoiser since that is very fast just for the shadows(I believe it can denoise all shadows together at once for locallights) and then somehow I combine that back in Composite or just do one "NEE" light , SIGMA denoise and combine back.
I tried to use SIGMA to denoise multiple shadows. Nope. I failed. Please, look at the last section of NRD README :(
You mentioned I create a "DarkSouls PT" thread would that be ok?
Yes, sure! Whatever works best for you.
P.S. The screenshot is nice! P.P.S "Performance : DLSS Quality 1440p, 3060ti, 16ms Frame time for whole thing" - this is a good demonstration that an RT Remaster doesn't necessarily put a GPU to the knees ;) P.P.P.S Hope I didn't miss anything...
- SIGMA is per light only, unfortunately. I tried, I failed. But it's very fast and it's just a must have for sun/moon shadows (Fashion Souls series always have the primary directional light source + local light sources). SIGMA can handle the primary one
Thanks for letting me know on SIGMA. Yeah this will be the final part of Path tracing to move sun shadows , lots of Foliage animation in game, this is something I need to run super tight, if lot of grass area. This is not much of problem for local lighting as areas which are local lighting dominant don't have much of foliage so is ok.
P.S. The screenshot is nice! P.P.S "Performance : DLSS Quality 1440p, 3060ti, 16ms Frame time for whole thing" - this is a good demonstration that an RT Remaster doesn't necessarily put a GPU to the knees ;) P.P.P.S Hope I didn't miss anything...
Thank you. Not possible without all the support through NRD and NRD sample. I will integrate DLSS-RR later as well, but I think learning the concepts of PT its interaction with light transport, problems/issues is the real thing to achieve better results in long term.
PS : I can't get over how cool somethings are with PT, I guess you been doing this
- Specular lighting, its so natural, you start to realize how some of all the "Influence radius" concepts in raster are just placeholders or the fact that light originates from infinitely small points
- Every light, Every object shadow, well nothing needs to be said about it, this is amazing technology, I am not a games industry veteran, primarily work in Cloud services before, but even I can see how this will scale better in future, the limiting factor is sample count rather than the light count
I will do some experiements with the "Confidence" input and get back to you,(And create a new thread then) I realize that input requirements are these. You can close this thread.
- Need to be non-noisy, not perfect, not accurate, but it cannot be raw noisy samples
- Avoid false disocclusions, these is implicit as history resets are just the worst
Regards, Ganesh
lots of Foliage animation
Animation and BLAS management is always a pain point. But tracing performance can be uplifted with Opacity Micromaps.
I realize that input requirements are these
Yes, it's correct. We will discuss details for your specific use case in the other thread.
PT is computationally heavy, but fits naturally. Previously there were complicated math to approximate lighting from a tube, disk or rectangular light. Now even a star shape works out of the box with PT :)