RT: Global illumination

[Try]

Concept

• Aim for to one ray-pixel budget. In theory 2rpp is may work for low-poly game.
• Micro-probes or surfels or hybrid
• Assume that game-level is static for now (special case for MoveTriggers maybe)
• Build g-buffer (or hit-list) per probe.
  • Ray queries can be cached, until something changes in world;
  • Probe g-buffer can be relight every-frame - except for shadow-ray it's cheap

First prototype (branch: https://github.com/Try/OpenGothic/tree/gi-rendering)

and probes(~30k):

Reference materials:

Lumen: https://advances.realtimerendering.com/s2022/SIGGRAPH2022-Advances-Lumen-Wright%20et%20al.pdf

Combination of screen-space probes and world-space cache, for HW-RT part. In paper there is a lot about SDF and software ray-tracing, yet we wont do it, as so requires precomputational part in asset-pipeline

• one probe per 4x4 tile. (64 rays per probe, accumulated over 4 frames)

EA-surfels: https://www.youtube.com/watch?v=h1ocYFrtsM4

• Surfel is directional and spawns on g-buffer geometry. No light leaks!
• Doesn't really work for grass (need to span surfel per grass-blade)
• Doesn't really work for animated objects (even with single-bone tracking surfel data becomes obsolete immediately on move)
• Cubemap-like acceleration structure for surfels

PRT probes: http://mrakobes.com/Nikolay.Stefanov.GDC.2016.pdf

• probes do represent computed irradiance
• surfels act like a sparse g-buffer + 1 bit to cache last known shadowmap test
• 4m spacing between probes
• 2-level hash-grid to store surfels
• hacky solution for multibounce GI

Personally prefer this workflow (except we will have to do ray-casting in runtime)

Good summary on light-leaks : https://handmade.network/p/75/monter/blog/p/7288-engine_work__global_illumination_with_irradiance_probes

Nice open-source project with multiple techniques implemented: https://github.com/EmbarkStudios/kajiya/blob/main/docs/gi-overview.md

[Try]

Added probe-caching; continue cleaning-up lightingcode:

~3k probes are recalculated each frame ~27k in total No SSAO/Bent-normal integration (yet?)

Need to refine caching scheme: cache ray-hit instead of irradicance

[Try]

Some more screenshoots:

[Try]

Rework for probe-grid lod:

Now probes do have almost consistent distribution in screen-space with similar budget

[Try]

Still working on secondary bounces. More screenshots:

[Try]

black-and-white version

[clort81]

Wanted to play with RT for ages and never found a FOSS implementation i liked. Following your work here, and very thankful to read your notes and results.

[Try]

Metal backend also works now:

Not much FPS on M1, yet still happy that Metal-backed now has less bugs :)

[Try]

Updated albedo-fetch heuristics:

Notes: albedo-fetch is a horrible hack that enables usage photo-texture of vanilla game, and attempts to extract usable color information

[Try]

Working on narrowing down, what is not working. When similar view-angle rendered with albedo=0.9 image look correct:

[Try]

Still tuning textures/color-bleeding. GI: No-GI:

[Try]

Going thru https://physicallybased.info

Sun (above atmosphere): 143'000.f -> 128'000.f Sun (on land): engine results in ~100.000f, matches real world observation Sky (median) was ~3% of sun-light (too low), after changing ground albedo factor (0.1 -> 0.3), becomes ~4.7% (correct)

Uniform-ambient is estimated as: ~sun * transmissioni * ang * p * 0.5 * 2pi~ - bunch of hacks; TODO.

After many adjustment:

[toolatetotheparty]

The latest screenshot looks more ~~and less~~ physically correct ~~at the same time~~. The intensity of direct sunlight looks less clipped, ~~but the GI deep inside the building looks gone, allowing that ghastly blueish fake GI they did to show through.~~ 🤔

upd. Though, on a second thought, there wouldn't be enough energy for indirect sunlight to bounce so many times and color everything orange... Anyways, some AO feeding on your GI implementation is also needed. Removing the effects of these blue-tinted light probes would do wonders for the general aesthetics.

[Try]

Main issue with GI (or lighting in general), when it comes to gothic, are materials. Or should I say - lack there of.

GI (and diffuse lighting) needs albedo data, but only thing we have in assets is a "photo-texture". For the sake of rendering I have heuristics here:

• assume that photo_texture = gamma(tonemapping(exposure*pbr(material)))
• if we inverse gamma&tonemap, we will get exposure * pbr(material)
• if we assume only diffuse lobe (specular is way to hard anyway): exposure * (1/PI) * dot(N,L) * albedo
• since it's a photo - assume dot(N,L)=1.0, simplify and we have C * albedo, where C is some constant

This is path one. Part 2: this heuristic has to be symmetrical with engine lighting.

• gamma(tonemapping((engine_lighting(albedo))) ~= photo_texture - this is formal way to say image should be similar to vanilla,
• this restricts constant C, to be something close to 1.0

Part 3. Due to the way inverse-tonemapping works, acesTonemapInv(1.0/*white*/) = inf

• We can't just use acesTonemapInv, resulting in this abomination:

vec3 textureAlbedo(vec3 rgb) {
  const vec3 linear = srgbDecode(rgb);
  return acesTonemapInv(linear*0.78+0.001)*5.0; // adjusted to have 'realistic' albedo values
  }

... and now it violate P2, to avoid inf

[Try]

Anyways, some AO feeding on your GI implementation is also needed

I do simply multiply GI by AO (what is not correct, but give at least some detail). Here is AO

Probably it would be better to use output bent-normal, from AO pass and use it intead

[Try]

Is it possible to further adjust AO so that it would work not only by seeking intersecting geometry but also volumetrically, with respect to how your GI works by gradually shading the whole interior?

Sorry, I don't know what do you mean. If AO is by definition ratio of ray-hit/ray-miss, and if shader checking not only intersection - than it's not AO

[Try]

gradually shading the whole interior?

Scene above is not interior:

When it comes to castle wall, it expected to be split 50:50, between ray-misses (samples sky-color) and hit on street floor. Street-floor has no direct component (it's in shadow), and indirect is also sky*albedo

[Try]

2k shadowmap unfortunately not good enough to take care of shadow-term. Also secondary-bounces are disabled, as I'm debugging atm.

almost like lumen:

[Try]

Debug view of ray-hits (needed to evaluate solution for secondary bounces):

[Try]

Been evaluating different approaches for iradiance cache and secondary bounces. Unlike Lumen, in OpenGothic we cannot do offline baking, and have to have runtime solution for gi-scene. Here example of hash-map of primitives that been touched by rays:

Every colored triangle is a single cache-entry and small primitives are also tessellated by bird-curve, to improve resolution a bit. Still not OK unfortunately: too much memory (~200+MB) and too low res