Evaluation options to generate depth values for gaussian splatting

[zhj296409022]

export class DepthTest extends Script { static scriptName = 'depthTest'

cameraFrame: CameraFrame | null = null

initialize() {
    const camera = this.app.root.findComponent('camera') as CameraComponent
    const cameraFrame = new CameraFrame(this.app, camera)
    cameraFrame.rendering.sceneDepthMap = true
    cameraFrame.options.ssaoBlurEnabled = false
    cameraFrame.updateOptions()
    cameraFrame.update()

    this.cameraFrame = cameraFrame
}

update() {
    // @ts-ignore
    this.app.drawTexture(0.7, -0.7, 0.5, 0.5, this.cameraFrame!.renderPassCamera.prePass.linearDepthTexture, null)
}

}

[mvaligursky]

That's correct, we do not have a good way to store depth. I've tried writing depth of quads used to render splats, and that's not very good. I'll leave this opened as a feature request, as long time it would be good to have. Note that perfect depth will not be able to be obtained, as splats are rendered by blending colors at different distances, there is no solid surface that defines depth.

One way around this would be to use a mesh generated offline to be rendered to depth buffer only - but that would most likely break rendering of the splat itself, as it does depth testing .. and so splats inside this mesh would not render.

[zhj296409022]

If the splats are small enough, is there a way to get an approximate depth map? I just want to place something on the object surface. Also, the scene may contain meshes.

[mvaligursky]

I'm currently working on Picker to be able to return 3d picked position, instead of just a mesh or gsplat that was picked, so that would be directly useful.

There is no built in way to do what you want currently though. You could customize picker to do so, or even customize the RenderPassPrepass to render depth (I had a test version of that in the past, with 2 small hacks, so I know its doable).

[mvaligursky]

I'm currently working on Picker to be able to return 3d picked position

Closing, and this is now done here: https://github.com/playcanvas/engine/pull/8154