Different results in `rasterization`, `rasterization_2dgs` and `rasterization_2dgs_inria_wrapper` for a single flat gaussian

[belkakari]

I was debugging rasterization_2dgs on a toy example of a single gaussian rotating slightly and found a weird behavior that I was not expecting as well as some inconsistencies between rasterization_2dgs and rasterization_2dgs_inria_wrapper which I've installed via git clone https://github.com/hbb1/diff-surfel-rasterization.git && cd diff-surfel-rasterization && uv pip install . --no-build-isolation

My version of gsplat is 1.4.0@dd66cbd597f376f103e90de8b6265087b482eac1

For a gsplats implementation it starts ok as a rotating disk but then transforms into two reflected parts of the disk, then scales and then becomes almost like a single square Inria's code behaves differently and the gaussian actually disappears for a small rotation and then stays as a small line (which is a bit more expected but also weird). When I render the same gaussian with a simple rasterization function it works as I'd expect

Am I missing something and this is an expected behavior or is there a bug in implementation?

the code I've used:

import torch
from gsplat import rasterization, rasterization_2dgs, rasterization_2dgs_inria_wrapper
import matplotlib.pyplot as plt
import roma
import numpy as np

renders = []

W = 256
H = 256
mean = torch.tensor([[0., 0., 0.3]] , device ="cuda")
color = torch.tensor([[(1., 1., 1.)]] , device ="cuda")
opac = torch.ones((1,) , device ="cuda")
scale = torch.tensor([[1., 1., 0.]] , device ="cuda") * 5
view = torch.eye(4, device ="cuda") [ None ]
K = torch.tensor(
    [[[1., 0., W / 2],
      [0., 1., H / 2] ,
      [0. , 0. , 1.]]] , device ="cuda") # camera intrinsics

angles = [0, 0.5, 2, 2.5, 3, 60, 90]

# gsplat render
for ang in angles:
    euler = torch.tensor((0, 0, ang), device='cuda', dtype=torch.float32)
    quat = roma.euler_to_unitquat(convention='xyz', angles=euler, degrees=True).unsqueeze(0)
    (
        render_colors,
        render_alphas,
        render_normals,
        normals_from_depth,
        render_distort,
        render_median,
        info,
    ) = rasterization_2dgs(
        means=mean,
        quats=quat,
        scales=scale,
        opacities=opac,
        colors=color,
        viewmats=torch.linalg.inv(view),  # [C, 4, 4]
        Ks=K,  # [C, 3, 3]
        width=W,
        height=H,
        near_plane=0.01,
        far_plane=1e10
    )
    renders.append(torch.clamp(render_colors, 0.0, 1.0).cpu().data.numpy()[0])

#inria render 
renders_inria = []

for ang in angles:
    euler = torch.tensor((0, 0, ang), device='cuda', dtype=torch.float32)
    quat = roma.euler_to_unitquat(convention='xyz', angles=euler, degrees=True).unsqueeze(0)

    (render_colors_inria, render_alphas_inria), meta = rasterization_2dgs_inria_wrapper(
            means=mean,
            quats=quat,
            scales=scale,
            opacities=opac,
            colors=color,
            viewmats=torch.linalg.inv(view),  # [C, 4, 4]
            Ks=K,  # [C, 3, 3]
            width=W,
            height=H,
            near_plane=0.01,
            far_plane=1e10
    )
    renders_inria.append(torch.clamp(render_colors_inria[..., :3], 0.0, 1.0).cpu().data.numpy()[0])

# 3dgs render
renders_3d = []

for ang in angles:
    euler = torch.tensor((0, 0, ang), device='cuda', dtype=torch.float32)
    quat = roma.euler_to_unitquat(convention='xyz', angles=euler, degrees=True).unsqueeze(0)
    
    render_colors, render_alphas, info = rasterization(
        means=mean,
        quats=quat,
        scales=scale,
        opacities=opac,
        colors=color,
        viewmats=torch.linalg.inv(view),  # [C, 4, 4]
        Ks=K,  # [C, 3, 3]
        width=W,
        height=H,
        near_plane=0.01,
        far_plane=1e10,
        sh_degree=None,
    )
    renders_3d.append(torch.clamp(render_colors, 0.0, 1.0).cpu().data.numpy()[0])

[Golbstein]

I tried to use now rasterization_2dgs and rasterization_2dgs_inria_wrapper for rendering and also got 2 slightly different results rasterization_2dgs

rasterization_2dgs_inria_wrapper

the model was trained with rasterization_2dgs though

btw, maybe it's because of this argument?

        eps2d: An epsilon added to the egienvalues of projected 2D covariance matrices.
            This will prevents the projected GS to be too small. For example eps2d=0.3
            leads to minimal 3 pixel unit. Default is 0.3.

[belkakari]

Interesting, here are the results of rasterization_2dgs with eps2d=0.1, 0.2, 0.3 and 0.4, looks quite simillar

@Golbstein can you also render your scene with a regular rasterization ?