nerfstudio icon indicating copy to clipboard operation
nerfstudio copied to clipboard

ExportGaussianSplat produces an invalid ply file when sh_degree == 0

Open MLLeKander opened this issue 1 year ago • 0 comments

Describe the bug ExportGaussianSplat produces an invalid ply file when sh_degree == 0

To Reproduce Steps to reproduce the behavior:

  1. Train GaussianSplat model with ns-train splatfacto --pipeline.model.sh_degree 0
  2. Export model with ns-export gaussian-splat
  3. View the exported ply file, and see numerous data issues (e.g. "colors" is a single uchar, data in other fields does not match the trained model).

Expected behavior A ply file with data that matches that of the model.

Additional context The root cause is that ExportGaussianSplat.write_ply expects each of the values in map_to_tensors to have count elements, but map_to_tensors["colors"] has 3 * count elements. Because of this, ply_file.write(value.tobytes()) actually writes 3 values when 1 value is expected, causing all subsequent fields (and elements) to be misaligned.

This behavior seems to have been (inadvertently) changed in #3005. Prior to that PR, the exporter used open3d to generate a ply. That old conversion process implicitly transformed the colors into red/green/blue, which you can verify by looking at the output of:

import open3d as o3d
import numpy as np
a = np.arange(12).reshape(-1,3).astype(float)
cld = o3d.t.geometry.PointCloud({"positions": a, "colors": a})
o3d.t.io.write_point_cloud("/tmp/asdf.ply", cld, write_ascii=True)

MLLeKander avatar Aug 23 '24 04:08 MLLeKander