nerfstudio
nerfstudio copied to clipboard
Add depth supervision to the semantic-nerfw model
This PR integrates depth supervision to the existing semantic-nerfw model.
- Added a
include_depthflag to thesemantic-nerfwmodel - Added (optional) depth information to
SemanticDataset - Added
semantic_filenamesto thenerfstudio_dataparser - Updated configurations for
semantic-nerfwinmethod_configs.py
Note: The sitcom3d_dataparser is removed in method_configs.py to pass the check. Looks like it is more unified to have the semantics info included in the nerfstudio_dataparser just like depth info.
Trained the model on Replica Dataset with 2 views of a single scene using both semantics and depth supervision, the results of rendered rgb, depth and semantics of two other adjacent views in the eval set are shown below (left: groundtruth, right: rendered result)
view1 (RGB)

view1 (depth)

view1 (semantics)

view2 (RGB)

view2 (depth)

view2 (semantics)
