IsaacLab icon indicating copy to clipboard operation
IsaacLab copied to clipboard

[Bug Report] Inconsistent image shape between different camera implementations

Open Mayankm96 opened this issue 1 year ago • 0 comments

Describe the bug

Right now, the three camera types -- Camera, TiledCamera, and RayCasterCamera -- each return data in different shapes for different types. For instance, Camera returns (H, W, 4) and (H, W) for RGB and depth, while TiledCamera returns (H, W, 3) and (H, W, 1).

We should make all the images to follow a consistent pattern (H, W, C) and also make sure RGB does not imply a four-channeled RGBA.

Steps to reproduce

./isaaclab.sh -p source/standalone/demos/cameras.py

Output:

Camera @ '/World/envs/env_.*/Robot/base/front_cam': 
	data types   : ['distance_to_image_plane', 'rgb'] 
	semantic filter : *:*
	colorize semantic segm.   : True
	colorize instance segm.   : True
	colorize instance id segm.: True
	update period (s): 0.1
	shape        : (480, 640)
	number of sensors : 4
Received shape of rgb   image:  torch.Size([4, 480, 640, 4])
Received shape of depth image:  torch.Size([4, 480, 640])
-------------------------------
Tiled Camera @ '/World/envs/env_.*/Robot/base/front_cam': 
	data types   : ['depth', 'rgb'] 
	update period (s): 0.1
	shape        : (480, 640)
	number of sensors : 4
Received shape of rgb   image:  torch.Size([4, 480, 640, 3])
Received shape of depth image:  torch.Size([4, 480, 640, 1])
-------------------------------
Ray-Caster-Camera @ '/World/envs/env_.*/Robot/base': 
	view type            : <class 'omni.physics.tensors.impl.api.ArticulationView'>
	update period (s)    : 0.1
	number of meshes     : 1
	number of sensors    : 4
	number of rays/sensor: 307200
	total number of rays : 1228800
	image shape          : (480, 640)

System Info

  • Commit: 692b17794c0e3c63e12cf614973b8cb5906eeb20
  • Isaac Sim Version: 4.1
  • OS: Ubuntu 20.04

Checklist

  • [x] I have checked that there is no similar issue in the repo (required)
  • [x] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

  • [ ] All implementations of camera return consistent shapes for different modalities

Mayankm96 avatar Aug 02 '24 12:08 Mayankm96