msbaines

The 3dscenegraph semantic dataset is currently limited to gibson_tiny. However, semantics for gibson_medium are expected to be release soon.

The mesh used for semantics is different from the mesh used for Habitat. The coordinate system is also different but both meshes. The Y and Z axis are switched but...

The semantic data from 3DSceneGraph is available via an [npz](https://docs.scipy.org/doc/numpy/reference/generated/numpy.savez.html) file. To access the data: `data = np.load(npz_path, allow_pickle=True)['output'].item()` This will return the following dictionary: `dict_keys(['building', 'room', 'object', 'camera', 'panorama'])`...

Was hoping to load the .npz file in C++ using [cnpy](https://github.com/rogersce/cnpy), but the .npz contains pickled data which cnpy can't handle. I could potentially handle the pickled data using [http://www.picklingtools.com/](http://www.picklingtools.com/)....

The semantic mask information is located in: `data['building']['object_inst_segmentation']` which an array that is the same size as the number of faces. Each element in the array contains the semantic object_id...

We should be able to write out the object ids using the following code: ``` f = open("out.bin", "wb") object_ids = data['building']['object_inst_segmentation'] f.write(object_ids.astype(np.int16).tobytes()) f.close() ```

Bounding box information: Looking at the object schema: ``` 33: {'action_affordance': ['sit at', 'lay on', 'pick up', 'move', 'clean', 'set', 'decorate'], 'class_': 'dining table', 'floor_area': 3.473003668596787, 'id': 33, 'location': array([4.48357247,...

Semantics seem to be working with the following transformation: x1 = x0 y1 = -z0 z1 = y1