Add Google Colaboratory Demo

[travisdriver]

Here is a demo of the door reconstruction on Colab. I was able to visualize the final reconstruction with visu3d by using the source code instead of the pypi wheel.

Open to any comments/edits!

[johnwlambert]

Very cool. Looks like the following commands are very verbose

conda env update --file environment_linux.yml
pip install -e .

Can those be run with a silent flag?

Also, do we need to run pip install -e . twice?

[johnwlambert]

Can we put the following code chunk into a library function?

cameras, images, points3d = read_model("results/ba_output")
pcd = np.array([_p3d.xyz for _p3d in points3d.values()])
rgb = np.array([_p3d.rgb for _p3d in points3d.values()])
cams = []
for _i in images.values():
    _c = cameras[_i.camera_id]
    R = _i.qvec2rotmat()
    t = (-R.T @ _i.tvec.reshape((3, 1))).flatten()
    cams.append(
        v3d.Camera(
             spec=v3d.PinholeCamera.from_focal(resolution=(_c.height, _c.width), focal_in_px=_c.params[0]),
             world_from_cam=v3d.Transform(R=R.T, t=t)
        )
    )

What's the conversion that's happening here?

[akshay-krishnan]

could you post a link to the colab?

[travisdriver]

could you post a link to the colab?

The link is at the top of the notebook

[travisdriver]

Can we put the following code chunk into a library function?

cameras, images, points3d = read_model("results/ba_output")
pcd = np.array([_p3d.xyz for _p3d in points3d.values()])
rgb = np.array([_p3d.rgb for _p3d in points3d.values()])
cams = []
for _i in images.values():
    _c = cameras[_i.camera_id]
    R = _i.qvec2rotmat()
    t = (-R.T @ _i.tvec.reshape((3, 1))).flatten()
    cams.append(
        v3d.Camera(
             spec=v3d.PinholeCamera.from_focal(resolution=(_c.height, _c.width), focal_in_px=_c.params[0]),
             world_from_cam=v3d.Transform(R=R.T, t=t)
        )
    )

This would require adding visu3d and dataclass_array as submodules to GTSfM since the current release of the pypi wheel has some known bugs. I think it would be better to leave it in the notebook for now until the next (stable) release of visu3d.

What's the conversion that's happening here?

visu3d has specific types it uses for plotting

[johnwlambert]

Can we put the following code chunk into a library function?

cameras, images, points3d = read_model("results/ba_output")
pcd = np.array([_p3d.xyz for _p3d in points3d.values()])
rgb = np.array([_p3d.rgb for _p3d in points3d.values()])
cams = []
for _i in images.values():
    _c = cameras[_i.camera_id]
    R = _i.qvec2rotmat()
    t = (-R.T @ _i.tvec.reshape((3, 1))).flatten()
    cams.append(
        v3d.Camera(
             spec=v3d.PinholeCamera.from_focal(resolution=(_c.height, _c.width), focal_in_px=_c.params[0]),
             world_from_cam=v3d.Transform(R=R.T, t=t)
        )
    )

This would require adding visu3d and dataclass_array as submodules to GTSfM since the current release of the pypi wheel has some known bugs. I think it would be better to leave it in the notebook for now until the next (stable) release of visu3d.

What's the conversion that's happening here?

visu3d has specific types it uses for plotting

Oh I just meant all the (R,t) stuff. If we use the gtsfm loader, or library functioncolmap2gtsfm, then we can leave out all these low level dirty transposes out. Those are easy to get wrong, and should be abstracted away from the notebook IMHO.

Sounds good to leave out visu3d from the env

[akshay-krishnan]

could you post a link to the colab?

The link is at the top of the notebook

cannot see the changes unless I clone locally:

[travisdriver]

could you post a link to the colab?

The link is at the top of the notebook

cannot see the changes unless I clone locally:

If you go into the actual branch and open up the notebook it should load, at least it did for me.

Here's the link in case it doesn't load: https://colab.research.google.com/github/borglab/gtsfm/blob/add-colab/notebooks/gtsfm_door_demo.ipynb