[Bug Report] ViewerCfg origin type = asset_root is broken

[KyleM73]

Describe the bug

When training with rsl_rl in the default locomotion/velocity environments, if you switch the viewer origin type to track the asset_root the terrain is not visualized when recording videos. see image below. visualization is not affected when the viewer is attached to the world, only when it tracks the robot. I have confirmed this occurs with multiple locomotion tasks.

Steps to reproduce

Reproduce by pasting this into the post_init() of any task cfg:

self.viewer.eye = (2.5, 0.5, 0.3) self.viewer.origin_type = "asset_root" self.viewer.asset_name = "robot"

Go1-Flat

Anymal-D-Flat

System Info

• Commit: [9ab6b485bad1b57fb93d119907d749d4d2bf289e] (main)
• Isaac Sim Version: [4.0]
• OS: [Ubuntu 20.04]
• GPU: [RTX 3080 12Gb]
• CUDA: [12.2]
• GPU Driver: [535.183.01]

Checklist

• [x] I have checked that there is no similar issue in the repo (required)
• [x] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

• [ ] The terrain is visualized normally when the viewer tracks the asset_root

[Mayankm96]

Hi,

Thanks for reporting this issue.

• Is the camera at an angle where the terrain is perpendicular to the camera plane?
• Does it also happen for the rough terrain?

[KyleM73]

1. no, the camera eye is at (2.5, 0.5, 0.3). I tried other angles (eg 3.5, 3.5, 3.5) as well and it did not change anything (example in the image below)

2. just tried it with the rough terrain envs- with the same angle as before (2.5, 0.5, 0.3) it looks like you can see the outline of the pyramid steps, but with the angle (3.5, 3.5, 3.5) you can clearly see the ground, but the color is incorrect

[Mayankm96]

I am unable to reproduce this issue. I added the following to the post init call in the LocomotionVelocityRoughEnvCfg:

# adds tracking of the robot
self.viewer.eye = (2.5, 0.5, 0.3)
self.viewer.origin_type = "asset_root"
self.viewer.asset_name = "robot"

Here's what I see:

My configuration is as follows:

• GPU: NV A6000
• Driver: 535.183.01

[KyleM73]

Are you able to visualize the terrain on flat ground as well? I am able to visualize the terrain as you were for rough terrains, but not for the default flat terrain (None generator, terrain set to ‘plane’, etc.)

[Mayankm96]

When I ran the flat terrain example, that worked for me.

[KyleM73]

Hmmm I'm not sure what's going on then. I tried pulling from main, rebuilding the container and retesting but I'm getting the same result- I can visualize the rough terrains fine but when tracking the asset_root the terrain is invisible for flat terrains... What else could be affecting this? same rsl_rl version? Same train.py options? I'm testing using the default env cfgs in isaac.lab_tasks/*

Note that I'm not doing any of this with the GUI, these are all coming from saved wandb videos, everything I do is in headless mode since I can't do x11 forwarding to my MacBook.

[Mayankm96]

I haven't tested it within a container. Also I was running the random_agent.py script instead of a trained policy.

Will have to check with the above and see if I can reproduce the issue.

[KyleM73]

Hi, I just wanted to see if you had a chance to test this within the container? Thanks @Mayankm96 !

[KyleM73]

This appears to be related to the comment made in #1031 in this comment from @glvov-bdai

https://github.com/isaac-sim/IsaacLab/issues/1031#issuecomment-2374710986

Do you have any suggestion for this specific issue when following the asset with the viewer @glvov-bdai ?

I am using the most recent version of Isaac Lab and Isaac Sim (as of 10/10/24) in the docker workflow. The videos are fine with 1 env, but with 4096 envs the ground is no longer visible, which sounds to me like may be caused by the ground plane not extending far enough? How else could I debug this? With a fixed camera viewer eg not tracking the asset this works fine, however.

[garylvov]

Hi Kyle,

Unfortunately, I was unable to recreate your issue, even when running inside the container.

I put your

self.viewer.eye = (2.5, 0.5, 0.3)
self.viewer.origin_type = "asset_root"
self.viewer.asset_name = "robot"

in the __post__init in the base velocity_cfg env.

I trained with RSL RL, with

 python train.py --task=Isaac-Velocity-Flat-Anymal-D-v0 --headless --video --video_length 100 --video_interval 500

from the /workspace/isaaclab/source/standalone/workflows/rsl_rl directory inside of the container. I grabbed the videos with

docker cp isaac-lab-base:/workspace/isaaclab/source/standalone/workflows/rsl_rl/logs/rsl_rl/anymal_d_flat/2024-10-11_01-40-02/videos/train rsl_rl_vids 

In my video I can see the ground plane

Not sure if I'm doing something differently from you, but some things you could try

A.) Fresh clone and build of Isaac Lab container in a new location separate from what you already have B.) Try using newer NVIDIA cuda / driver C.) Upgrade to 22.04 D.) If it's only for flat environments, and you really want the ground plane, maybe you could try a visualization marker

Let me know if I missed anything would be happy to help you troubleshoot further

System info: Ubuntu 22.04, NVIDIA-SMI 550.54.14, Driver Version: 550.54.14, CUDA Version: 12.4 , NVIDIA GeForce RTX 3080 16Gb Laptop GPU

(P.S anyone at NVIDIA have a spare A6000 or RTX 4090 workstation kicking around they want to donate to my cause :laughing: ? My personal laptop feels like it'll light on fire when running these things :skull: )

[KyleM73]

Great news- updating to driver 550 + docker system prune -a + a fresh clone fixed it. Thanks for the suggestions!

[garylvov]

Ws in the chat bdai interns run the world ;)