microsoft
Airsim ROS2 jerky movement and control when capturing imagery
Bug report
- AirSim Version: v1.7.0-linux
- Unreal Version: 4.27.2
- autopilot version: SimpleFlight
- OS Version: Ubuntu 20.04
What's the issue you encountered?
The Airsim ROS2 package produces jerky drone movement every time an image is captured while using any movement command such as the vel_cmd_body_frame subscriber. I verified this by slowing the capture rate by setting the roslaunch update_airsim_img_response_every_n_sec to 1Hz. When flying with a constant velocity the drone stops moving for a split second at 1Hz. When using the default rate of 20Hz the drone struggles to move at all since its movement is interrupted by frequent image captures. This jerky movement gets worse the more cameras or higher resolution images are used. I'm guessing this is due to a threading or client issue with the package although the settings make it seem like they run in separate callback threads. The standard airsim client obviously doesn't produce this behavior. I will also mention that the ROS2 package runs just fine with image capture in realtime when using an RC controller.
This is a show stopper for using the new ROS2 interface and I want to avoid rewriting/forking it. I was hoping to run several vehicles with multiple cameras, but this seems like a currently unscalable proposition. Am I using it incorrectly? Can someone verify that this is a current issue? It seem strange to me that this is a bug/issue when moving and capturing is one of the core use cases of airsim.
Settings
{ "SeeDocsAt": "https://microsoft.github.io/AirSim/settings/", "SettingsVersion": 1.2, "SimMode": "Multirotor", "Vehicles": { "vehicle0": { "VehicleType": "SimpleFlight", "AllowAPIAlways": true, "X": 0, "Y": 0, "Z": 0, "Roll": 0, "Pitch": 0, "Yaw" : 0, "Cameras": { "vehicle0_front": { "CaptureSettings": [ { "ImageType": 0, "Width": 800, "Height": 800, "FOV_Degrees": 90 } ], "X": 0.5, "Y": 0.0, "Z": 0.0, "Pitch": 0.0, "Roll": 0.0, "Yaw": 0.0 }, "vehicle0_frontdepth": { "CaptureSettings": [ { "ImageType": 1, "Width": 800, "Height": 800, "FOV_Degrees": 90 } ], "X": 0.5, "Y": 0.0, "Z": 0.0 , "Pitch": 0.0, "Roll": 0.0, "Yaw": 0.0 } } } } }
@nikola-j I did a quick test with your PR, even when the vehicle stand still, similar behavior of this issue still existed (It seems like the camera processing also affects the timestamp of other topics). You can refer to this issue
Hi @hoangvietdo, do you mean similar behavior to the current issue (jerky movement) or to your issue (timestamps)? Was there a difference in timestamps without my changes? Did you build ros2 airsim in Debug or Release?
Hi @nikola-j, what I mean is that both of these two issues share the same problem which involves running a vehicle and capturing images. This setup causes both time-delay in the timestamp as well as the problem in this issue.
I actually don't know about Debug or Release, I just simply copy and paste your airsim_ros_wrapper.cpp, build, run the simulation and check the timestamp.
The time difference when using a stereo camera has been detected and discussed for a long time in ros1, still, no PR has been proposed to solve it. I have not checked the timestamp before your PR in ros2.
Yes, but this PR isn't meant to fix the issue with the timestamps being out of sync, but to make the ros2 implementation usable. Without this fix, the drone won't move properly, and the timestamps would be even more out of sync.