isaac_ros_visual_slam icon indicating copy to clipboard operation
isaac_ros_visual_slam copied to clipboard
verified publisher icon NVIDIA-ISAAC-ROS

Add Depthai support

Open Serafadam opened this issue 2 years ago • 3 comments

This one adds:

  • Launch file for running VSLAM together with an OAK device
  • Readme for setup
  • Configuration file One issue that seems to appear is that VSLAM node outputs following warnings: [visual_slam_node]: Delta between current and previous frame [44.023430] is above threshold [22.000000] Although camera FPS is ~90 according to ros2 topic echo, not sure what could be the issue is. I'd be happy to improve the integration in the near future. Related PR: https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/pull/92

Serafadam avatar Sep 08 '23 13:09 Serafadam

Hi. I was testing out your PR and I'm running into the issue where the calculated baseline of the images for the VSLAM algorithm is 0 and it's throwing an error. Have you seen this before?

admin@BB-8:/workspaces/isaac_ros-dev$ ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_oak-d.launch.py 
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-10-31-15-56-25-236927-BB-8-13327
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container-1]: process started with pid [13341]
[INFO] [component_container-2]: process started with pid [13343]
[component_container-1] [INFO] [1698785785.688523791] [visual_slam_launch_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_visual_slam/lib/libvisual_slam_node.so
[component_container-2] [INFO] [1698785785.752385267] [oak_container]: Load Library: /opt/ros/humble/install/lib/librobot_state_publisher_node.so
[component_container-2] [INFO] [1698785785.756013469] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<robot_state_publisher::RobotStatePublisher>
[component_container-2] [INFO] [1698785785.756044347] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<robot_state_publisher::RobotStatePublisher>
[component_container-1] [INFO] [1698785785.757271811] [visual_slam_launch_container]: Found class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_slam::VisualSlamNode>
[component_container-1] [INFO] [1698785785.757302819] [visual_slam_launch_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_slam::VisualSlamNode>
[component_container-2] [INFO] [1698785785.763172940] [oak_state_publisher]: got segment oak
[component_container-2] [INFO] [1698785785.763196795] [oak_state_publisher]: got segment oak-d-base-frame
[component_container-2] [INFO] [1698785785.763201844] [oak_state_publisher]: got segment oak_imu_frame
[component_container-2] [INFO] [1698785785.763205631] [oak_state_publisher]: got segment oak_left_camera_frame
[component_container-2] [INFO] [1698785785.763209298] [oak_state_publisher]: got segment oak_left_camera_optical_frame
[component_container-2] [INFO] [1698785785.763212925] [oak_state_publisher]: got segment oak_model_origin
[component_container-2] [INFO] [1698785785.763216572] [oak_state_publisher]: got segment oak_rgb_camera_frame
[component_container-2] [INFO] [1698785785.763219908] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame
[component_container-2] [INFO] [1698785785.763223084] [oak_state_publisher]: got segment oak_right_camera_frame
[component_container-2] [INFO] [1698785785.763226310] [oak_state_publisher]: got segment oak_right_camera_optical_frame
[component_container-2] [INFO] [1698785785.764516242] [oak_container]: Load Library: /workspaces/isaac_ros-dev/install/depthai_ros_driver/lib/libdepthai_ros_driver.so
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak_state_publisher' in container 'oak_container'
[component_container-1] [INFO] [1698785785.770041846] [visual_slam_node]: cuVSLAM version: 11.0
[component_container-2] [INFO] [1698785785.816698090] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1698785785.816761609] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1698785785.822580033] [oak]: No ip/mxid specified, connecting to the next available device.
[component_container-1] [INFO] [1698785786.657038580] [visual_slam_node]: Time taken by CUVSLAM_WarmUpGPU(): 0.886957
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/visual_slam_node' in container '/visual_slam_launch_container'
[component_container-2] [INFO] [1698785788.514943290] [oak]: Camera with MXID: 18443010B176681200 and Name: 1.1 connected!
[component_container-2] [INFO] [1698785788.515844091] [oak]: USB SPEED: HIGH
[component_container-2] [INFO] [1698785788.562302813] [oak]: Device type: OAK-D-LITE
[component_container-2] [INFO] [1698785788.565119361] [oak]: Pipeline type: depth
[component_container-2] [ERROR] [1698785788.578589720] [oak]: rgb parameters not set, defaulting to 1280x720 unless specified otherwise.
[component_container-2] [WARN] [1698785789.616953828] [oak]: IMU enabled but not available!
[component_container-2] [INFO] [1698785789.617959877] [oak]: Finished setting up pipeline.
[component_container-2] [INFO] [1698785789.816932580] [oak]: Setting up stereo pair sync timer with period 11 ms based on left sensor FPS.
[component_container-2] [INFO] [1698785790.284650386] [oak]: Published URDF
[component_container-2] [WARN] [1698785790.285325614] [oak]: IMU extrinsics appear to be default. Check if the IMU is calibrated.
[component_container-2] [INFO] [1698785790.287026107] [oak]: Camera ready!
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak' in container '/oak_container'
[component_container-1] [INFO] [1698785790.288250766] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.288284008] [visual_slam_node]: Baseline is too small: -0.000000
[component_container-1] [INFO] [1698785790.290741352] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.290777780] [visual_slam_node]: Baseline is too small: -0.000000
[component_container-2] [INFO] [1698785790.294824547] [oak_state_publisher]: got segment oak
[component_container-2] [INFO] [1698785790.294850256] [oak_state_publisher]: got segment oak-d-base-frame
[component_container-2] [INFO] [1698785790.294855896] [oak_state_publisher]: got segment oak_model_origin
[component_container-1] [INFO] [1698785790.302124382] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.302184675] [visual_slam_node]: Baseline is too small: -0.000000
[component_container-1] [INFO] [1698785790.313034194] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.313105829] [visual_slam_node]: Baseline is too small: -0.000000

mstiehm-NF avatar Nov 01 '23 13:11 mstiehm-NF

Hi @mstiehm-NF, thanks for the notification, I'll investigate it next week, I assume that some things changed in the latest release of VSLAM library

Serafadam avatar Nov 03 '23 16:11 Serafadam