Mono-Inertial: Fail to track local map after IMU initialization

[BartKevers]

I would like to use ORB-SLAM3 on images created in Blender. I am running a camera in a circle around a stationary target.

When I feed the rendered images to ORB-SLAM3 in monocular mode, it works without any issues. However, mono+inertial gives a problem. It successfully detects and tracks features until the IMU initializes. Then it fails to track the local map. This continues for the full duration of the simulation. I assume the issue is the data fusion of the inertial and visual measurements, but I have no idea how to solve this. I successfully ran the EuroC visual inertial datasets, so the ORB-SLAM algorithm should work correctly.

The environment is as follows. The camera moves in a constant (rotational and translational) velocity, so I manually computed the (ideal) IMU measurements according to

omega = 2pi / T V = 2pi*r / T a = V^2 / r

I also added the gravitational acceleration. Since r=5m, and T=100s, I obtained the following constant measurements The IMU reference frame used has the same orientation as the opencv standard reference frame., so the transformation matrix is the identity matrix.

I used the following yaml file:

%YAML:1.0

Camera.RGB: 1
Camera.cx: 960.0
Camera.cy: 720.0
Camera.fps: 10
Camera.fx: 4266.666666666667
Camera.fy: 4266.666666666667
Camera.height: 1440.0
Camera.k1: 0.0
Camera.k2: 0.0
Camera.p1: 0.0
Camera.p2: 0.0
Camera.type: PinHole
Camera.width: 1920.0
IMU.AccWalk: 0.0
IMU.Frequency: 100
IMU.GyroWalk: 0.0
IMU.NoiseAcc: 0.0
IMU.NoiseGyro: 0.0
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7
ORBextractor.nFeatures: 1000
ORBextractor.nLevels: 8
ORBextractor.scaleFactor: 1.2
Tbc: !!opencv-matrix
  cols: 4
  data:
    - 1
    - 0
    - 0
    - 0
    - 0
    - 1
    - 0
    - 0
    - 0
    - 0
    - 1
    - 0
    - 0
    - 0
    - 0
    - 1
  dt: f
  rows: 4
Viewer.CameraLineWidth: 3
Viewer.CameraSize: 0.08
Viewer.GraphLineWidth: 0.9
Viewer.KeyFrameLineWidth: 1
Viewer.KeyFrameSize: 0.05
Viewer.PointSize: 2
Viewer.ViewpointF: 500
Viewer.ViewpointX: 0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -3.5

I double checked the camera intrinsics from Blender, they should be right. my camera sensor size is 36x27 mm and the focal length of the lens used is 80 mm. Since I use 'ideal' imu measurements, there is no noise and walk in the gyro and accelerometer measurements.

So why is the local map lost after the IMU initializes and how do I solve it? Thanks in advance!

[vikshree]

Were you able to fix it? I am facing similar issues when running Mon-Inertial SLAM with phone.

[BartKevers]

Were you able to fix it? I am facing similar issues when running Mon-Inertial SLAM with phone.

@vikshree no unfortunately not yet. I'll post it here once I find a solution

[FSRyan]

I have the same problem with mobile data

[RoZhong]

I have the same proble with zed2 camera

[YanhaoZhang]

Hi I got the same problem. Have you solved this problem yet?

[BartKevers]

Hi I got the same problem. Have you solved this problem yet?

@YanhaoZhang no unfortunately I have not managed to find the root problem and a solution for it

[zgxsin]

Also have the same issue. I also saw this output NOT INERTIAL LINK TO PREVIOUS FRAME! quite often

[wrxsky]

I am facing the same issue when i try to run ORBSLAM3 mono-i/stereo-i with Unity. It also works well without imu (mono/stereo).