Camera to IMU transformation

[TreeHugger101]

Hello everyone,

I’m currently working on a project with ORB-SLAM3 (Stereo/Monocular-Inertial mode) and I need some clarification on how the system defines the camera and IMU coordinate axes.

From my understanding so far:

ORB-SLAM3 follows the standard pinhole camera model, where:

x-axis → points right in the image plane

y-axis → points down in the image plane

z-axis → points forward (optical axis)

For the IMU, the convention is less clear to me. In some references I’ve seen:

x-axis → points forward

y-axis → points left

z-axis → points upward

What is the exact coordinate frame definition for the camera and the IMU in ORB-SLAM3?

When specifying the camera-IMU extrinsics in the YAML configuration, should the transform be defined as T_cam_imu (IMU to Camera) or T_imu_cam (Camera to IMU)?

Does ORB-SLAM3 internally enforce any gravity alignment during IMU initialization (e.g., Z-axis aligned with gravity)?

[Samael4999]

From my understanding

The IMU frame is created by rotating the camera frame around its x-axis in that way, that the z-axis aligns with the gravity direction, but z points against the gravity direction (so up instead of down).

The transform should be defined as T_imu_cam (see Calibration Tutorial).

[TreeHugger101]

Thankyou for your response, but i have tried that, i am using a simulated dataset from XPLANE 11, (https://drive.google.com/drive/folders/1iAi7JLshuc3CBjI6Eq-ikuuOEybRvEo3?usp=sharing ) their is no real way to run this with Kalibr to calibrate the parameters, , even with rough calibration that i manually put the tracking verry off, and the commandline output says, resetting, etc etc. could you please help me with this,

About the simulated dataset, camera runs at 25fps and imu at 100hz, using a cessna plane, camera is 1.5m in front and 0.5 lower, wrt to the cockpit view, this is to simulate the camera arrangements in drones, could you please help me with this for my masters project.

thankyou

[Samael4999]

The terminal output says "reseting map" or something like that, I guess?

If thats the case then the initialization of the IMU was not successfull. At the beginning the IMU needs to be moved constantly in preferably more than one direction to initialize. This allows the algorithm to build a map and determine the gravity direction. This is necessary for the algorithm to work.

Other than that i looked at your sensor parameters and they seem okay.
If your tracking is off you could try the methods mentioned here #736
In my experience adjusting the IMU noise parameters (e.g. multiply them by factor 10-20) helps with tracking accuracy in high to mid contrast environments.

[TreeHugger101]

Yes , got it, but in my case. I want to use it for a drone. From small to large scale. In that case. I can't have enough motion. Probably even none at initial few minutes of the run. Is their a work around aldready their for this kind of situations?