Extrinsic IMU Calibration in .json

[peetzha]

For my application I would like to access to all calibration information through the ROS2 topic /tf_static. Setting publish_tf_from_calibration:= true in the camera.launch file of depthai-ros works well for publishing the correct camera calibration. However I struggle with showing the right IMU extrinsics.

For some background, following the suggestion in https://github.com/luxonis/depthai-ros/issues/562, I use the depthai/calibrate.py script for the Stereo calibration and Kalibr https://github.com/ethz-asl/kalibr/wiki/installation for the IMU extrinsics. I manually add the Kalibr IMU extrinsics to the .json file, which I flash to the device.

The specific problem is that I don't understand how the rotation matrix and translation is defined within the .json calibration files:

"imuExtrinsics": {
        "rotationMatrix": [
            [0.0, 0.0, 0.0],
            [0.0, 0.0, 0.0],
            [0.0, 0.0, 0.0],
        ],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },

From my expectation, "rotationMatrix" was supposed to be R_imu_camSocket and translation the camSocket coordinate within the IMU frame.

However from my testing it seems that the rotation (show in /tf_static) gets left-multiplied by

[[ 0. -1.  0.]
 [ 0.  0. -1.]
 [ 1.  0.  0.]],

which seems to be the rotation $R_{camopt-cam}$ between the camera_optical_frames and the camera_frames. Also the translation appear to be within some fixed coordinate frame.

Could you explain what the expected format for the IMU extrinsics is?

P.S.: as a temporary fix, for the "rotationMatrix" I am using $R_{cam-camopt}R_{IMU-cam}$ which logically does not make sense, but gives a result which matches orientation and placement of the model-based extrinsics.