So recently I found myself needing to use Google’s Ceres Solver to perform a non-linear batch optimization for my masters. This should have been easy as Ceres makes implementing this sort of thing rather trivial. I took a look at their “Simple bundle adjustment” code and soon realized that my Eigen Quaternions and the Ceres Quaternions are in a different order.
Eigen uses the correct initialization order [w, x, y, z] however in memory it stores Quaternions as [x, y, z, w]. Thus you cannot use the Ceres QuaternionLocalParameterization to ensure the optimization generates normalized quaternions. Furthermore, you cannot use any of the Ceres rotation functions to transform the world points into the camera frame. So after some investigation and some help from the awesome guys on the Ceres Solver Google group I finally got it all working. I implemented a LocalParameterization to handle Eigen Quaternions and with some help I managed to use the Eigen based rotations with Ceres Jet objects.