This work addresses the task of camera localization in a known 3D scene, given a single input RGB image. State-of-the-art approaches accomplish this with two steps. Firstly, regressing for every pixel in the image its so-called 3D scene coordinate and, subsequently, using those coordinates to estimate the final 6D camera pose via RANSAC. To solve the first step, Random Forests (RFs) are typically used. On the other hand, Neural Networks (NNs) currently reign in many dense regression problems, but are not test-time efficient. We ask the question: Which of the two is the best choice for camera localization? To address this, we make two method contributions: (1) a test-time efficient NN architecture which we term a ForestNet that is derived and initialized from a RF, and (2) a new fully-differentiable robust averaging technique for regression ensembles which can be trained end-to-end with a NN architecture. Our experimental findings show that traditional NN architectures are superior to test-time efficient RFs and ForestNets in terms of scene coordinate regression, however, this does not translate to final 6D camera pose accuracy where ForestNets and RFs perform slightly better. To summarize, our best method, a ForestNet with a robust average, improves over the state-of-the-art for camera localization on the 7-Scenes dataset. While this work focuses on scene coordinate regression for camera localization, our innovations may also be applied to other continuous regression tasks.
['Daniela Massiceti', 'Alexander Krull', 'Eric Brachmann', 'Carsten Rother', 'Philip H. S. Torr']
Random Forests versus Neural Networks - What's Best for Camera Relocalization?