  We present an on-line 3D visual object tracking framework for monocular
cameras by incorporating spatial knowledge and uncertainty from semantic
mapping along with high frequency measurements from visual odometry. Using a
combination of vision and odometry that are tightly integrated we can increase
the overall performance of object based tracking for semantic mapping. We
present a framework for integration of the two data-sources into a coherent
framework through information based fusion/arbitration. We demonstrate the
framework in the context of OmniMapper[1] and present results on 6 challenging
sequences over multiple objects compared to data obtained from a motion capture
systems. We are able to achieve a mean error of 0.23m for per frame tracking
showing 9% relative error less than state of the art tracker.

Computer Vision and Pattern Recognition