This paper presents a robust but simple object pose tracking algorithm based on Kalman filtering. Compared to markerless pose tracking, a fiducial marker called ArUco provides a fast and accurate solution to the problem. With these advantages, this marker-based technique is ready to be used in virtual reality and operate in low-cost wearable devices. However, it still suffers from the problem of occlusion and noise. If a large part of the marker is occluded, no pose information can be acquired for the moment. Noises due to hand shaking also affect the quality of the resulting pose. This is not desirable in a real-time environment. We tackle the problems by employing a linear Kalman filter. The pose information can be estimated even if the camera view is blocked temporarily. We have performed real experiments to demonstrate the effects of the application of Kalman filter.