  In this paper, we present the first transformer-based model to address the
challenging problem of egocentric gaze estimation. We observe that the
connection between the global scene context and local visual information is
vital for localizing the gaze fixation from egocentric video frames. To this
end, we design the transformer encoder to embed the global context as one
additional visual token and further propose a novel Global-Local Correlation
(GLC) module to explicitly model the correlation of the global token and each
local token. We validate our model on two egocentric video datasets - EGTEA
Gaze+ and Ego4D. Our detailed ablation studies demonstrate the benefits of our
method. In addition, our approach exceeds previous state-of-the-arts by a large
margin. We also provide additional visualizations to support our claim that
global-local correlation serves a key representation for predicting gaze
fixation from egocentric videos. More details can be found in our website
(https://bolinlai.github.io/GLC-EgoGazeEst).

Computer Vision and Pattern Recognition