Fully convolutional deep correlation networks are integral components of state-ofthe-art approaches to single object visual tracking. It is commonly assumed that these networks perform tracking by detection by matching features of the object instance with features of the entire frame. Strong architectural priors and conditioning on the object representation is thought to encourage this tracking strategy. Despite these strong priors, we show that deep trackers often default to "tracking by saliency" detection -without relying on the object instance representation. Our analysis shows that despite being a useful prior, salience detection can prevent the emergence of more robust tracking strategies in deep networks. This leads us to introduce an auxiliary detection task that encourages more discriminative object representations that improve tracking performance.
This paper investigates representations learned by Siamese trackers. The paper argues that existing trackers rely on saliency detection despite being designed to track by template matching in feature space. An auxiliary detection task is proposed to induce stronger target representations in order to improve tracking performance. Experiments are performed on VOT2018 tracking dataset.
The paper investigates an interesting and active research problem of stronger object representations for deep visual object tracking. However, the proposed solution of just integrating an additional detection task branch within the Siamese tracking architecture is naive. The main idea of integrating instance driven detection as an auxiliary task is borrowed from [1].  [1] also utilizes a Siamese architecture that is similar to the ones generally used in visual object tracking to localize particular instances of objects. Therefore, the novelty of the proposed tracking approach is limited.
Some recent works, such as [2, 3] have also investigated a similar problem of richer object representations for deep visual tracking. These approaches are desired to be discussed and empirically compared in order to fully validate the strength of the proposed approach. 
The paper shows some qualitative analysis. However, most of it is limited to just few frames of an image sequence. Tracking datasets, such as VOT and OTB, provide additional analysis tools (i.e., attribute analysis) to thoroughly evaluate visual trackers. Such analysis is missing in the paper. For instance, the main argument of this paper is that current approaches rely on center saliency and likely struggle in the presence of occlusion. How does the proposed approach fare, compared to SOTA, on the subset of VOT image sequences that are labeled with occlusion?
On page 3, it is stated that "Our model uses a lightweight backbone network (MobileNetV2), and is somewhat simpler than recent state-of-the-art models  .....................  Although the model doesn’t outperform state-of-the-art, it attains competitive performance." The reviewer does not fully agree with this statement. A comprehensive empirical evaluation is crucial to fully access the merits of the contributions. State-of-the-art visual object trackers [2, 3, 4] achieve competitive tracking performance while being computationally efficient and fast. Therefore, a proper state-of-the-art comparison is desired to compare the proposed tracker with SOTA methods. Further, currently experiments are only performed on the VOT2018 dataset. The reviewer recommends to perform additional experiments on other large-scale datasets, such as TrackingNet [5] and Lasot [6] and compare the performance with SOTA methods that are also investigating the problem of richer object representations for tracking. 
[1] Phil Ammirato, Cheng-Yang Fu, Mykhailo Shvets, Jana Kosecka, Alexander C. Berg: Target Driven Instance Detection. CoRR abs/1803.04610 (2018).
[2] Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg: ATOM: Accurate Tracking by Overlap Maximization. CVPR 2019.
[3] Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte: Learning Discriminative Model Prediction for Tracking. CoRR abs/1904.07220 (2019).
[4] Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, Junjie Yan: SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks. CVPR 2019.
[5] Matthias Müller, Adel Bibi, Silvio Giancola, Salman Al-Subaihi, Bernard Ghanem: TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild. ECCV  2018.
[6] Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, Haibin Ling:
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking. CVPR 2019.