summaries: This paper collects a new validation dataset via crowdsourcing to benchmark the progress of state-of-the-art VQA models.  The dataset (containing 22K questions and 10x answers) is adversarially collected such that annotators have to come up with questions that can successfully fool a state-of-the-art VQA model, and such questions are undoubtedly more challenging to various existing models. The authors perform thorough data analysis as well as extensive experiments with a variety existing models, to validate the usefulness of the new dataset. This new dataset can potentially be a valuable benchmark to track the progress of VQA research, if there is no obvious bias in the dataset (potential dataset bias is not thoroughly discussed in this paper).  ========================================================================================  Thanks for the response from the authors. I've updated my rating after reading the rebuttal as well as the comments from other reviewers.

summaries: This paper works on the topic of visual question answering and aims to test how well state-of-the-art models perform on adversarial examples. Specifically, the authors collect a new adversarial benchmark on VQA, named "Adversarial VQA" (AdVQA), including a large evaluation dataset of 28,522 examples in total. The data samples in AdVQA are collected by putting both human annotators and state-of-the-art models in the collection loop: Annotators are requested to create questions that can fool the state-of-the-art models, and these questions are further verified by other annotators. The authors benchmark state-of-the-art models on these collected adversarial questions and their performance drops significantly.  The main contribution of this paper is the collected AdVQA dataset which includes adversarial examples created by human. The experimental results suggest future research directions on VQA.

summaries: Update after rebuttal: Thanks to the authors for their rebuttal in response to my questions/concerns. After reading the rebuttal and the other reviews, I am updating my score to 6, to better reflect the contributions made by the work. _______________________________________________________________________________________________________________ The paper introduces an adversarial VQA dataset collected with human-and-model in the loop by directly asking humans to write questions to attack the winning model of VQA challenge. Extensive evaluation of existing models on this adversarial VQA dataset suggest that current VQA systems are far from really solving the VQA problem. Many well-performed models on VQA v2 seem to fail on AdVQA. AdVQA potentially can be used to benchmark the advances of VQA models in the future, considering VQA v2 performance is close to human parity.

summaries: This paper proposes AdVQA, a new (adversarial) test set for the Visual Question Answering.  Annotators and a SOTA model are put in the loop to make sure the model can be fooled. Experiments show that other VQA models tend to perform poorly on AdVQA, and a comprehensive analysis discusses the questions and answers that are in this dataset (compared with VQA and TextVQA).

abstractText: Performance on the most commonly used Visual Question Answering dataset (VQA v2) is starting to approach human accuracy. However, in interacting with state-of-the-art VQA models, it is clear that the problem is far from being solved. In order to stress test VQA models, we benchmark them against human-adversarial examples. Human subjects interact with a state-of-the-art VQA model, and for each image in the dataset, attempt to find a question where the model’s predicted answer is incorrect. We find that a wide range of state-of-the-art models perform poorly when evaluated on these examples. We conduct an extensive analysis of the collected adversarial examples and provide guidance on future research directions. We hope that this Adversarial VQA (AdVQA) benchmark can help drive progress in the field and advance the state of the art.

authors:
Vedika Agarwal
Rakshith Shetty
Mario Fritz

title: Towards causal vqa: Revealing and reducing spurious correlations by invariant and covariant semantic editing

venue: In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition,

authors:
Aishwarya Agrawal
Dhruv Batra
Devi Parikh
Aniruddha Kembhavi

title: Don’t just assume; look and answer: Overcoming priors for visual question answering

authors:
Peter Anderson
Xiaodong He
Chris Buehler
Damien Teney
Mark Johnson
Stephen Gould
Lei Zhang

title: Bottom-up and top-down attention for image captioning and visual question answering

venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,

authors:
Stanislaw Antol
Aishwarya Agrawal
Jiasen Lu
Margaret Mitchell
Dhruv Batra
C. Lawrence Zitnick
Devi Parikh

title: VQA: Visual Question Answering

venue: In Proceedings of IEEE/CVF International Conference on Computer Vision,

authors:
Max Bartolo
Alastair Roberts
Johannes Welbl
Sebastian Riedel
Pontus Stenetorp

title: Beat the ai: Investigating adversarial human annotation for reading comprehension

venue: Transactions of the Association for Computational Linguistics,

authors:
Ali Furkan Biten
Ruben Tito
Andres Mafla
Lluis Gomez
Marçal Rusinol
Ernest Valveny
CV Jawahar
Dimosthenis Karatzas

title: Scene text visual question answering

authors:
Fedor Borisyuk
Albert Gordo
Viswanath Sivakumar

title: Rosetta: Large scale system for text detection and recognition in images

venue: In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining,

authors:
Kaylee Burns
Lisa Anne Hendricks
Trevor Darrell
Anna Rohrbach

title: Women also snowboard: Overcoming bias in captioning models

venue: In Proceedings of European Conference on Computer Vision,

authors:
Nicholas Carlini
David Wagner

title: Towards evaluating the robustness of neural networks

venue: In 2017 ieee symposium on security and privacy (sp),

authors:
Xinlei Chen
Hao Fang
Tsung-Yi Lin
Ramakrishna Vedantam
Saurabh Gupta
Piotr Dollár
C Lawrence Zitnick

title: Microsoft coco captions: Data collection and evaluation

venue: server. arXiv preprint arXiv:1504.00325,

authors:
Yen-Chun Chen
Linjie Li
Licheng Yu
Ahmed El Kholy
Faisal Ahmed
Zhe Gan
Yu Cheng
Jingjing Liu

title: Uniter: Universal image-text representation learning

authors:
Moustapha Cisse
Piotr Bojanowski
Edouard Grave
Yann Dauphin
Nicolas Usunier

title: Parseval networks: Improving robustness to adversarial examples

venue: In Proceedings on International Conference on Machine Learning,

authors:
Jia Deng
Wei Dong
Richard Socher
Li-Jia Li
Kai Li
Li Fei-Fei

title: Imagenet: A large-scale hierarchical image database

authors:
Jacob Devlin
Ming-Wei Chang
Kenton Lee
Kristina Toutanova. Bert

title: Pre-training of deep bidirectional transformers for language understanding

venue: arXiv preprint arXiv:1810.04805,

authors:
Emily Dinan
Samuel Humeau
Bharath Chintagunta
Jason Weston

title: Build it break it fix it for dialogue safety: Robustness from adversarial human attack

venue: In Proceedings of Empirical Methods in Natural Language Processing,

authors:
Akira Fukui
Dong Huk Park
Daylen Yang
Anna Rohrbach
Trevor Darrell
Marcus Rohrbach

title: Multimodal compact bilinear pooling for visual question answering and visual grounding

authors:
Zhe Gan
Yen-Chun Chen
Linjie Li
Chen Zhu
Yu Cheng
Jingjing Liu

title: Large-scale adversarial training for vision-and-language representation learning

venue: In Proceedings of Conference on Advances in Neural Information Processing Systems,

authors:
Tejas Gokhale
Pratyay Banerjee
Chitta Baral
Yezhou Yang

title: Vqa-lol: Visual question answering under the lens of logic

authors:
Ian J Goodfellow
Jonathon Shlens
Christian Szegedy

title: Explaining and harnessing adversarial examples

venue: In Proceedings of International Conference on Learning Representations,

authors:
Yash Goyal
Tejas Khot
Douglas Summers-Stay
Dhruv Batra
Devi Parikh

title: Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering

authors:
Danna Gurari
Qing Li
Abigale J. Stangl
Anhong Guo
Chi Lin
Kristen Grauman
Jiebo Luo
Jeffrey P. Bigham

title: Vizwiz grand challenge: Answering visual questions from blind people

authors:
Kaiming He
Xiangyu Zhang
Shaoqing Ren
Jian Sun

title: Deep residual learning for image recognition

authors:
Sepp Hochreiter
Jürgen Schmidhuber

title: Long short-term memory

venue: Neural computation,

authors:
Ronghang Hu
Amanpreet Singh

title: Unit: Multimodal multitask learning with a unified transformer

venue: arXiv preprint arXiv:2102.10772,

authors:
Ronghang Hu
Amanpreet Singh
Trevor Darrell
Marcus Rohrbach

title: Iterative answer prediction with pointer-augmented multimodal transformers for textvqa

authors:
Drew A. Hudson
Christopher D. Manning

title: Gqa: A new dataset for real-world visual reasoning and compositional question answering

authors:
Yu Jiang
Vivek Natarajan
Xinlei Chen
Marcus Rohrbach
Dhruv Batra
Devi Parikh

title: Pythia v0. 1: the winning entry to the vqa challenge 2018

venue: arXiv preprint arXiv:1807.09956,

authors:
Justin Johnson
Bharath Hariharan
Laurens van der Maaten
Li Fei-Fei
C. Lawrence Zitnick
Ross B. Girshick

title: CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning

authors:
Douwe Kiela
Max Bartolo
Yixin Nie
Divyansh Kaushik
Atticus Geiger
Zhengxuan Wu
Bertie Vidgen
Grusha Prasad
Amanpreet Singh
Pratik Ringshia
Zhiyi Ma
Tristan Thrush
Sebastian Riedel
Zeerak Waseem
Pontus Stenetorp
Robin Jia
Mohit Bansal
Christopher Potts
Adina Williams

title: Dynabench: Rethinking benchmarking in nlp

venue: In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics,

authors:
Douwe Kiela
Suvrat Bhooshan
Hamed Firooz
Ethan Perez
Davide Testuggine

title: Supervised multimodal bitransformers for classifying images and text

authors:
Douwe Kiela
Hamed Firooz
Aravind Mohan
Vedanuj Goswami
Amanpreet Singh
Pratik Ringshia
Davide Testuggine

title: The hateful memes challenge: Detecting hate speech in multimodal memes

authors:
Wonjae Kim
Bokyung Son
Ildoo Kim

title: Vilt: Vision-and-language transformer without convolution or region supervision

venue: arXiv preprint arXiv:2102.03334,

authors:
Diederik P Kingma
Jimmy Ba

title: Adam: A method for stochastic optimization

authors:
Ranjay Krishna
Yuke Zhu
Oliver Groth
Justin Johnson
Kenji Hata
Joshua Kravitz
Stephanie Chen
Yannis Kalantidis
Li-Jia Li
David A Shamma

title: Visual genome: Connecting language and vision using crowdsourced dense image annotations

venue: Proceedings of International Journal of Computer Vision,

authors:
Jie Lei
Licheng Yu
Tamara L Berg
Mohit Bansal

title: What is more likely to happen next? video-and-language future event prediction

authors:
Linjie Li
Jie Lei
Zhe Gan
Jingjing Liu

title: Adversarial vqa: A new benchmark for evaluating the robustness of vqa models

authors:
Liunian Harold Li
Mark Yatskar
Da Yin
Cho-Jui Hsieh
Kai-Wei Chang

title: Visualbert: A simple and performant baseline for vision and language

venue: In Proceedings of Annual Meeting of the Association for Computational Linguistics,

authors:
Wei Li
Can Gao
Guocheng Niu
Xinyan Xiao
Hao Liu
Jiachen Liu
Hua Wu
Haifeng Wang

title: Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning

authors:
Tsung-Yi Lin
Michael Maire
Serge Belongie
James Hays
Pietro Perona
Deva Ramanan
Piotr Dollár
C Lawrence Zitnick

title: Microsoft coco: Common objects in context

venue: In European conference on computer vision,

authors:
Ilya Loshchilov
Frank Hutter

title: Decoupled weight decay regularization

authors:
Jiasen Lu
Dhruv Batra
Devi Parikh
Stefan Lee

title: Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

venue: arXiv preprint arXiv:1908.02265,

authors:
Jiasen Lu
Vedanuj Goswami
Marcus Rohrbach
Devi Parikh
Stefan Lee

title: 12-in-1: Multitask vision and language representation learning

authors:
Kenneth Marino
Mohammad Rastegari
Ali Farhadi
Roozbeh Mottaghi

title: Ok-vqa: A visual question answering benchmark requiring external knowledge

authors:
Duy-Kien Nguyen
Vedanuj Goswami
Xinlei Chen

title: Revisiting modulated convolutions for visual counting and beyond

authors:
Yixin Nie
Adina Williams
Emily Dinan
Mohit Bansal
Jason Weston
Douwe Kiela

title: Adversarial nli: A new benchmark for natural language understanding

authors:
Vicente Ordonez
Girish Kulkarni
Tamara Berg

title: Im2text: Describing images using 1 million captioned photographs

authors:
Adam Paszke
Sam Gross
Francisco Massa
Adam Lerer
James Bradbury
Gregory Chanan
Trevor Killeen
Zeming Lin
Natalia Gimelshein
Luca Antiga
Alban Desmaison
Andreas Köpf
Edward Yang
Zach DeVito
Martin Raison
Alykhan Tejani
Sasank Chilamkurthy
Benoit Steiner
Lu Fang
Junjie Bai
Soumith Chintala

title: Pytorch: An imperative style, high-performance deep learning library

authors:
Christopher Potts
Zhengxuan Wu
Atticus Geiger
Douwe Kiela

title: Dynasent: A dynamic benchmark for sentiment analysis

authors:
Pedro Rodriguez
Shi Feng
Mohit Iyyer
He He
Jordan Boyd-Graber

title: Quizbowl: The case for incremental question answering

venue: arXiv preprint arXiv:1904.04792,

authors:
Ramprasaath R Selvaraju
Purva Tendulkar
Devi Parikh
Eric Horvitz
Marco Tulio Ribeiro
Besmira Nushi
Ece Kamar

title: Squinting at vqa models: Introspecting vqa models with sub-questions

authors:
Meet Shah
Xinlei Chen
Marcus Rohrbach
Devi Parikh

title: Cycle-consistency for robust visual question answering

authors:
Piyush Sharma
Nan Ding
Sebastian Goodman
Radu Soricut

title: Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning

authors:
Karen Simonyan
Andrew Zisserman

title: Very deep convolutional networks for large-scale image recognition

venue: arXiv preprint arXiv:1409.1556,

authors:
Amanpreet Singh
Vedanuj Goswami
Vivek Natarajan
Yu Jiang
Xinlei Chen
Meet Shah
Marcus Rohrbach
Dhruv Batra
Devi Parikh

title: Mmf: A multimodal framework for vision and language research

authors:
Amanpreet Singh
Vedanuj Goswami
Devi Parikh

title: Are we pretraining it right? digging deeper into visio-linguistic pretraining

venue: arXiv preprint arXiv:2004.08744,

authors:
Amanpreet Singh
Vivek Natarajan
Meet Shah
Yu Jiang
Xinlei Chen
Dhruv Batra
Devi Parikh
Marcus Rohrbach

title: Towards vqa models that can read

authors:
Alexander Trott
Caiming Xiong
Richard Socher

title: Interpretable counting for visual question answering

authors:
Ashish Vaswani
Noam Shazeer
Niki Parmar
Jakob Uszkoreit
Llion Jones
Aidan N Gomez
Lukasz Kaiser
Illia Polosukhin

title: Attention is all you need

authors:
Bertie Vidgen
Tristan Thrush
Zeerak Waseem
Douwe Kiela

title: Learning from the worst: Dynamically generated datasets to improve online hate detection

authors:
Zhou Yu
Jun Yu
Yuhao Cui
Dacheng Tao
Qi Tian

title: Deep modular co-attention networks for visual question answering

venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,

authors:
Jieyu Zhao
Tianlu Wang
Mark Yatskar
Vicente Ordonez
Kai-Wei Chang

title: Men also like shopping: Reducing gender bias amplification using corpus-level constraints

heading: 1 Introduction

text:
Visual question answering (VQA) is widely recognized as an important evaluation task for vision and language research. Besides direct applications such as helping the visually impaired or multimodal content understanding on the web, it offers a mechanism for probing machine understanding of images via natural language queries. Making progress on VQA requires bringing together different subfields in AI – combining advances from natural language processing (NLP) and computer vision together with those in multimodal fusion – making it an exciting task in AI research.
Over the years, the performance of VQA models has started to plateau on the popular VQA v2 dataset [20] – approaching inter-human agreement – as evidenced by Fig. 1. This raises important questions for the field: To what extent have we solved the problem? If we haven’t, what are we still missing? How good are we really?
An intriguing method for investigating these questions is dynamic data collection [29], where human annotators and state-of-the-art models are put “in the loop” together to collect data adversarially. Annotators are tasked with and rewarded for finding model-fooling examples, which are then verified by other humans. The easier it is to find such examples, the worse the model’s performance can be said to be. The collected data can be used to “stress test” current VQA models and serve as the next iteration of the VQA benchmark helping drive further progress.
The commonly used VQA dataset [20] was collected by instructing annotators to “ask a question about this scene that [a] smart robot probably can not answer” [4]. One way of thinking about our proposed human-adversarial data collection is that it explicitly ensures that the questions can not be answered by today’s “smartest” models. ∗Equal contribution. Correspondence to advqa@fb.com. †Work done as an intern at Facebook AI Research.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
This work is, to the best of our knowledge, the first to apply this human-adversarial approach to an image and language multimodal problem. We introduce Adversarial VQA (AdVQA), a large evaluation dataset of 46,807 examples in total, all of which fooled the VQA 2020 challenge winner, the MoViE+MCAN [45] model.
We evaluate a wide range of existing VQA models on AdVQA and find that their performance is significantly lower than on the commonly used VQA v2 dataset [20] (see Table 1). Furthermore, we conduct an extensive analysis of AdVQA characteristics, and contrast with the VQA v2 dataset.
We hope that this new benchmark can help advance the state of the art by shedding important light on the current model shortcomings. Our findings suggest that there is still considerable room for continued improvement, with much more work remaining to be done.

heading: 2 Related Work

text:
Stress testing VQA. Several attempts exist for stress testing VQA models. Some examine to what extent VQA models’ predictions are grounded in the image content, as opposed to them relying primarily on language biases learned from the training dataset. The widely used VQA v2 dataset [20] was one attempt at this. The dataset contains pairs of similar images that have different answers to the same question, rendering a language-based prior inadequate. VQA under changing priors (VQA-CP) [2] is a more stringent test where the linguistic prior not only is weak in the test set, but is adversarial relative to the training dataset (i.e., answers that are popular for a question type during training are rarer in the test set and vice versa). Other complementary datasets test model robustness against certain conditions, such as different logicial compositions [18], question
rephrasings [52], suprious correlations [1], and consistency in answering sub-questions [51]. Datasets which benchmark specific capabilities in VQA models also exist, such as reading and reasoning about text in images [57], leveraging external knowledge [44], and spatial [26] or compositional reasoning [28]. Other vision and language datasets, such as Hateful Memes [31], have also tried to make sure the task involves true multimodal reasoning, as opposed to any of the individual modalities sufficing for arriving at the correct label. Compared to the previous work, with AdVQA, we aim to have a more holistic and general dataset that pushes the boundaries of the state-of-the-art on VQA.
Saturating prior work. The VQA v2 [20] challenge has been running yearly since 2016 and has seen tremendous progress, as can be seen in Figure 1. From simple LSTM [23] and VGGNet [54] fused models to more advanced fusion techniques (MCB [16], Pythia [27]) to better object detectors (MCAN [61], MoViE+MCAN [45], BUTD [3]) to transformers [59, 56] and self-supervised pretraining (MCAN [61], UNIMO [38]), the community has brought the performance of models on the VQA v2 dataset close to human accuracy, thus starting to saturate progress on dataset. As we show through AdVQA, however, saturation is far from achieved on the overall task, and hence there is a need for new datasets to continue benchmarking progress in vision and language reasoning.
natural language inference [46], question answering [5, 50], sentiment analysis [49], hate
speech detection [60], next video-and-language event prediction [35] and dialogue safety [15]. To the best of our knowledge, ours is the first work to explore an image and language multimodal human-adversarial benchmark.1

heading: 3 AdVQA

text:
The aim of this work is to investigate state of the art VQA model performance via human-adversarial data collection. In this human-and-model-in-the-loop paradigm, human annotators are tasked with finding examples that fool the model. In this case, annotators are shown an image and are tasked with asking difficult but valid questions that the model answers incorrectly. We collected our dataset using Dynabench [29], a platform for dynamic adversarial data collection and benchmarking. For details on our labeling user interface, please refer to the supplementary material.
The VQA v2 dataset is based on COCO [39] images. We collected adversarial questions on both val2017 COCO images and testdev2015 COCO images. We then random sampled the collected set down to 2 questions per val2017 COCO images (10,000) and 1 question per testdev2015 COCO images (36,807). The random sampling was done to balance the annotator’s contributions. The data collection involves three phrases (Figure 2): First, in the question collection phase, Amazon Mechanical Turk workers interact with a state-of-the-art VQA model and self-report image-question pairs that the model failed to answer correctly. Second, in the question validation phase, a new set
1Concurrent work [36] jointly available with AdVQA at https://adversarialvqa.org also focuses on the same problem but uses images from a different source.
of workers validates whether the answers provided by the VQA model for the image-question pairs collected in the first phrase are indeed incorrect. Finally, in the answer collection phase, we collect 10 ground truth answers for the image-question pairs validated in the second phase. In what follows, we provide further details for each of these steps.

heading: 3.1 Question Collection

text:
In this phase, annotators on Amazon MTurk are shown an image and are tasked with asking questions about this image. The interface has a state-of-the-art VQA model in the loop. For each question the annotator asks, the model produces an answer. Annotators are asked to come up with questions that fool the model. Annotators are done (i.e., can submit the HIT) once they successfully fooled the model or after they tried a minimum of 10 questions (whichever occurs first). To account for the fact that it may be hard to think of many questions for some images, annotators are given the option to skip an image after providing non-fooling three questions. We use the VQA Challenge 2020 winner – MoViE+MCAN [45] – trained on the COCO 2017 train set as the model in the loop. This is to ensure that the adversarial benchmark we collect is challenging for the current state-of-the-art in VQA.
The interface provides a digital magnifier to zoom into an area of the image when the mouse hovers over it, allowing workers to examine the image closely if necessary. See Figure 3b. Keyboard shortcuts were provided for convenience.

heading: 3.2 Question Validation

text:
Note that in the question collection phase, annotators self-report when they have identified a question that is incorrectly answered by the model. Whether or not this was actually the case is verified in the question validation phase. Two different annotators are shown the image, question, and answer
predicted by the model, and asked whether the model’s answer is correct. As an additional quality control, the annotator is also asked whether the question is “valid”. A valid question is one where the image is necessary and sufficient to answer the question. Examples of invalid questions are: “What is the capital of USA?” (does not need an image) or “What is the person doing?” when the image does not contain any people or where there are multiple people doing multiple things. If the two validators disagree, a third annotator is used to break the tie. Examples are added to the dataset only if at least two annotators agree that the question is valid.

heading: 3.3 Answer Collection

text: In the final stage of data collection, following [20, 57], we collect 10 answers per question providing instructions similar to those used in [20] making sure that no annotator sees the same question twice. In addition, as an extra step of caution, to filter bad questions that might have passed through last two stages and to account for ambiguity that can be present in questions, we allow annotators to select “unanswerable” as an answer. Further, to ensure superior data quality, we occasionally provide annotators with hand-crafted questions for which we know the non-ambiguous single true answer, as a means to identify and filter out annotators providing poor quality responses.

heading: 3.4 Human-Adversarial Annotation Statistics

text:
The statistics for the first two stages (question collection and validation) are shown in Table 2. We find that annotators took about 5 tries and on average around 4 minutes to find a model-fooling example. For computing the model error rate, we can look at the instances where annotators claimed they had fooled the model, and where annotators were verified by other annotators to have fooled the model. The latter has been argued to be a particularly good metric for measuring model performance [29]. We also further confirm that the model was indeed fooled by running the model-in-the-loop on a subset of examples in which human agreement was 100% and found the accuracy to be 0.15% on val, and 0.18% on test splits of AdVQA respectively.
Interestingly, the “claimed” model error rate, based on self-reported model-fooling questions, is similar to that of text-based adversarial question answering, which was at 44.0% for RoBERTa and 47.1% for BERT [5]. The validated error rate for our task is much higher than e.g. for ANLI [46], which was 9.52% overall, suggesting that fooling models is a lot easier for VQA than for NLI. It appears to be not too difficult for annotators to find examples that the model fails to predict correctly.

heading: 4 Model Evaluation

heading: 4.1 Baselines and Methods

text:
We analyze the performance of several baselines and a wide variety of state-of-the-art VQA models on the AdVQA dataset. We evaluate the same set of models on VQA v2 dataset as a direct comparison.
Prior baselines. We start by evaluating two prior baselines: answering based on the overall majority answer, or the per answer type majority answer in the validation dataset. We use the same answer types as in [4]. The overall majority answer in AdVQA is no. The majority answer is no for “yes/no”, 2 for “numbers” and unanswerable for “others”. See Section 5 for more details on answer types.
Unimodal baselines. Next, we evaluate two unimodal pretrained models: i) ResNet-152 [22] pretrained on Imagenet [13]; and ii) BERT [14], both finetuned on the task using the visual (image)
or textual (question) modality respectively, while ignoring the other modality. We observe that the unimodal text model performs better than unimodal image for both VQA and AdVQA.
Multimodal methods. We evaluate two varieties of multimodal models: i) unimodal pretrained and ii) multimodal pretrained. In the unimodal pretrained category we explore MMBT [30], MoViE+MCAN [45] and UniT [24]. These models are initialized from unimodal pretrained weights: BERT pretraining for MMBT; Imagenet + Visual Genome [34] detection pretraining for MoViE+MCAN; and Imagenet + COCO [40] detection pretraining for the image encoder part and BERT pretraining for the text encoder part in UniT. In the multimodal pretrained category, we explore VisualBERT [37], VilBERT [42, 43], VilT [32], UNITER[11] and VILLA [17]. These models are first initialized from pretrained unimodal models and then pretrained on different multimodal datasets on proxy self-supervised/semi-supervised tasks before finetuning on VQA. VisualBERT is pretrained on COCO Captions [10]; VilBERT is pretrained on Conceptual Captions [53]; ViLT, UNITER and VILLA models are pretrained on COCO Captions [10] + Visual Genome [34] + Conceptual Captions [53] + SBU Captions [47] datasets.
We find that multimodal models in general perform better than unimodal models, as we would expect given that both modalities are important for the VQA task.
Multimodal OCR methods. As we will see in Section 5, a significant amount of questions in AdVQA can be answered using scene text.
We test a state-of-the-art TextVQA [55] model, M4C [25] on AdVQA. We evaluate two versions: (i) trained on VQA 2.0 dataset, and (ii) trained on TextVQA [55] and STVQA [6]. In both cases, we use OCR tokens extracted using the Rosetta OCR system [7]. We also use the same answer vocabulary used by other models for fair comparison.

heading: 4.2 Discussion

text:
Our model evaluation results show some surprising findings. We discuss our observations and hypotheses around these findings in this section.
Baseline comparison. Surprisingly, most multimodal models are unable to outperform a unimodal baseline (BERT [14]) and simple majority answer (per answer type) prior baseline which only predicts the most-frequently occurring word for the question’s category (no for “yes/no”, 2 for “numbers” and
unanswerable for “others”). A detailed breakdown of category-wise performance provided in Table 4 suggests that even though “yes/no” questions are often considered to be easy, the baseline multimodal models we evaluate are unable to beat the majority answer prediction. We also observe varied but close to majority answer performance in the “numbers” category, which is a question type known to be difficult for VQA models [45]. In contrast, the models outperform the majority answer in the “others” category even though “unanswerable” (the majority answer) is not in the answer vocabulary used. Interestingly, M4C outperforms all on “numbers” and “others” categories possibly thanks to its text reading capabilities. These trends showcase the difficulty of AdVQA and suggest that we have a long way to go still, given that we are as yet apparently unable to beat such simple baselines.
Model rankings. First, M4C [25] performs the best among the evaluated models. Interestingly, it is smaller than many of the more sophisticated model architectures that score higher on VQA. This is probably due to the importance of the ability to read and reason about text in the image for answering some AdVQA questions. Second, among models that can’t read text, UniT [24] is the best model, despite (or perhaps because of?) it being only unimodal pretrained. As UniT was trained jointly on unimodal as well as multimodal tasks, perhaps, it is less prone to biases in multimodal datasets. Third, the adversarially-trained VILLA [17] model performs surprisingly poorly. While it may be more robust to statistically-generated adversarial examples, it appears to be less so against human-adversarial examples. Fourth, we find that all of these models perform poorly compared to humans, while model performance on VQA is much closer to that of humans. A more detailed analysis on model capabilities required for AdVQA and BERT’s superior performance can be found in Section B.2 and B.3.
Table 4: The category-wise performance of VQA models. The state-of-the-art VQA models perform very close to the majority class prior, illustrating the challenge and difficulty of AdVQA.
Model Question Typeyes/no numbers others
Majority Class 65.87 35.55 8.83
ResNet-152 65.72 0.35 0.19
BERT 67.25 30.27 14.39
VisualBERT 56.77 37.06 21.23
ViLBERT 57.09 35.19 20.00
MoViE+MCAN∗ 50.42 34.53 19.26
M4C (VQA2) 64.66 38.01 21.62
Human performance. Another surprising finding is that inter-human agreement is higher on the AdVQA dataset than on VQA. This could be due to different data annotation procedures, requirements on annotators or just statistical noise. Human-adversarial questions may also be more specific due to annotators having to make crisp decisions about the model failing or not.
Model-in-the-loop’s performance. Interestingly, the MoViE+MCAN model that was not used in the loop and trained with a different seed, performs very similarly to other models. This suggests that to some extent, annotators overfit to the model instance. An alternative explanation is that model selection for all evaluated models was done on the AdVQA validation set, which was (obviously) not possible for the model in the loop used to construct the dataset. In Adversarial NLI [46], the entire model class of the in-the-loop model was affected. Note however that all VQA models perform poorly on AdVQA, suggesting that the examples are by and large representative of shortcomings of VQA techniques overall, and not of an individual model instance or class.
Train vs Test Distribution. We experiment with finetuning VisualBERT and VilBERT further on the AdVQA val set, finding that this improved test accuracy from 31.96 to 39.49 and 32.01 to 38.11, respectively, suggesting a difference in the VQA and AdVQA distributions, as we would expect.

heading: 4.3 Training Details

text:
We train all the models in Table 3 on the VQA train + val split excluding the COCO 2017 validation images. We also add questions from Visual Genome [34] corresponding to the images that overlap with the VQA training set to our training split. The VQA val set in our results contains all questions associated with the images in the COCO 2017 validation split. This is also consistent with the training split used for the model in the loop. We collect our AdVQA validation set on the images from the COCO 2017 val split, ensuring there is no overlap with the training set images. We choose the best checkpoint for each model by validating on the AdVQA validation set.
For all our experiments, we use the standard architecture for the models as provided by their authors. For the models that are initialized from pretrained models (whether unimodal or multimodal pretrained), we use off-the-shelf pretrained model weights and then finetune on our training set. We do not do any hyperparamter search for these models and use the best hyperparams as provided by respective authors. We finetune each model with three different seeds and report average accuracy.
We run most of our experiments on NVIDIA V100 GPUs. The maximum number of GPUs used for training is 8 for larger models. Maximum training time is 2 days. More details about hyperparameters, number of devices and time for training each model are provided in the supplementary material.

heading: 5 Dataset Analysis

text:
Questions. We first analyze the question diversity in AdVQA and compare them with popular VQA datasets. AdVQA contains 46,807 questions (10,000 in val and 36,807 in test) each with 10 answers.
Fig. 5a shows the distribution of question length compared with [20, 57]. The average question length in AdVQA is 8.1, which is higher than the VQA v2 (6.3), and TextVQA (7.2). The workers often need to get creative and more specific to fool the model-in-the-loop, leading to somewhat longer questions (e.g. specifying a particular person to ask about). Fig. 6a shows the top 15 most occurring questions from the val set, showcasing that questions involving text (e.g. time, sign) and counting (e.g. how many) are major failures for current state-of-the-art VQA models, corroborating the findings of prior work in [57, 21, 58, 28]. Fig. 4 shows a sunburst plot for the first 4 words in the AdVQA val set questions. We can observe that questions in AdVQA often start with “what” or “how” frequently inquiring about aspects like “many” (count) and “brand” (text).
Answers. In AdVQA val set, 75.4% (1,930) answers only occur once compared to 56.2% in VQA v2 [20], suggesting that the diversity of possible answers is much larger in AdVQA compared to
VQA v2. Fig. 5b shows the percentage of questions that are solvable with a vocabulary of top k most occurring answers. We observe that more questions in VQA v2 while fewer question in TextVQA are solvable with smaller vocabulary compare to AdVQA. This suggests that AdVQA is more diverse and difficult than VQA v2 but not as narrowly focused as TextVQA, making it a great testbed for future VQA models. We also showcase more qualitative examples in our supplementary to demonstrate that AdVQA’s diversity doesn’t lead to unnatural questions. Fig. 5c shows the cumulative percentage of questions where more than a particular number of annotators agree with each other. Fig. 6b shows the top 500 most occurring answers in the AdVQA val set; starting from very common answers such as “no” and counts (“1”, “5”), to gradually more specific and targeted answers like “32”, “peppers” and “horizontal”.
Table 5: The category-wise distribution of answers. Compared to VQA, AdVQA contains more “number”based and lesser “yes/no” questions supporting the prior work’s observations around failure of VQA models to count and read text.
Question Type VQA AdVQA VQA AdVQA test-dev test val
yes/no 38.36 23.22 37.70 24.58
number 12.31 35.73 11.48 32.44
others 49.33 41.05 50.82 42.98
Answer Vocabulary. To showcase the challenge of AdVQA, we take the original VQA v2 vocabulary used in the Pythia v0.3 model [55]. We find that 87.2% of the AdVQA val set’s questions are answerable using this vocabulary, suggesting that a model with powerful reasoning capability won’t be heavily limited by vocabulary on AdVQA. But, we also note that for high performance on AdVQA, a model will need to understand and reason about rare concepts, as 52.6% of the answers in AdVQA val and test sets don’t occur in VQA v2 train set.
Question Types. Table 5 shows the categorywise distribution for AdVQA questions compared with VQA v2 [20]. We can observe a shift from more easy questions of the “yes/no” category in VQA v2 dataset to more difficult questions in “numbers” category (as suggested in prior work [45]) in AdVQA. Please refer to Section B.1 for further detailed breakdown of question types.
Human Agreement. In the AdVQA val set, 27.8% human annotators agree on all 10 answers while 3 or more annotators agree an answer for 97.8%, which is higher compared to 93.4% on VQA v2 [20], even though as discussed AdVQA contains a large number of rare concepts.
Relationship to TextVQA [55]. To understand if the ability to read text is crucial for AdVQA, we extract Rosetta [7] tokens on the AdVQA val set and determine how many questions can be answered using OCR tokens at an edit distance of 1 to account for OCR errors. We find that 11.1% of the AdVQA val questions are solvable using OCR tokens suggesting that ability to read scene text would play a crucial role for AdVQA. Refer to Section B.4 for comparison with other important datasets.

heading: 6 Conclusion, Limitations & Outlook

text:
We introduced the AdVQA dataset, a novel human-adversarial multimodal dataset designed for accelerating progress on Visual Question Answering (VQA). Current VQA datasets have started plateauing and are approaching saturation with respect to human performance. In this work, we demonstrate that the problem is far from solved.
In particular, our analysis and model evaluation results suggest that current state-of-the-art models underperform on AdVQA due to a reasoning gap incurred from a combination of (i) inability to read text; (ii) inability to count; (iii) heavy bias towards the VQA v2 question and answer distribution; (iv) external knowledge; (v) rare unseen concepts; and (vi) weak multimodal understanding. We’ve shown the gap is unlikely to be due to (i) limited answer vocabulary; (ii) language representation (BERT performance compared to other); (iii) no pretraining (UniT); or (iv) lack of adversarial training (VILLA performance). The evaluation benchmark for AdVQA is available at https://adversarialvqa.org for the community and we hope that AdVQA will help bridge the gap by serving as a dynamic new benchmark for visual reasoning with a large amount of headroom for further progress in the field.2
In future work, it would be interesting to continue AdVQA as a dynamic benchmark. If a new state of the art emerges, those models can be put in the loop to examine how we can improve even further.
Broader Impact
This work analyzed state-of-the-art Visual Question Answering (VQA) models via a dataset constructed using a dynamic human-adversarial approach. We hope that this work can help make VQA models more robust.
VQA datasets contain biases, both in the distribution of images in these datasets, as well as the corresponding questions and answers which are likely amplified by the VQA models trained on these datasets. Biases studied in the context of image captioning [8, 62] are also relevant for VQA. English is the only language represented in this work and most annotators are based in the United States.
VQA models can be useful for aiding visually impaired users. The commonly used VQA datasets are not representative of the needs of visually impaired users – both in terms of the distribution of images (typically consumer photographs from the web), and in terms of the questions contained in the datasets (typically asked by sighted individuals while looking at the image). In contrast, the VizWiz dataset [21] contains questions asked by visually impaired users on images taken by them to accomplish day-to-day tasks. It is also worth mentioning to the researchers working on the VQA problem that VQA systems can also possibly be used in surveillance systems. It would be beneficial for the community to collect larger datasets in this context to enable progress towards useful and relevant technology while making efforts to reduce the harmful side-effects and applications. It would be important to do this with input from all relevant stakeholders, and in a responsible and privacy-preserving manner. VQA models are currently far from being accurate enough to be uesful or safe in these contexts.
Acknowledgments and Disclosure of Funding
We thank our collaborators in the Dynabench team for their support.

title: Human-Adversarial Visual Question Answering


summaries: The authors employ RNN models under different conditions to model MEC cells with both well-understood (e.g., place cells) and poorly-understood (heterogeneous cells) tuning properties. By training the network to minimize the place cell-mediated path integration, the network performed incredibly well in explaining MEC-cell dynamics. By removing neurons in the model, the network's path-integration performance was found to be at least as dependent on the model's heterogeneous neurons as on the stereotypical ones. The authors then finish by introducing a reward-based foraging model, again finding responses that match experimental results very well.

summaries: The authors address two important questions regarding the non-grid cells in MEC. First, is their response consistent, and not just noise? Second, can they be used as criteria for testing models of MEC functionality? The second question allows to probe the functional significance of these cells, and the effect of incorporating them on various model properties. The authors’ main tool is a linear mapping from a population of cells into a single target cell. This is done using rate maps of cells, and cross validation on the spatial bins. Several existing models are compared, with one model singled out as corresponding best to experimental data. The functional relevance of grid and non-grid cells is evaluated using ablation studies on the model, where path integration ability is used as a measure. Generalization of the models is assessed using either different arenas or a reward-biased setting.

summaries: The authors used linear regression to first demonstrate the robustness of non-grid-like responsive cells in experimental recordings of MEC cells. They then used the same regression to assess the similarity of responses derived from the goal driven training of RNNs to MEC responses. They found that responses derived from RNNS contained these non-grid-like responses and explained neural activity better than a low-rank decomposition of place fields.  These non-grid-like responses were crucial for path integration and likely explained the difference in similarity matching between goal-driven training and low rank decomposition.

summaries: By adopting 'similarity transform' methods, this study compares the predictability of different computational models of MEC neurons. The best model not only can explain the firing patterns of grid cells but also that of heterogeneous cells. These findings further allow them to examine the role of heterogeneous cells which has not been clearly identified with experimental approaches. This study suggests that the heterogeneous cells might play critical roles in path integration even more than what grid cells contribute. This manuscript is well-written, and the results provide a theoretical foundation for future studies using a more experimental approach. I only have minor clarification questions and suggestions to embrace a broader readership such as neuroscientists.

abstractText: Medial entorhinal cortex (MEC) supports a wide range of navigational and memory related behaviors. Well-known experimental results have revealed specialized cell types in MEC — e.g. grid, border, and head-direction cells — whose highly stereotypical response profiles are suggestive of the role they might play in supporting MEC functionality. However, the majority of MEC neurons do not exhibit stereotypical firing patterns. How should the response profiles of these more “heterogeneous” cells be described, and how do they contribute to behavior? In this work, we took a computational approach to addressing these questions. We first performed a statistical analysis that shows that heterogeneous MEC cells are just as reliable in their response patterns as the more stereotypical cell types, suggesting that they have a coherent functional role. Next, we evaluated a spectrum of candidate models in terms of their ability to describe the response profiles of both stereotypical and heterogeneous MEC cells. We found that recently developed task-optimized neural network models are substantially better than traditional grid cell-centric models at matching most MEC neuronal response profiles — including those of grid cells themselves — despite not being explicitly trained for this purpose. Specific choices of network architecture (such as gated nonlinearities and an explicit intermediate place cell representation) have an important effect on the ability of the model to generalize to novel scenarios, with the best of these models closely approaching the noise ceiling of the data itself. We then performed in silico experiments on this model to address questions involving the relative functional relevance of various cell types, finding that heterogeneous cells are likely to be just as involved in downstream functional outcomes (such as path integration) as grid and border cells. Finally, inspired by recent data showing that, going beyond their spatial response selectivity, MEC cells are also responsive to non-spatial rewards, we introduce a new MEC model that performs reward-modulated path integration. We find that this unified model matches neural recordings across all variable-reward conditions. Taken together, our results point toward a conceptually principled goal-driven modeling approach for moving future experimental and computational efforts beyond overly-simplistic single-cell stereotypes. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

authors:
M. Abadi
P. Barham
J. Chen
Z. Chen
A. Davis
J. Dean
M. Devin
S. Ghemawat
G. Irving
M. Isard

title: Tensorflow: A system for large-scale machine learning

venue: In 12th {USENIX} symposium on operating systems design and implementation ({OSDI}

authors:
D. Aronov
R. Nevers
D.W. Tank

title: Mapping of a non-spatial dimension by the hippocampal–entorhinal

venue: circuit. Nature,

authors:
A. Banino
C. Barry
B. Uria
C. Blundell
T. Lillicrap
P. Mirowski
A. Pritzel
M.J. Chadwick
T. Degris
J. Modayil

title: Vector-based navigation using grid-like representations in artificial agents

authors:
C.N. Boccara
M. Nardin
F. Stella
J. O’Neill
J. Csicsvari

title: The entorhinal cognitive map is attracted to goals

authors:
Y. Burak
I.R. Fiete

title: Accurate path integration in continuous attractor network models of grid cells

venue: PLoS Comput Biol,

authors:
W.N. Butler
K. Hardcastle
L.M. Giocomo

title: Remembered reward locations restructure entorhinal spatial

venue: maps. Science,

authors:
M.G. Campbell
S.A. Ocko
C.S. Mallory
I.I. Low
S. Ganguli
L.M. Giocomo

title: Principles governing the integration of landmark and self-motion cues in entorhinal cortical codes for navigation

venue: Nature neuroscience,

authors:
R. Cao
D. Yamins

title: Explanatory models in neuroscience: Part 1–taking mechanistic abstraction seriously

venue: arXiv preprint arXiv:2104.01490,

title: Explanatory models in neuroscience: Part 2–constraint-based intelligibility

venue: arXiv preprint arXiv:2104.01489,

authors:
K. Cho
B. van Merriënboer
D. Bahdanau
Y. Bengio

title: On the properties of neural machine translation: Encoder–decoder approaches

venue: In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation,

authors:
J. Collins
J. Sohl-Dickstein
D. Sussillo

title: Capacity and trainability in recurrent neural networks

venue: In ICLR,

authors:
A.O. Constantinescu
J.X. O’Reilly
T.E. Behrens

title: Organizing conceptual knowledge in humans with a gridlike

venue: code. Science,

authors:
J.J. Couey
A. Witoelar
S.-J. Zhang
K. Zheng
J. Ye
B. Dunn
R. Czajkowski
M.-B. Moser
E.I. Moser
Y. Roudi

title: Recurrent inhibitory circuitry as a mechanism for grid formation

authors:
C.J. Cueva
X.-X. Wei

title: Emergence of grid-like representations by training recurrent neural networks to perform spatial localization

authors:
Y. Dordek
D. Soudry
R. Meir
D. Derdikman

title: Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis

title: Finding structure in time

venue: Cognitive Science,

authors:
U.M. Erdem
M. Hasselmo

title: A goal-directed spatial navigation model using forward trajectory planning based on grid cells

venue: European Journal of Neuroscience,

authors:
S. Farhoodi
M. Plitt
L. Giocomo
U. Eden

title: Estimating fluctuations in neural representations of uncertain environments

venue: Advances in Neural Information Processing Systems,

authors:
M.C. Fuhs
D.S. Touretzky

title: A spin glass model of path integration in rat medial entorhinal cortex

venue: Journal of Neuroscience,

authors:
T. Hafting
M. Fyhn
S. Molden
M.-B. Moser
E.I. Moser

title: Microstructure of a spatial map in the entorhinal cortex

authors:
K. Hardcastle
N. Maheswaranathan
S. Ganguli
L.M. Giocomo

title: A multiplexed, heterogeneous, and adaptive code for navigation in medial entorhinal

venue: cortex. Neuron,

authors:
J.R. Hinman
M.P. Brandon
J.R. Climer
G.W. Chapman
M.E. Hasselmo

title: Multiple running speed signals in medial entorhinal

authors:
S. Hochreiter
J. Schmidhuber

title: Long short-term memory

venue: Neural Computation,

authors:
A.J. Kell
D.L. Yamins
E.N. Shook
S.V. Norman-Haignere
J.H. McDermott

title: A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing

venue: hierarchy. Neuron,

authors:
D.P. Kingma
J. Ba

title: Adam: A method for stochastic optimization

authors:
E. Kropff
J.E. Carmichael
M.-B. Moser
E.I. Moser

title: Speed cells in the medial entorhinal cortex

authors:
R.F. Langston
J.A. Ainge
J.J. Couey
C.B. Canto
T.L. Bjerknes
M.P. Witter
E.I. Moser
M.-B. Moser

title: Development of the spatial representation system in the rat

authors:
I.I. Low
A.H. Williams
M.G. Campbell
S.W. Linderman
L.M. Giocomo

title: Dynamic and reversible remapping of network representations in an unchanging environment

venue: bioRxiv,

authors:
C.S. Mallory
K. Hardcastle
M.G. Campbell
A. Attinger
I.I. Low
J.L. Raymond
L.M. Giocomo

title: Mouse entorhinal cortex encodes a diverse repertoire of self-motion signals

venue: Nature communications,

authors:
J.A. Michaels
S. Schaffelhofer
A. Agudelo-Toro
H. Scherberger

title: A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping

venue: Proceedings of the National Academy of Sciences,

authors:
S.A. Ocko
K. Hardcastle
L.M. Giocomo
S. Ganguli

title: Emergent elasticity in the neural code for space

authors:
J. O’Keefe
J. Dostrovsky

title: The hippocampus as a spatial map: Preliminary evidence from unit activity in the freely-moving rat

venue: Brain research,

authors:
F. Sargolini
M. Fyhn
T. Hafting
B.L. McNaughton
M.P. Witter
M.-B. Moser
E.I. Moser

title: Conjunctive representation of position, direction, and velocity in entorhinal cortex

authors:
W.E. Skaggs
B.L. McNaughton
K.M. Gothard
E.J. Markus

title: An information-theoretic approach to deciphering the hippocampal code

venue: In Proceedings of the 5th International Conference on Neural Information Processing Systems,

authors:
T. Solstad
C.N. Boccara
E. Kropff
M.-B. Moser
E.I. Moser

title: Representation of geometric borders in the entorhinal cortex

authors:
B. Sorscher
G. Mel
S. Ganguli
S.A. Ocko

title: A unified theory for the origin of grid cells through the lens of pattern formation

authors:
B. Sorscher
G.C. Mel
S.A. Ocko
L. Giocomo
S. Ganguli

title: A unified theory for the computational and mechanistic origins of grid cells

authors:
K.L. Stachenfeld
M. Botvinick
S.J. Gershman

title: Design principles of the hippocampal cognitive map

venue: Advances in neural information processing systems,

authors:
D.L. Yamins
J.J. DiCarlo

title: Using goal-driven deep learning models to understand sensory cortex

authors:
K. Yoon
M.A. Buice
C. Barry
R. Hayman
N. Burgess
I.R. Fiete

title: Specific evidence of low-dimensional continuous attractor dynamics in grid cells

title: Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory

heading: 1 Introduction

text:
From exploring new areas, planning shortcuts, and returning to remembered locations, the ability to self-localize within an environment subserves a range of navigational behaviors that are essential for survival. The hippocampus (HPC) and medial entorhinal cortex (MEC) are known to contain cells that encode the position of an animal by displaying firing fields with strikingly regular response patterns, influenced by self-motion [O’Keefe and Dostrovsky, 1971, Hafting et al., 2005, Kropff et al., 2015, Solstad et al., 2008, Sargolini et al., 2006]. For example, MEC grid cells possess characteristically symmetric and periodic tuning curves, yielding hexagonal arrays of neural activity over physical space. Additionally, border cells that fire maximally near environmental boundaries and head direction cells that fire only when an animal faces a particular direction are among the other MEC cell types with interpretable tuning curves.
However, a large fraction of the MEC population have unconventional and heterogeneous tuning to navigational variables and are less obviously well-described in terms of simple, stereotypical tuning patterns [Hinman et al., 2016, Hardcastle et al., 2017]. How can we characterize what these other, more heterogeneous, populations of cells do? Are they critical to the abilities of the animal in real-world memory tasks? And if so, how is their role distinct from the more stereotypical grid-like cells? Here, we take a quantitative computational modeling approach to answering these questions.
To start with, we address the question of whether there is a phenomenon to model in the first place, and how one would measure model accuracy quantitatively. To this end, we identify the similarity transform between neural populations in different animals — that is, a mapping which takes neuronal population vectors in a “source” animal and maps it to corresponding neuronal population vectors in a “target” animal. To the extent that robust similarity transforms can be found that match up MEC population responses across multiple animals measured in multiple experimental conditions, then a reliable pattern of neural response behaviors (across conditions) has been identified. This identification strategy is well-defined even when there is no known a priori taxonomy of functional response types. We find here that with the right mapping class (mostly Ridge-like linear regression), the inter-animal consistency of MEC neuronal responses is in absolute terms very high (> 0.8), and in relative terms just as high as for heterogeneous cells as for more stereotypical grid cells.
Using this same similarity transform, we then evaluate the ability of each of multiple computational models to explain response variance of MEC neurons, treating each candidate model as a potential “source animal” and measuring how well it maps to each target real animal. We look to the recent literature to identify potential candidate models. Over the past several decades, collaborations between experimental and computational neuroscientists have led to the formulation of dynamicalsystems models of grid cell formation, and helped illustrate possible functional roles for grid cells in supporting path integration-based hippocampal place cells [Skaggs et al., 1992, Zhang, 1996, Fuhs and Touretzky, 2006, Burak and Fiete, 2009]. These powerful models make a number of nonobvious predictions about MEC neural properties, some of which have been confirmed in subsequent experimental work [Ocko et al., 2018, Campbell et al., 2018]. Despite their success, such models are limited in their explanatory scope, hand-designed to capture the properties of one stereotypical cell-type class (e.g. grid or border cells) at a time, or combinations of several cell types via multiple dedicated type-specific modules [Couey et al., 2013, Yoon et al., 2013].
A potential solution to this problem arises out of recent work creating learned neural networks that achieve path integration [Cueva and Wei, 2018, Banino et al., 2018]. Intriguingly, these models have been found to contain internal units that resemble grid cells, suggesting that such stereotypical cells embody a computational solution to path integration that naturally arises from satisfying an end-toend functional constraint. Recent work has demonstrated that the underlying mathematical reason for this fact is due to pattern forming dynamics under a nonnegativity constraint [Sorscher et al., 2019, 2020]. Intriguingly, in addition to having units that resemble stereotypical grid or border cells, these learned neural networks also naturally possess a wide variety of other less easily described unit types, raising the possibility that these artificial “heterogeneous” units might be somehow resemble the actual heterogeneous cells making up the majority of real MEC populations.
Motivated by these ideas, we generate a wide variety of candidate model networks by varying architectural structure and end-to-end optimization objectives, each expressing a different hypothesis for MEC circuit structure and function. Architecturally, we formulate variants based on using different types of nonlinearities and different types of local recurrent memory circuits (e.g. RNNs [Elman,
1990], UGRNNs [Collins et al., 2017], GRUs [Cho et al., 2014], and LSTMs [Hochreiter and Schmidhuber, 1997]). From a task point of view, we test both simple dimensionality reduction [Stachenfeld et al., 2014, Dordek et al., 2016] as well as place cell mediated vector path-integration [Banino et al., 2018] and direct position estimation [Cueva and Wei, 2018].
Our core result is that there is substantial variation in the models’ abilities to explain MEC responses, especially those of the heterogeneous cells, as a function of a model architecture and task. Some models, such as the classic “Grid Cell” model based on low-rank decomposition of place cell fields, do a reasonable job explaining grid cell responses but are quite poor at explaining most other neurons. Task-optimized learned models typically do better, especially those optimized for place cell-mediated vector path-integration. The best model – with memory-gated rectified nonlinearities – essentially solves the neurons, capturing nearly 100% of the noise ceiling of the data. This is of substantial interest, given that the nonlinear components of this model are not directly optimized to match neural data (just to solve the task), and given that a series of strong control models capture much less of the MEC neural variability. This same model also best generalizes to a variety of novel experimental conditions, and has the best match to the empirical data on a grid score distribution metric.
With this predictive model in hand, we then begin to address our second core question: what is the functional role of heterogeneous neurons? We generate several results suggesting that heterogeneous neurons are important for path integration, including cell-type specific virtual knockout experiments, in which we compare performance degradation when deleting grid and/or border cells as compared to heterogeneous units. Overall, we find that models are quite robust to knockouts, and differences between heterogeneous and stereotypical cell knockouts are very small, suggesting that stereotypical cell-types may not be especially more functionally important than units with less easily-characterized response profiles.
Building on the above results, we extend models to encompass MEC cell responses as a function of reward as well as spatial position, introducing a simple modeling paradigm that performs rewardmodulated foraging in the context of the path-integration task. We find that this unified task-optimized model matches neural responses across all reward and spatial conditions, and that a reward-response mechanism at an intermediate point on the explore-exploit continuum best explains neural responses. Taken together, our results suggest how specific processes of biological performance optimization may have directly shaped the neural mechanisms in MEC as a whole, and provides a path for enlarging the study of MEC beyond overly-restrictive response stereotypes.

heading: 2 Reliability of heterogeneous cell response profiles

text:
What firing patterns of MEC cell populations are common across multiple animals, and thus worthy of computational explanation? This question is comparatively straightforward for stereotypical MEC cells, because the very presence of these stereotypical features (e.g. hexagonal grids of a given orientation and spatial frequency) allows the definition and measurement of observables that arise reliably across trials and animals (e.g. the grid score distribution). But how can this be done generally for populations in which one does not have a prior characterization of what each cell encodes? We take inspiration from methods that have proven useful in modeling visual, auditory, and motor cortex [Yamins and DiCarlo, 2016, Kell et al., 2018, Michaels et al., 2020]. Specifically, we aim to identify the narrowest class of similarity transforms needed to map the firing patterns of one animal’s MEC population to that of another (Fig. 1a). As with other cortical areas, this transform class likely cannot be so strict as to require fixed neuron-to-neuron mappings between MEC cells, since even within the same animal at different times, MEC and HPC populations can undergo remapping that shift cell responses across the population [Farhoodi et al., 2020, Low et al., 2020]. However, the transform class for MEC also cannot be so loose as to allow a completely unconstrained linear mapping, since the highly structured response patterns of stereotypical MEC cell types such as grid cells may not be guaranteed to be preserved under arbitrary linear transforms.
To identify this transform class, we utilize data collected from electrophysiology in 12 awake behaving mice (n = 620 cells) performing open foraging for randomly scattered crushed cereal in a 100cm2 2D arena [Mallory et al., 2021]. We explore a variety of mapping transform classes between the population rate maps of these neurons (Fig. 1b). As a baseline we evaluate a strict one-to-one mapping transform, in which each target unit is mapped to the single most correlated unit in the source animal. We also evaluate more powerful linear transforms, including Lasso, Ridge, and ElasticNet regression,
as well as population-level Partial Least Squares (PLS) regression. For all methods with fittable parameters, mapping fit is performed on a random 20% of spatial position bins, and evaluated on the remaining 80% of position bins (see supplement for more details).
The strict one-to-one mapping yielded low inter-animal consistency among the maps considered, capturing around 50% of the target neural response variability. Linear regression with strong sparseness priors, such as Lasso (L1 penalty) and balanced Lasso-Ridge (equal L1 and L2 penalty) regression, also proved to be too strict when evaluated with fixed regularization level α = 1, yielding hardly any improvement over the one-to-one mapping. However, pure Ridge (L2 penalty) regression at this regularization level was highly effective, recovering nearly all target variability for most neurons. Cross-validating the L1 regularization constant on a per-cell basis improved the fits, at the cost of requiring substantially looser regularization than for L2 (Fig. 1c). Under an ElasticNet mapping in which both sparsity penalty and regularization constants were chosen with cross validation, most cells were generally still best explained by an essentially Ridge-like transform with no sparsity penalty (Fig. 1d). However, different target cells required different numbers of source units to achieve effective mapping (Fig. 1e). As expected, cells with more stereotypically grid-like response patterns on average chose a higher sparsity penalty (slight positive slope in Fig. 1d) and required fewer source cells to capture (slight negative slope in Fig. 1e) as compared to heterogeneous cells, though this effect is weak. Critically, heterogeneous cells did not have lower inter-animal consistency than grid cells (Fig. 1f).
These results show that the heterogeneous non-stereotyped cell populations are reasonably similar across animals – at least up to (mostly Ridge-regularized) linear transform – establishing that there is a reliable target pattern to study in the first place.

heading: 3 Task-optimized models of MEC spatial response variation

text:
Evaluating a spectrum of candidate models. We evaluated models of several basic types. First, we considered learnable neural networks, all of which accept a stream of two-dimensional velocity input vectors, have a single layer of hidden neurons identified as the putative MEC population, and
which are optimized to perform some form of path integration readout on simulated motion paths (Erdem and Hasselmo [2012]) in a fixed-sized arena. The networks varied according to the nature of their local recurrent cell structure (ungated RNN, UGRNN, GRU, or LSTM), activation functions (Linear, Tanh, Sigmoid, or ReLU), and output objective function (explicitly constructing a set of simulated place field neurons [Banino et al., 2018] or directly performing two-dimensional position integration [Cueva and Wei, 2018]). Following Banino et al. [2018], all networks consisted of three nonlinear layers, though for the ungated RNN we included its original two layer version (following Cueva and Wei [2018]) and a three layer version (“SRNN”). We ensured that comparisons were fair by equalizing the size of the hidden layer across models, and included an architecture-only control with untrained filter weights.
We also implemented a class of models describing MEC activations as Non-negative Matrix Factorization (NMF) operating on simulated place cells. The lowest-rank version acts a “Grid Cell” model, inspired by recent work positing MEC as a low-dimensional embedding of hippocampal place fields [Stachenfeld et al., 2014, Dordek et al., 2016], and further validated by the mathematical theory developed in Sorscher et al. [2019, 2020] to explain the emergence of grid cells. Higher-rank NMF models titrate between the low-rank Grid Cell model and a full-rank control that measures how well MEC cells can be explained as a linear projection of their putative place cell outputs.
For evaluation, we map each model’s proposed MEC activations to empirically measured firing patterns of the real neurons, using the same type of cross-validated ElasticNet regression transform used to measure inter-animal similarity as in Section 2. The predicted neural responses under the mapping are then compared to real target neural responses on a neuron-by-neuron basis, and the median of accuracy of these predictions taken over target neurons. In addition to evaluating modeldata match when constructed on arenas of the same size in which the neural data were collected (100cm2), we also evaluated models trained on larger arenas (2.2m2) but tested on the 100cm2 arena.
Neural prediction results. The results of our model evaluation, shown in Fig. 2a, support several inferences:
• Different models are substantially different in their ability to predict neural responses. MEC electrophysiology data collected during 2D open-field foraging is thus a strong model target that effectively separates candidate models from each other.
• Task-optimized models are reliably better than their untrained controls, across all architecture types and objective functions.
• Rectification is substantially better than Linear, Tanh, or Sigmoid activation, especially for promoting generalization to new arena sizes.
• Under the rectification nonlinearity, gated circuit architectures (UGRNN, GRU, LSTM) improve model fits compared to the simple ungated alternatives (RNN and SRNN).
• The explicit place cell construction task leads to substantially better fits than the direct twodimensional path integration task (“Position Loss”), even though both were reported in the literature to create grid cell-like units. Enforcing a place field representation at the output of the network thus appears to be an important constraint in order to recapitulate responses in MEC.
• The Grid Cell (low-rank NMF) model is a poor fit to the MEC population overall. • Full-rank NMF and “Place Cell Linear MEC” output-based controls capture approximately half
the explainable variance of MEC neurons, significantly more than the simple velocity linear input control, but substantially less than any of the input-driven ReLU networks.
• The best model is the UGRNN ReLU trained with the place cell loss (“UGRNN-ReLU-Place Cell”). This model captures nearly 100% of the explainable variance of the MEC neural population response when trained for place cell construction on the arenas of the same size as that on which neural data was collected.
Generalization to novel experimental conditions. If a model is truly correct, then once trained, it should capture neural responses in any new tested condition. We tested generalization in two ways. First, as shown in the second from the right bars of Fig. 2a for each model architecture class, we performed a 2D arena size generalization test by constructing models on one arena size, testing against neural data on another. We found this generalization test gives essentially the same rank-order comparison as in the same-arena-size test, with one striking exception: the RNN Tanh model performs well within arena size, but fails to generalize. The UGRNN-ReLU-Place Cell model again performs the best, capturing 90% explained variance of the neural responses on the novel arena size.
Second, we also evaluated the same 2D-pretrained models by running them on a 1D track, comparing models to neural data collected from mice in a 1D virtual reality setup, using the same ElasticNet mapping procedure as in the 2D comparisons (see supplement for more experimental details). We found (Fig. 2b) that 2D arena trained model results had some similar rank order as for the original 2D results (0.43 Spearman rank correlation), with the UGRNN-ReLU-Place Cell model trained in the 2D arena (and then evaluated on the 1D track) achieving the best match (82%) when trained for place-field construction. We hypothesized that the larger gap between the best model and the inter-animal noise ceiling, as compared to the 2D case, was due to the fact that during the 1D experimental data collection, mice were also presented with visual cues. MEC neurons are known to respond to visual input, but as none of our evaluated models had a visual front-end, they could not respond accordingly. To test this hypothesis we trained the UGRNN-ReLU-Place Cell model in 2D, but with phantom visual cue locations added as input while performing the path integration task. We then evaluated this model against 1D neural prediction (with the real visual cue locations as input), and found that including cues rescued model performance back to essentially the noise ceiling, as in 2D (Fig. 2b, orange bars).
Assessing grid and border score distribution match. The grid score is a metric of how stereotypically grid-like the response pattern of a given unit is (see supplementary material for specific definitions), with “grid cell” typically defined as having a grid score of greater than 0.3. Similarly, the border score is a measure of how responsive a given unit is in the presence of environmental boundaries, with a “border cell” typically defined as having a border score greater than 0.5 [Solstad et al., 2008]. The ground-truth distribution of grid and border scores for cells in an unbiased experimental population (gray bars in each subpanel of Fig. 3a,c) characterize the extent to which real MEC populations are non-stereotypical in their responses. To further assess model accuracy, we also compared the distribution of grid and border scores within each model to that of the real data, using the (negative) Kolmogorov-Smirnov (KS) distance of the empirical and model grid score distributions as a quantitative metric. This metric is both stronger and weaker than the mapping accuracy metric used above — stronger in the sense that, since there is no parameter fitting in the metric, to fit it well a model has to have the correct distribution in its raw feature output; and weaker in that it only assesses cells on one component of their profile (i.e. “grid-ness” or “border-ness”). We observed that the low-rank Grid Cell model has poor fit on this metric, essentially because it contains too many grid cells (see Fig. 3a, upper left). In contrast, the same model that achieves best fits on the neural-fit
metric (UGRNN-ReLU-Place Cell) also achieves the best on the grid cell distribution match metric, and is the only architecture that cannot be distinguished from the ground truth at the KS < 0.05 significance level. Across model architectures, KS-distance was generally in line with neural regression fit metrics (see Fig. 3b,d and Fig. S2). One key difference between the metrics, however, is that for most architectures, the untrained filters provided better grid score distribution matches as models with task-trained parameters — except for the that best matched architecture, UGRNN ReLU, where the trained place cell construction model is better than its untrained counterpart (and similar to it for the border score) — and models trained for direct path integration were especially poor on this metric. These results suggest the two metrics in Figs. 2 and 3 are complementary which aspects of model correctness they address, and together help zero in on the most effective models overall.

heading: 4 Predicting the functional relevance of heterogeneous cells

text:
Correlating task performance and neural predictivity. To begin to address the question of the functional relevance of heterogeneous cells, we first looked at the overall correlation between model task performance and neural fit (Fig. 4a). Though the correlation is imperfect, the most task performant models (e.g. UGRNN- and LSTM-ReLU-Place Cell) achieve the best matches to neural fit suggesting that improved task performance may be causally related to the ability capture neuron response patterns across the population. In contrast, there is a comparatively weaker relationship between grid and border score distribution match and model performance (Fig. 4b). While the most task performant models achieve the best match here, models with a gating architecture and ReLU are strong matches to these metrics with untrained filters, illustrating the importance in matching cell properties other than “grid-ness” or “border-ness” in predicting task performance.
Relative predictivity gain for heterogeneous cells. The above result is put into greater perspective by comparing neural predictivity differential between a task-trained UGRNN-ReLU-Place Cell model and the Grid Cell model, on a per-neuron basis, as a function of grid score (Fig. 4c). The task-trained model has improved neural predictivity relative to the Grid Cell model for grid cells (grid score > 0.3) and heterogeneous cells, but the improvements on the latter (as well as border cells, Fig. S3) are larger than for grid cells. This again suggests that the heterogeneous cells are playing a substantial role in allowing the trained model to achieve improved performance.
Virtual knockout experiment. To address the question most directly, we performed a cell-typetargeted virtual knockout comparison experiment. In doing this, we used the UGRNN-ReLU-Place Cell and LSTM-ReLU-Place Cell models, the two best models emerging from the previous section with essentially similar neural predictivity across multiple metrics. We identified the units in the trained model with high grid score (> 0.3) or border score (> 0.5), and gradually increased the threshold, while measuring task generalization performance (see supplement for details of this
knockout procedure). We similarly ablated matched numbers of heterogeneous units (grid score ≤ 0.3 or border score ≤ 0.5 or both, see Fig. S4 for the latter), again measuring model performance. The main result of this experiment (Fig. 4d) is that all networks are highly robust to knockouts, experiencing only at most 1-4% performance degradation relative to the full model even when substantial fractions of units are knocked out (corresponding to 21-24% of a single layer’s units for grid score > 0.3 and 16-20% of these units for border score > 0.5). At stricter grid and border cell thresholds, the heterogeneous knockout is similarly injurious to the cell-type specific knockouts, across the two model architectures, suggesting that highly-stereotypical “classical” border and grid score cells are not more essential to the path integration behavior than heterogeneous cells. At low thresholds (when counting many relatively heterogeneous cells as “grid” or “border” cells), the two model architectures give divergent predictions, with the UGRNN model showing a small but significant effect of grid cells relative heterogeneous cells, and the LSTM model showing the opposite. It would be of substantial future interest to confirm or reject either of these models’ predictions with a real targeted knockout experiment in vivo.

heading: 5 Modeling reward-driven modulation in MEC

text:
Recent work in both rodents and humans has uncovered that MEC and HPC neurons represent not only literal space, but also capture spatialized layouts in a more abstract sense in modalities other than spatial position [Constantinescu et al., 2016, Aronov et al., 2017]. It has also been seen [Butler et al., 2019, Boccara et al., 2019] that non-spatial rewards can influence the shape of MEC response profiles, restructuring them to incorporate the location of the learned reward. These intriguing phenomena represent a natural direction for modeling, but are not captured by any of the neural network models described in Section 3, as they do not have reward-modulated inputs. (Note that while Banino et al. [2018] used a pretrained LSTM-Tanh-Place Cell network as a front-end on which reinforcement-based navigation tasks are evaluated downstream, reward state is not input to their network or otherwise propagated back into the MEC-like layers of their model, and thus cannot address the neural modeling question raised here.)
We thus sought to build new network models that respond to the existence of an extrinsic reward in a behaviorally-meaningful fashion. One natural approach would be to build a full reinforcementlearning (RL) agent in which reward depends on environmental features (e.g.total number of food units encountered during a forage path). A model optimized end-to-end to forage under these conditions could be successful in creating a representation that matches MEC reward-response modulation. However, we took a more direct approach: simply change the behavior of the agent in response to reward in the way RL would be expected to do if successful, by modifying the foraging trajectories used during training and testing to no longer be pure random walks.
Specifically, we trained the UGRNN-ReLU-Place Cell model with a variety of foraging policies titrating between pure exploration and exploitation. Paths could either purely exploit a known reward resource by directly moving to the reward location when it is present (Fig. 5a, ε = 0 condition), purely explore with a random walk as in the original unmodulated model (ε = 1), or take an intermediate policy (0 < ε < 1). In creating these scenarios, we sought to roughly mimic experimental observations showing animals often take rapid and direct paths to the reward zone [Butler et al., 2019] (see supplement for details of implementation). Throughout, the actual training task of the network remained the same as above — place cell construction — but the network now would have to be tolerant to reward-modulated input velocity changes while maintaining positional knowledge.
We then compared these networks to neural data from [Butler et al., 2019] for animals in conditions both with reward (reward+) and without it (reward-), in which the animal navigates to a 20cm2 reward zone within a 150cm2 arena to receive 0.5-1 units of cereal. We first performed inter-animal consistency checks, using the same method as we did with the purely spatial-condition data in Section 2 above, finding that inter-animal consistency for neural responses is high both within reward condition (e.g. just spatial modulation in reward+ and reward- separately), and across both spatial and reward-modulated conditions overall (Fig. S6).
As a baseline, we then evaluated the ability of the original (non-reward-modulated) UGRNN-ReLUPlace Cell model to match neural data. Consistent with results reported in the previous sections, this model achieved high neural predictivity in reward+ and reward- separately (Fig. 5b, hatched bars). However, as expected, this non-modulated model was ineffective at predicting response patterns across both reward conditions (Fig. 5b, reward+/-). Moreover, simply augmenting the network input to receive an additional binary reward state (Fig. 5b,“Reward Input”), but without using that input to specifically modulate input velocities or output behavior, did not substantially improve neural predictivity, showing the need for nontrivial integration of reward-modulated state.
We then evaluated the reward-modulated networks, generating network outputs for comparison with the reward+ and reward- conditions by modulating velocity inputs to match each condition during testing (see supplement for details). We found that the pure-explore (ε = 1) model is substantially
better at matching neural responses across reward conditions than the reward+/- baseline. This is perhaps somewhat surprising since in this comparison the underlying neural network model is identical, just evaluated with or without reward-modulated input data during testing. This result suggests that a substantial fraction of the reported reward-modulated effect in MEC may actually simply be input-driven, lending a new interpretation to results of [Butler et al., 2019]. (It may be useful to note that this result represents a model-based control that was inaccessible to the authors of [Butler et al., 2019].) However, we did find that exposing the network to a mixture of exploration and exploitation behaviors during training does lead to networks with somewhat improved neural predictivity. Results were largely robust to the specific proportion of exploration-vs-exploitation (Fig. 5c), though the best model (at ε = 0.375) model was statistically-significantly better than alternatives and had a substantially closer match (KS = 0.13 vs. KS = 0.81) to the data’s unit-level remapping across conditions than the original ε = 1 model (Fig. 5d). These results are consistent with there being some nontrivial within-MEC reward-modulated responses beyond simple input modulation alone.

heading: 6 Discussion

text:
We have identified a goal-driven neural network model of MEC that is quantitatively accurate across a wide variety of common experimental conditions, and that can be used to generate nontrivial insights about the underlying mechanisms and functional roles of mouse MEC. More generally, our results suggest that constraint-driven neural networks may provide a fruitful approach for studying navigation and memory in the MEC, and beyond.
Our work suggests the existence not of a specialized class of heterogeneous cells that is functionally segregated from classic cell types, but rather a continuum of cells within a single unified network that naturally encompasses grid, border, and heterogeneous cells. Future research on the computational foundations of MEC/HPC may thus be well-served by putting less emphasis on identifying cleanlystereotypical cell types (such as grid cells) or perfecting mathematically simple models of single such cell-types, and looking instead for holistic computational ideas that move beyond “easy to visualize” but perhaps overly-simplistic tuning-curve categories.
The nature of the understanding afforded by such a modeling approach comes from their ability to make inferences about what constraints (both structural and functional) are consistent with the data. The current work rules out a variety of simple network connectivity diagrams as inconsistent with MEC cell data, and narrows the space of functional goals MEC circuit weights might be optimized for over evolutionary timescales. Improving the results here will hopefully narrow these constraints yet further, with the ultimate goal of identifying constraints that yield the uniquely correct MEC circuit diagram and synaptic weights – or at least, the narrowest set of such networks consistent with the inherent variability between animals in the real population.
There are some key limitations, however, on our results. First, we attempted to identify the simplest underlying transform that would map animals within a population to each other, using this as the basis for conclusions both about unit-type reliability and model-data comparisons. While the philosophy of this approach may be sound [Cao and Yamins, 2021a], in practice it is possible that we could narrow the transform class further by (e.g.) enforcing that it be invariant to one more more properties of classical cell-types (e.g. periodicity). Finding algorithms to better identify sharp inter-animal transform classes will be an important topic for future work.
Moreover, we remain unconvinced that MEC just “is” a UGRNN-ReLU-Place Cell network, despite a network of this architecture having explained essentially all the data we had available to challenge it. It is possible that matching all our existing data is too easy a test. Would this network generalize to more complex situations, with increased variability along key axes such as spatial structure (e.g. environments with corridors and looping interconnections), nontrivial but spatially informative cues, and rewards exhibiting complex and temporally-variable patterns? We do not think it is at all obvious that it would. The proper conclusion from this work is thus not that our current best model is actually correct, but rather that the fairly simplified setup typical of experiments in MEC is not sufficiently complex to falsify it. This present situation is a concrete manifestation of the “contravariance principle” of neural modeling [Cao and Yamins, 2021b] – the idea that working in a more complex experimental environment might actually make it easier to identify an actually correct model, by virtue of reducing susceptibility to spurious models that are apparently consistent with too-simple data. Future experiments should engage mice in more complex environments and behaviors.

heading: 7 Acknowledgements

text: We are grateful to Alex Gonzalez, David Sussillo, and John H. Wen for helpful discussions. We thank the anonymous reviewers for their feedback on a draft of this manuscript. A.A. received support from the Swiss National Science Foundation (P2BSP3_181743 and P400PB_191076). I.L. is supported by funding from the Wu Tsai Neurosciences Institute under Stanford Interdisciplinary Graduate Fellowships and a Bertarelli fellowship. A.H.W. received funding support from the National Institutes of Health BRAIN initiative (1F32MH122998-01), and the Wu Tsai Stanford Neurosciences Institute Interdisciplinary Scholar Program. S.G. is supported by the James S. McDonnell Foundation, Simons Foundation, and National Science Foundation CAREER Award for funding while at Stanford. L.M.G. is supported by the Office of Naval Research N00141812690, Simons Foundation SCGB 542987SPI, the James S. McDonnell Foundation, and the Vallee Foundation. D.L.K.Y. is supported by the James S. McDonnell Foundation (Understanding Human Cognition Award Grant No. 220020469), the Simons Foundation (Collaboration on the Global Brain Grant No. 543061), the Sloan Foundation (Fellowship FG-2018-10963), the National Science Foundation (RI 1703161 and CAREER Award 1844724), the DARPA Machine Common Sense program, and hardware donation from the NVIDIA Corporation.

title: Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks