From Cognition to Computation: A Comparative Review of Human Attention and Transformer Architectures
<|reference_start|>From Cognition to Computation: A Comparative Review of Human Attention and Transformer Architectures: Attention is a cornerstone of human cognition that facilitates the efficient extraction of information in everyday life. Recent developments in artificial intelligence like the Transformer architecture also incorporate the idea of attention in model designs. However, despite the shared fundamental principle of selectively attending to information, human attention and the Transformer model display notable differences, particularly in their capacity constraints, attention pathways, and intentional mechanisms. Our review aims to provide a comparative analysis of these mechanisms from a cognitive-functional perspective, thereby shedding light on several open research questions. The exploration encourages interdisciplinary efforts to derive insights from human attention mechanisms in the pursuit of developing more generalized artificial intelligence.<|reference_end|>
@article{zhao2024from,
    title={From Cognition to Computation: A Comparative Review of Human Attention
  and Transformer Architectures},
    author={Minglu Zhao, Dehong Xu, Tao Gao},
    journal={arXiv preprint arXiv:2407.01548},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.01548},
    primaryClass={q-bio.OT cs.AI cs.LG}
}