Solving Vision Tasks with Simple Photoreceptors Instead of Cameras
<|reference_start|>Solving Vision Tasks with Simple Photoreceptors Instead of Cameras: A de facto standard in solving computer vision problems is to use a common high-resolution camera and choose its placement on an agent (i.e., position and orientation) based on human intuition. On the other hand, extremely simple and well-designed visual sensors found throughout nature allow many organisms to perform diverse, complex behaviors. In this work, motivated by these examples, we raise the following questions: 1. How effective simple visual sensors are in solving vision tasks? 2. What role does their design play in their effectiveness? We explore simple sensors with resolutions as low as one-by-one pixel, representing a single photoreceptor First, we demonstrate that just a few photoreceptors can be enough to solve many tasks, such as visual navigation and continuous control, reasonably well, with performance comparable to that of a high-resolution camera. Second, we show that the design of these simple visual sensors plays a crucial role in their ability to provide useful information and successfully solve these tasks. To find a well-performing design, we present a computational design optimization algorithm and evaluate its effectiveness across different tasks and domains, showing promising results. Finally, we perform a human survey to evaluate the effectiveness of intuitive designs devised manually by humans, showing that the computationally found design is among the best designs in most cases.<|reference_end|>
@article{atanov2024solving,
    title={Solving Vision Tasks with Simple Photoreceptors Instead of Cameras},
    author={Andrei Atanov, Jiawei Fu, Rishubh Singh, Isabella Yu, Andrew
  Spielberg, Amir Zamir},
    journal={arXiv preprint arXiv:2406.11769},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.11769},
    primaryClass={cs.CV}
}