Answers.Body: <p>I think you are slightly confusing 2 problems. 1 being classification of meta visual elements and the other being the visual system itself.</p>
  
  <p>Our visual system, when it comes to processing information, has had billions of years of iteration(training), so that at birth(and before), we are already tuned for the processing of visual stimuli, as well as have the mechanisms to decipher objects in our spatial field of view. </p>
  
  <p>These two papers(<a href="https://www.sciencedirect.com/science/article/abs/pii/S006528060860065X" rel="nofollow noreferrer">L1</a>, <a href="https://www.researchgate.net/profile/Adrian_Horridge/publication/20270930_The_Evolution_of_Visual_Processing_and_the_Construction_of_Seeing_Systems/links/574ce89808aec988526a266d.pdf" rel="nofollow noreferrer">L2</a>), have a great deal of information about the evolution of our visual system and its processing. The second speculates on the connection of said evolution and the construction of "seeing systems" very interesting.</p>
  
  <p>For further inquiry on this in particular, check out <a href="https://en.wikipedia.org/wiki/David_Marr_(neuroscientist)" rel="nofollow noreferrer">David Marr</a>. He was probably the most influential early computer vision mind. He still is mentioned in many top-down AGI and computer vision research projects to this day.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-05T13:56:00.547Z
Answers.Text: Do you mean babies are born right off the bat already identifying people, dogs, cats, mountains, water, intuitive physics, predicting object trajectories, etc.? Or do their brains learn this over time instead? If your answer to the last question is yes, then how do brains manage to find training examples to train their neural networks without having access to manually annotated datasets like ImageNet, COCO, etc.?
Answers.UserDisplayName: Pablo Messina
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-05T14:09:21.560Z
Answers.Text: @PabloMessina while tests on infants are difficult for obvious reasons, yes. We're born with ingrained classification functionality and can identify spatial forms after and likely before we are out of the womb. Physics is evolutionary as well(at least for examples relevant to what our ancestors would of faced, lions, snakes, etc.). Some things get learned over time as well, like object permanence. Trying to compare our visual system to convolutional neural networks is a very rough analogy and usually is only used for instructive reasons. Our visual system is more likely than not convolutional.
Answers.UserDisplayName: hisairnessag3
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-07T06:03:05.583Z
Answers.Text: Likely not convolutional*
Answers.UserDisplayName: hisairnessag3
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-07T14:26:38.730Z
Answers.Text: Hey @hisairnessag3, your answer motivated me to ask [this question](https://psychology.stackexchange.com/questions/21971/is-the-visual-cortex-of-a-newborn-baby-immediately-capable-of-object-detection-o) in Psychology & Neuroscience, feel free to answer it if you want
Answers.UserDisplayName: Pablo Messina
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-05T08:34:14.050Z
Answers.LastActivityDate: 2019-04-05T08:34:14.050Z
Answers.UserDisplayName: hisairnessag3
Body: <p>In computer vision is very common to use supervised tasks, where datasets have to be manually annotated by humans. Some examples are object classification (class labels), detection (bounding boxes) and segmentation (pixel-level masks). These datasets are essentially pairs of inputs-outputs which are used to train Convolutional Neural Networks to learn the mapping from inputs to outputs, via gradient descent optimization. But animals don't need anybody to show them bounding boxes or masks on top of things in order for them to learn to detect objects and make sense of the visual world around them. This leads me to think that brains must be performing some sort of self-supervision to train themselves to see.</p>
  
  <p>What does current research say about the learning paradigm used by brains to achieve such an outstanding level of visual competence? Which tasks do brains use to train themselves to be so good at processing visual information and making sense of the visual world around them? Or said in other words: how does the brain manage to train its neural networks without having access to manually annotated datasets like ImageNet, COCO, etc. (i.e. what does the brain use as ground truth, what is the loss function the brain is optimizing)? Finally, can we apply these insights in computer vision?</p>
  
  <hr>
  
  <p>Update: I posted a related question on Psychology &amp; Neuroscience StackExchange, which I think complements the question I posted here: <a href="https://psychology.stackexchange.com/questions/21971/is-the-visual-cortex-of-a-newborn-baby-immediately-capable-of-object-detection-o">check it out</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-05T17:20:29.370Z
Comments.Text: Cross-posted: https://psychology.stackexchange.com/q/21965/11209, https://ai.stackexchange.com/q/11666/1794, https://datascience.stackexchange.com/q/48645/8560.  Please [do not post the same question on multiple sites](https://meta.stackexchange.com/q/64068). Each community should have an honest shot at answering without anybody's time being wasted.
Comments.UserDisplayName: D.W.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-10T22:25:40.560Z
Comments.Text: Your brain is not a computer (see [this](https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer) or [this](https://psychcentral.com/blog/your-brain-is-not-a-computer/)). Your brain doesn't optimize a loss function.
  
  Also, even if I reluctantly accept this analogy, I feel like the visual system has plenty of "external supervision" from the other senses, etc.
Comments.UserDisplayName: sfmiller940
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-11T03:17:23.847Z
Comments.Text: @sfmiller940 if the brain is not optimizing a loss, then what mechanism does the brain use to learn new things? Regarding other senses, can you elaborate more on how they exert supervision on the visual system?
Comments.UserDisplayName: Pablo Messina
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-11T16:59:47.507Z
Comments.Text: @PabloMessina I don't think anybody knows exactly how the brain learns. But I'm not an expert, so I imagine that neuroscientists (or similar) would know much more about what is and isn't known. As far as supervision, let's consider the senses of touch and hearing. If we see a ball, we don't necessarily know if it's a discrete object but, if we can touch it, then we get additional information about it's shape and relation to other objects. Similarly we can hear sounds and voices that can instruct us directly or indirectly. Other senses seem a form of supervision that guides visual learning.
Comments.UserDisplayName: sfmiller940
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-11T21:00:48.323Z
Comments.Text: @sfmiller940 "Other senses seem a form of supervision that guides visual learning", probably, but how?
Comments.UserDisplayName: Pablo Messina
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-12T22:22:48.820Z
Comments.Text: @PabloMessina That's a question more for neuroscientists than data scientists.
Comments.UserDisplayName: sfmiller940
Tags: <machine-learning><computer-vision><cognitive-science><biology>
Title: Which loss function is the brain optimizing in order to learn advanced visual skills without expert/human supervision?
UserDisplayName: Pablo Messina
ThreadText: # Which loss function is the brain optimizing in order to learn advanced visual skills without expert/human supervision?
  
  (Asked by: username_0 on 2019-04-05)
  
  In computer vision is very common to use supervised tasks, where datasets have to be manually annotated by humans. Some examples are object classification (class labels), detection (bounding boxes) and segmentation (pixel-level masks). These datasets are essentially pairs of inputs-outputs which are used to train Convolutional Neural Networks to learn the mapping from inputs to outputs, via gradient descent optimization. But animals don't need anybody to show them bounding boxes or masks on top of things in order for them to learn to detect objects and make sense of the visual world around them. This leads me to think that brains must be performing some sort of self-supervision to train themselves to see.
  
  What does current research say about the learning paradigm used by brains to achieve such an outstanding level of visual competence? Which tasks do brains use to train themselves to be so good at processing visual information and making sense of the visual world around them? Or said in other words: how does the brain manage to train its neural networks without having access to manually annotated datasets like ImageNet, COCO, etc. (i.e. what does the brain use as ground truth, what is the loss function the brain is optimizing)? Finally, can we apply these insights in computer vision?
  
  ---
  
  Update: I posted a related question on Psychology & Neuroscience StackExchange, which I think complements the question I posted here: [check it out](https://psychology.stackexchange.com/questions/21971/is-the-visual-cortex-of-a-newborn-baby-immediately-capable-of-object-detection-o)
  
  **Tags:** machine-learning, computer-vision, cognitive-science, biology
  
  > **Comments:**
  > **username_1**: Cross-posted: https://psychology.stackexchange.com/q/21965/11209, https://ai.stackexchange.com/q/11666/1794, https://datascience.stackexchange.com/q/48645/8560.  Please [do not post the same question on multiple sites](https://meta.stackexchange.com/q/64068). Each community should have an honest shot at answering without anybody's time being wasted.  
  > **username_2**: Your brain is not a computer (see [this](https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer) or [this](https://psychcentral.com/blog/your-brain-is-not-a-computer/)). Your brain doesn't optimize a loss function.  Also, even if I reluctantly accept this analogy, I feel like the visual system has plenty of "external supervision" from the other senses, etc.  
  > **username_0**: @username_2 if the brain is not optimizing a loss, then what mechanism does the brain use to learn new things? Regarding other senses, can you elaborate more on how they exert supervision on the visual system?  
  > **username_2**: @username_0 I don't think anybody knows exactly how the brain learns. But I'm not an expert, so I imagine that neuroscientists (or similar) would know much more about what is and isn't known. As far as supervision, let's consider the senses of touch and hearing. If we see a ball, we don't necessarily know if it's a discrete object but, if we can touch it, then we get additional information about it's shape and relation to other objects. Similarly we can hear sounds and voices that can instruct us directly or indirectly. Other senses seem a form of supervision that guides visual learning.  
  > **username_0**: @username_2 "Other senses seem a form of supervision that guides visual learning", probably, but how?  
  > **username_2**: @username_0 That's a question more for neuroscientists than data scientists.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-04-05. Score: 3
  
  I think you are slightly confusing 2 problems. 1 being classification of meta visual elements and the other being the visual system itself.
  
  Our visual system, when it comes to processing information, has had billions of years of iteration(training), so that at birth(and before), we are already tuned for the processing of visual stimuli, as well as have the mechanisms to decipher objects in our spatial field of view.
  
  These two papers([L1](https://www.sciencedirect.com/science/article/abs/pii/S006528060860065X), [L2](https://www.researchgate.net/profile/Adrian_Horridge/publication/20270930_The_Evolution_of_Visual_Processing_and_the_Construction_of_Seeing_Systems/links/574ce89808aec988526a266d.pdf)), have a great deal of information about the evolution of our visual system and its processing. The second speculates on the connection of said evolution and the construction of "seeing systems" very interesting.
  
  For further inquiry on this in particular, check out [David Marr](https://en.wikipedia.org/wiki/David_Marr_(neuroscientist)). He was probably the most influential early computer vision mind. He still is mentioned in many top-down AGI and computer vision research projects to this day.
  
  > **Comments:**
  > **username_0**: Do you mean babies are born right off the bat already identifying people, dogs, cats, mountains, water, intuitive physics, predicting object trajectories, etc.? Or do their brains learn this over time instead? If your answer to the last question is yes, then how do brains manage to find training examples to train their neural networks without having access to manually annotated datasets like ImageNet, COCO, etc.?  
  > **username_3**: @username_0 while tests on infants are difficult for obvious reasons, yes. We're born with ingrained classification functionality and can identify spatial forms after and likely before we are out of the womb. Physics is evolutionary as well(at least for examples relevant to what our ancestors would of faced, lions, snakes, etc.). Some things get learned over time as well, like object permanence. Trying to compare our visual system to convolutional neural networks is a very rough analogy and usually is only used for instructive reasons. Our visual system is more likely than not convolutional.  
  > **username_3**: Likely not convolutional*  
  > **username_0**: Hey @username_3, your answer motivated me to ask [this question](https://psychology.stackexchange.com/questions/21971/is-the-visual-cortex-of-a-newborn-baby-immediately-capable-of-object-detection-o) in Psychology & Neuroscience, feel free to answer it if you want  
  
  ---
  
  