Answers.Body: <p>What do you mean by "understood the concept of a car" ? A classification algorithm need not know that cars are used to transport humans or anything else it's used for. It only needs to know what combination of features we decide to label as a car. </p>
  
  <p>Or maybe you meant we could use some other metric for understanding that is not classification accuracy?</p>
  
  <p>This is a kind of Catch-22 situation. </p>
  
  <p>When we want to classify a car we can reformulate the task to learning what set of a combination of features do <em>us humans</em> decide to call a car. What labels we want to assign a combination of features is arbitrary to a degree. It preferably has to be  consistent for inference but an NN can learn a dataset assigned completely random features to a 100% accuracy.</p>
  
  <p>For instance, perhaps in an alternative universe we label the front-view of a car as 'B' and the rear view as 'A' and the notion/label of 'car' is nonexistent: even though we are ostensibly referencing the same object in physical space, we can assign it different labels. </p>
  
  <p>It follows that to measure if a model has properly interpolated what combination of features we like to call car, we ourselves must be able list out such features; and if we were able to do that, we would not not need the process of learning/teaching algorithms in the first place!</p>
  
  <p>A much more interesting subject is testing the extrapolative powers of neural networks- something much more proximate to what we canonically refer to as 'reasoning'.</p>
  
  <p>It's quite hard to measure this in static environments such as the classification of images, but the ability to extrapolate is more apparent in reinforcement learning settings where the model requires us to train it for times order of magnitudes above the time for needed for humans. </p>
  
  <p>The whole notion of pristine reasoning in machine learning breaks down when it comes to induction(see <a href="https://plato.stanford.edu/entries/sufficient-reason/#WhatSuffReas" rel="nofollow noreferrer">sufficient reason</a>). Thus, we resort to probabilities that can be roughly translated to "this line of reasoning yielded the correct result 99.9% of the time, so it would make sense that we keep using it. </p>
  
  <p>In fact, the idea of perfect-as-possible induction has been mathematically formulated except it isn't computable :( <a href="https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference" rel="nofollow noreferrer">Solomonoff Induction</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-26T21:43:52.173Z
Answers.LastActivityDate: 2018-10-26T21:43:52.173Z
Answers.UserDisplayName: Daniel
Body: <p>Which possibilities exist to evaluate the visual reasoning capabilities of neural networks in the field of image recognition?</p>
  
  <p>Are there methods to measure the ability of machine reasoning?</p>
  
  <p>Or something more specific: Is it possible to measure if a network understood the concept of a car / a cat / a human without using the classification accuracy.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-25T10:16:31.263Z
Comments.Text: I think it's a duplicate of : https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks (at least related)
Comments.UserDisplayName: Jérémy Blain
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-26T00:33:24.227Z
Comments.Text: @JérémyBlain thanks for linking that related QA.  My sense is this is a slightly different question, focused on how to measure capability as opposed to understanding the internal reasoning process.
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-27T13:29:54.277Z
Comments.Text: You must define "machine reasoning", so that an answer can be given.
Comments.UserDisplayName: nbro
Tags: <convolutional-neural-networks><image-recognition><reasoning>
Title: How to measure the reasoning capabilities of neural networks
UserDisplayName: Gilfoyle
ThreadText: # How to measure the reasoning capabilities of neural networks
  
  (Asked by: username_0 on 2018-10-25)
  
  Which possibilities exist to evaluate the visual reasoning capabilities of neural networks in the field of image recognition?
  
  Are there methods to measure the ability of machine reasoning?
  
  Or something more specific: Is it possible to measure if a network understood the concept of a car / a cat / a human without using the classification accuracy.
  
  **Tags:** convolutional-neural-networks, image-recognition, reasoning
  
  > **Comments:**
  > **username_1**: I think it's a duplicate of : https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks (at least related)  
  > **username_2**: @username_1 thanks for linking that related QA.  My sense is this is a slightly different question, focused on how to measure capability as opposed to understanding the internal reasoning process.  
  > **username_3**: You must define "machine reasoning", so that an answer can be given.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2018-10-26. Score: 1
  
  What do you mean by "understood the concept of a car" ? A classification algorithm need not know that cars are used to transport humans or anything else it's used for. It only needs to know what combination of features we decide to label as a car.
  
  Or maybe you meant we could use some other metric for understanding that is not classification accuracy?
  
  This is a kind of Catch-22 situation.
  
  When we want to classify a car we can reformulate the task to learning what set of a combination of features do *us humans* decide to call a car. What labels we want to assign a combination of features is arbitrary to a degree. It preferably has to be consistent for inference but an NN can learn a dataset assigned completely random features to a 100% accuracy.
  
  For instance, perhaps in an alternative universe we label the front-view of a car as 'B' and the rear view as 'A' and the notion/label of 'car' is nonexistent: even though we are ostensibly referencing the same object in physical space, we can assign it different labels.
  
  It follows that to measure if a model has properly interpolated what combination of features we like to call car, we ourselves must be able list out such features; and if we were able to do that, we would not not need the process of learning/teaching algorithms in the first place!
  
  A much more interesting subject is testing the extrapolative powers of neural networks- something much more proximate to what we canonically refer to as 'reasoning'.
  
  It's quite hard to measure this in static environments such as the classification of images, but the ability to extrapolate is more apparent in reinforcement learning settings where the model requires us to train it for times order of magnitudes above the time for needed for humans.
  
  The whole notion of pristine reasoning in machine learning breaks down when it comes to induction(see [sufficient reason](https://plato.stanford.edu/entries/sufficient-reason/#WhatSuffReas)). Thus, we resort to probabilities that can be roughly translated to "this line of reasoning yielded the correct result 99.9% of the time, so it would make sense that we keep using it.
  
  In fact, the idea of perfect-as-possible induction has been mathematically formulated except it isn't computable :( [Solomonoff Induction](https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference)
  
  
  ---
  
  