Answers.Body: <p>The KPI's would probably be different for each methodology.  For example an NB classification algorithm can easily reach %97 and above (and it would be very easy to measure this) but it would be difficult to measure say a game playing algorithm.  Would you suggest counting the wins/losses or another way to measure them?</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-11T15:54:48.047Z
Answers.Text: Take a 4,5 years old child for example. It takes 2 or 3 times to point on a cat and a child is able to recognise it. Take Google for example. 10^7 cat pictures is probably nearly enough to learn Google how cat look like. I would like to measure this gap.
Answers.UserDisplayName: kakaz
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-13T05:51:14.083Z
Answers.Text: It would entirely depend on what algorithm you chose to use.  I'm a huge fan of NB (personal choice) as it is easy to construct and works surprisingly well compared to more sophisticated methods.  Try using Weka and play around with the differences in algorithms, it will also provide the statistics you are looking for.  You can download it for free from the Waikato Uni website.
Answers.UserDisplayName: solarflare
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-11T07:24:32.457Z
Answers.LastActivityDate: 2017-09-11T07:24:32.457Z
Answers.UserDisplayName: solarflare
Body: <p>There should be some <a href="https://en.wikipedia.org/wiki/Performance_indicator" rel="nofollow noreferrer">Key Performance Indicators</a> designed for measuring AI performance. For example, the number of entities examples you have to feed it in order to obtain single task on a testing entity with repeatable 97% accuracy.</p>
  
  <p>Is there any of such measure constructed?</p>
  
  <p>Motivation: you cannot learn AI to be good at playing a game released 1 month ago because there is not enough data on how to play. It doesn't matter how clever you are, present brute force paradigm in AI just doesn't fit in such circumstances.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-11T16:40:06.293Z
Comments.Text: Welcome to AI!  I feel this needs a slight qualification in that game complexity is a determining factor.  If complexity is low, brute force is more than adequate. If the game is non-trivial, solution and AI performance becomes an issue. Great question though.
Comments.UserDisplayName: DukeZhou
Tags: <machine-learning><deep-learning>
Title: AI efficiency KPI
UserDisplayName: kakaz
ThreadText: # AI efficiency KPI
  
  (Asked by: username_0 on 2017-09-10)
  
  There should be some [Key Performance Indicators](https://en.wikipedia.org/wiki/Performance_indicator) designed for measuring AI performance. For example, the number of entities examples you have to feed it in order to obtain single task on a testing entity with repeatable 97% accuracy.
  
  Is there any of such measure constructed?
  
  Motivation: you cannot learn AI to be good at playing a game released 1 month ago because there is not enough data on how to play. It doesn't matter how clever you are, present brute force paradigm in AI just doesn't fit in such circumstances.
  
  **Tags:** machine-learning, deep-learning
  
  > **Comments:**
  > **username_1**: Welcome to AI!  I feel this needs a slight qualification in that game complexity is a determining factor.  If complexity is low, brute force is more than adequate. If the game is non-trivial, solution and AI performance becomes an issue. Great question though.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2017-09-11. Score: 3
  
  The KPI's would probably be different for each methodology. For example an NB classification algorithm can easily reach %97 and above (and it would be very easy to measure this) but it would be difficult to measure say a game playing algorithm. Would you suggest counting the wins/losses or another way to measure them?
  
  > **Comments:**
  > **username_0**: Take a 4,5 years old child for example. It takes 2 or 3 times to point on a cat and a child is able to recognise it. Take Google for example. 10^7 cat pictures is probably nearly enough to learn Google how cat look like. I would like to measure this gap.  
  > **username_2**: It would entirely depend on what algorithm you chose to use.  I'm a huge fan of NB (personal choice) as it is easy to construct and works surprisingly well compared to more sophisticated methods.  Try using Weka and play around with the differences in algorithms, it will also provide the statistics you are looking for.  You can download it for free from the Waikato Uni website.  
  
  ---
  
  