Answers.Body: <p>Yes. Here are some of the most prominent ones and their respective state-of-the-art errors:</p>
  
  <ul>
  <li><a href="https://www.cs.toronto.edu/~kriz/cifar.html" rel="nofollow">CIFAR-10</a>: ~3.5% error</li>
  <li><a href="https://www.cs.toronto.edu/~kriz/cifar.html" rel="nofollow">CIFAR-100</a>: ~24% error  </li>
  <li><a href="https://cs.stanford.edu/~acoates/stl10/" rel="nofollow">STL-10</a>: ~26% error</li>
  <li><a href="http://ufldl.stanford.edu/housenumbers/" rel="nofollow">SVHN</a>: ~1.7% error</li>
  <li><a href="http://www.image-net.org/" rel="nofollow">ImageNet</a> tasks: the best 2012 classification task solution got 15% top-5 error, better results are currently available</li>
  </ul>
  
  <p>You can check an updated list of solutions <a href="http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html" rel="nofollow">here</a>. Also, a more comprehensive list of modern datasets can be found <a href="http://deeplearning.net/datasets/" rel="nofollow">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-08T09:54:46.663Z
Answers.Text: The most competitive ILSVRC 2012 recently got a 3% top-5 error, see Inception-v4 [paper](http://arxiv.org/abs/1602.07261). It has been below 15% for a long time actually.
Answers.UserDisplayName: nom
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-07T18:09:03.483Z
Answers.LastActivityDate: 2016-08-08T13:58:28.580Z
Answers.LastEditDate: 2016-08-08T13:58:28.580Z
Answers.UserDisplayName: rcpinto
Body: <p>For example there is <a href="https://en.wikipedia.org/wiki/MNIST_database" rel="nofollow">the MNIST database</a> which is used to test artificial neural network (ANN), however it's not so challenging, because some hierarchical systems of convolutional neural networks manages to get an error rate of 0.23 percent.</p>
  
  <p>Are there any similar, especially the most challenging tasks with dataset which are used as benchmark tests to challenge the AI which are fairly reliable and it's possible to pass, but most AAN are struggling to achieve the lower error rate?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-02-10T12:08:21.283Z
Comments.Text: See also: The [HASYv2 dataset](https://arxiv.org/abs/1701.08380) (my paper). The data is similar to MNIST, but 369 classes. The best model has an accuracy of ~82%.
Comments.UserDisplayName: Martin Thoma
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-08-06T09:30:25.450Z
Comments.Text: In the field of game playing Ai the next challenge (after Go) is Starcraft.
Comments.UserDisplayName: Conor Cosnett
Tags: <deep-learning><image-recognition><data-science>
Title: What are the most challenging tasks aiming to achieve the lowest error rate?
UserDisplayName: kenorb
ThreadText: # What are the most challenging tasks aiming to achieve the lowest error rate?
  
  (Asked by: username_0 on 2016-08-06)
  
  For example there is [the MNIST database](https://en.wikipedia.org/wiki/MNIST_database) which is used to test artificial neural network (ANN), however it's not so challenging, because some hierarchical systems of convolutional neural networks manages to get an error rate of 0.23 percent.
  
  Are there any similar, especially the most challenging tasks with dataset which are used as benchmark tests to challenge the AI which are fairly reliable and it's possible to pass, but most AAN are struggling to achieve the lower error rate?
  
  **Tags:** deep-learning, image-recognition, data-science
  
  > **Comments:**
  > **username_2**: In the field of game playing Ai the next challenge (after Go) is Starcraft.  
  > **username_1**: See also: The [HASYv2 dataset](https://arxiv.org/abs/1701.08380) (my paper). The data is similar to MNIST, but 369 classes. The best model has an accuracy of ~82%.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2016-08-07. Score: 5 | Accepted answer
  
  Yes. Here are some of the most prominent ones and their respective state-of-the-art errors:
  
  * [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html): ~3.5% error
  * [CIFAR-100](https://www.cs.toronto.edu/~kriz/cifar.html): ~24% error
  * [STL-10](https://cs.stanford.edu/~acoates/stl10/): ~26% error
  * [SVHN](http://ufldl.stanford.edu/housenumbers/): ~1.7% error
  * [ImageNet](http://www.image-net.org/) tasks: the best 2012 classification task solution got 15% top-5 error, better results are currently available
  
  You can check an updated list of solutions [here](http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html). Also, a more comprehensive list of modern datasets can be found [here](http://deeplearning.net/datasets/).
  
  > **Comments:**
  > **username_4**: The most competitive ILSVRC 2012 recently got a 3% top-5 error, see Inception-v4 [paper](http://arxiv.org/abs/1602.07261). It has been below 15% for a long time actually.  
  
  ---
  
  