Answers.Body: <p>In theory, most neural networks can approximate any continuous function on compact subsets of <span class="math-container">$\mathbb{R}^n$</span>, provided that the activation functions satisfy certain mild conditions. This is known as the <em>universal approximation theorem</em> (UAT), but that should <em>not</em> be called  <em>universal</em>, given that there are a lot more discontinuous functions than continuous ones, although certain discontinuous functions can be approximated by continuous ones. The UAT shows the theoretical powerfulness of neural networks and their purpose. They represent and approximate functions. If you want to know more about the details of the UAT, for different neural network architectures, see <a href="https://ai.stackexchange.com/a/13319/2444">this answer</a>.</p>
  
  <p>However, in practice, neural networks trained with gradient descent and backpropagation face several issues and challenges, some of which are due to the training procedure and not just the architecture of the neural network or available data.</p>
  
  <p>For example, it is well known that <a href="https://ai.stackexchange.com/a/13293/2444">neural networks are prone to catastrophic forgetting (or interference)</a>, which means that they aren't particularly suited for <strong>incremental learning</strong> tasks, although some more sophisticated incremental learning algorithms based on neural networks have already been developed.</p>
  
  <p>Neural networks can also be <a href="https://arxiv.org/abs/1712.07107" rel="noreferrer">sensitive to their inputs</a>, i.e. a small change in the inputs can drastically change the output (or answer) of the neural network. This is partially due to the fact that they learn a function that isn't really the function you expect them to learn. So, a system based on such a neural network can potentially be hacked or fooled, so they are probably not well suited for <strong>safety-critical applications</strong>. This issue is related to the <a href="https://arxiv.org/pdf/2001.02522.pdf" rel="noreferrer">low interpretability and explainability of neural networks</a>, i.e. they are often denoted as <a href="https://stats.stackexchange.com/q/93705/82135">black-box models</a>.</p>
  
  <p><a href="https://arxiv.org/abs/1505.05424" rel="noreferrer">Bayesian neural networks (BNNs)</a> can potentially mitigate these problems, but they are unlikely to be the ultimate or complete solution. Bayesian neural networks maintain a distribution for each of the units (or neurons), rather than a point estimate. In principle, this can provide more uncertainty guarantees, but, in practice, this is not yet the case.</p>
  
  <p>Furthermore, neural networks often require a lot of data in order to approximate the desired function accurately, so in cases where <strong>data is scarce</strong> neural networks may not be appropriate. Moreover, the training of neural networks (especially, deep architectures) also <strong>requires a lot of computational resources</strong>. Inference can also be sometimes problematic, when you need real-time predictions, as it can also be expensive.</p>
  
  <p>To conclude, neural networks are just function approximators, i.e. they approximate a <em>specific</em> function (or set of functions, in the case of Bayesian neural networks), given a specific configuration of the parameters. They can't do more than that. They cannot magically do something that they have not been trained to do, and it is usually the case that you don't really know the specific function the neural network is representing (hence the expression <em>black-box model</em>), apart from knowing your training dataset, which can also contain spurious information, among other issues.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-15T23:52:29.990Z
Answers.Text: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/105593/discussion-on-answer-by-nbro-what-are-some-well-known-problems-where-neural-netw).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-11T18:20:08.750Z
Answers.LastActivityDate: 2020-03-12T23:31:27.883Z
Answers.LastEditDate: 2020-03-12T23:31:27.883Z
Answers.UserDisplayName: nbro
Answers.Body: <p>In our deep learning lecture, we discussed the following example (from <a href="https://www.nature.com/articles/s41467-019-08987-4" rel="noreferrer">Unmasking Clever Hans predictors and assessing what machines really learn</a> (2019) by Lapuschkin et al.).</p>
  
  <p><a href="https://i.sstatic.net/cLggJ.jpg" rel="noreferrer"><img src="https://i.sstatic.net/cLggJ.jpg" alt="enter image description here"></a></p>
  
  <p>Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong "relevant components". In the sensitivity maps next to the pictures, we can see that the watermark was used to identify if there is a horse present in the picture. If we remove the watermark, the classification is no longer made. Even more worryingly, if we add the tag to a completely different picture, it gets identified as a horse!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T23:33:39.723Z
Answers.Text: This is similar to the famous [tank story](https://www.lesswrong.com/posts/5o3CxyvZ2XKawRB5w/machine-learning-and-unintended-consequences).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T22:45:57.283Z
Answers.LastActivityDate: 2020-03-23T08:32:22.043Z
Answers.LastEditDate: 2020-03-23T08:32:22.043Z
Answers.UserDisplayName: ViktorStein
Answers.Body: <p>This is more in the direction of 'what kind of problems can be solved by neural networks'. In order to train a neural network you need a large set of training data which is labelled with correct/ incorrect for the question you are interested in. So for example 'identify all pictures that have a cat on them' is very suitable for neural networks. On the other hand 'summarize the story of this toddler picture book' is very hard. Although a human can easily decide whether a given summary is any good or not it would be very difficult to build a suitable set of training data for this kind of problem. So if you can't build a large training data set with correct answers, you can't train a neural network to solve the problem.</p>
  
  <p>The answer of Anshuman Kumar is also an instance of that, also a potentially solvable one. The neural network that misidentified upside-down school buses presumably had very few if any upside-down school buses in its training data. Put them into the training data and the neural network will identify these as well. This is still a flaw in neural networks, a human can correctly identify an upside-down school bus the first time they see one if they know what school busses look like.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T12:47:06.687Z
Answers.Text: Good point.  Also using a feature map which rotates and stretches immages to create more "artificial" immage samples should bypass Anshuman's problem
Answers.UserDisplayName: ABIM
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T03:19:12.980Z
Answers.Text: @AIM_BLB you'd need a feature map that can do so in 3D. So you pretty much need the whole pipeline the brain does: edge detection -> depth map -> segmentation -> 3D feature map -> classifier search (with each stage feeding back to the edge detection (!) stage and all later ones). And that's without any depth cues, meaning that the depth map can only take input from the classifier and the segmentation map.
Answers.UserDisplayName: John Dvorak
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T07:59:50.753Z
Answers.LastActivityDate: 2020-03-12T07:59:50.753Z
Answers.UserDisplayName: quarague
Answers.Body: <p>I don't know if it might be of use, but many areas of NLP are still hard to tackle, and even if deep models achieve the state of the art results, they usually beat baseline shallow models by very few percentage points. 
  One example that I've had the opportunity to work on is stance classification <a href="https://www.aclweb.org/anthology/S16-1075.pdf" rel="nofollow noreferrer">1</a>. In many datasets, the best F score achievable is around 70%. </p>
  
  <p>Even though it's hard to compare results since in NLP many datasets are really small and domain-specific (especially for stance detection and similar SemEval tasks), many times SVM, conditional random fields, sometimes even Naive Bayes models are able to perform almost as good as CNN or RNN.  Other tasks for which this holds are argumentation mining or claim detection.</p>
  
  <p>See e.g. the paper <a href="https://www.aclweb.org/anthology/S16-1075.pdf" rel="nofollow noreferrer">TakeLab at SemEval-2016 Task 6: Stance Classification in Tweets Using a Genetic Algorithm Based Ensemble</a> (2016) by Martin Tutek et al.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-03T23:42:27.703Z
Answers.Text: Thanks for this answer.  I'm interested in this subject from the standpoint of (satisficing)[https://en.wikipedia.org/wiki/Satisficing]!  (i.e. if time or space is sufficiently restricted, the "less optimal" method may prevail.)
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T13:45:32.253Z
Answers.LastActivityDate: 2020-03-14T01:37:09.160Z
Answers.LastEditDate: 2020-03-14T01:37:09.160Z
Answers.UserDisplayName: Edoardo Guerriero
Answers.Body: <p>My 50cents:
  <a href="https://en.wikipedia.org/wiki/NP_(complexity)" rel="nofollow noreferrer">NP_(complexity)</a> - is still hard to solve, even with NeuralNets.</p>
  
  <blockquote>
    <p>In computational complexity theory, NP (nondeterministic polynomial
    time) is a complexity class used to classify decision problems. NP is
    the set of decision problems for which the problem instances, where
    the answer is "yes", have proofs verifiable in polynomial time by a
    deterministic Turing machine.</p>
  </blockquote>
  
  <p>The easiest example, to imagine what is speech about, it is cryptography's <a href="https://en.wikipedia.org/wiki/Integer_factorization" rel="nofollow noreferrer">Integer_factorization</a>, which is basement of <a href="https://en.wikipedia.org/wiki/RSA_(cryptosystem)" rel="nofollow noreferrer">RSA cryptosystem</a>.</p>
  
  <p>For example, we have two simple numbers:</p>
  
  <ul>
  <li>12123123123123123123123.....45456</li>
  <li>23412421341234124124124.....11112</li>
  </ul>
  
  <p>NeuralNetwork shall answer us exactly digit to digit both this numbers, when we will show it only multiplication of this two numbers... This is not guessing about school bus. The field of numbers much more bigger than number of words in all languages on whole Earth. Imagine, that there are billions of billion different school buses, billions of billions of different fire-hydrants and billions of such classes, and NN shall answer exactly - what is on the picture - no way. The chance to guess is so little... </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-15T16:35:43.893Z
Answers.LastActivityDate: 2020-04-15T16:35:43.893Z
Answers.UserDisplayName: ShnaierOnCryptography
Answers.Body: <p>A checkerboard with missing squares is impossible for a neural network to learn the missing color. The more it learns on training data, the worse it does on test data. </p>
  
  <p>See e.g. this article <a href="https://journals.blythinstitute.org/ojs/index.php/cbi/article/view/48" rel="nofollow noreferrer">The Unlearnable Checkerboard Pattern</a> (which, unfortunately, is not freely accessible). In any case, it should be easy to try out yourself that this task is difficult.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T09:02:51.017Z
Answers.Text: Impossible is not the right word here; I'm pretty sure a neural network specially trained for identifying checkerboards will have no problem, but we'll have to see. Impossible is just too strong a word.
Answers.UserDisplayName: Sny
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-12T12:58:36.043Z
Answers.LastActivityDate: 2020-03-16T17:26:52.967Z
Answers.LastEditDate: 2020-03-16T17:26:52.967Z
Answers.UserDisplayName: yters
Answers.Body: <p>From my experience in industry, a lot of data science (operating on customer information, stored in a database) is still dominated by decision trees and even SVMs. Although neural networks have seen incredible performance on "unstructured" data, like images and text, there still do not appear to be great results extending to structured, tabular data (yet).  </p>
  
  <p>At my old company (loyalty marketing with 10 million+ members) there was a saying, "<em>You can try any model you like, but you must try XGBoost</em>". And let's just say that I did try comparing it to a neural network, and ultimately I did go with XGBoost ;) </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-23T08:25:51.397Z
Answers.Text: This sounds very promising.  So something like Boston housing would be a toy example data-set of this type?
Answers.UserDisplayName: ABIM
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-23T15:24:32.003Z
Answers.Text: Yes exactly! Same with the classic Adult dataset: https://archive.ics.uci.edu/ml/datasets/adult
Answers.UserDisplayName: information_interchange
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-23T15:32:12.060Z
Answers.Text: So in that case XGBoost is the best?
Answers.UserDisplayName: ABIM
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-23T17:36:51.713Z
Answers.Text: Well it's usually difficult to say in advance, and especially before you do tinkering like hyperparameter sweeps, regularization etc. But in general, XGBoost gives you good results right out of the box, with the added bonus that it is also somewhat interpretable (such as giving you feature importances), allowing you to explain things in a business setting nicely
Answers.UserDisplayName: information_interchange
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-23T00:34:02.677Z
Answers.LastActivityDate: 2020-03-23T00:34:02.677Z
Answers.UserDisplayName: information_interchange
Answers.Body: <p>Neural networks seem to have a great deal of difficulty handling <em>adversarial input</em>, i.e., inputs with certain changes (often imperceptible or nearly imperceptible by humans) designed by an attacker to fool them.</p>
  
  <p>This is not the same thing as just being highly sensitive to certain changes in inputs. Robustness against wrong answers in that case can be increased by reducing the probability of such inputs. (If only one in 10^15 possible images causes a problem, it's not much of a problem.) However, in the adversarial case reducing the space of problematic images doesn't reduce the probability of getting one because the images are specifically selected by the attacker.</p>
  
  <p>One of the more famous papers in this area is <a href="https://arxiv.org/pdf/1707.07397.pdf" rel="nofollow noreferrer">"Synthesizing Robust Adversarial Examples"</a>, which produced not only examples where a few modified pixels or other invisible-to-humans modifications to a picture fooled a neural network-based image classifier, but also perhaps the first examples of 3D objects designed to fool similar classifiers and successfully doing so (from every angle!).</p>
  
  <p>(Those familiar with IT securitity will no doubt recognise this as a familiar asymmetry: roughly, a defender must defend against <em>all</em> attacks launched against a system, but an attacker need find only <em>one</em> working attack.)</p>
  
  <p>In <a href="https://arxiv.org/pdf/1901.10861.pdf" rel="nofollow noreferrer">"A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance"</a>, Adi Shamir et al. propose a mathematical framework for analyzing the problem based on Hamming distances that, while currently a less practical attack than the MIT/Lab6 one, has some pretty disturbing theoretical implications, including that current approaches to preventing these attacks may be, in the end, ineffective. For example, he points out that blurring and similar techniques that have been used to try to defend against adversarial attacks can be treated mathematically as simply another layer added on top of the existing neural network, requiring no changes to the attack strategy.</p>
  
  <p>(I attended a talk by Shamir a few months ago that was much easier going than the paper, but unfortunately I can't find a video of that or a similar talk on-line; if anybody knows of one please feel free to edit this answer to add a link!)</p>
  
  <p>There's obviously still an enormous amount of research to be done in this area, but it seems possible that neural networks alone are not capable of defense against this class of attack, and other techniques will have to be employed in addition to make neural networks robust against it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-03T23:53:48.507Z
Answers.Text: The attacks aren't quite the same if you apply random blurring. In principle adding random noise should provide sufficient defense, if your network on average handles noise well. I have seen attempts to use lipschitz like constraints but for deep networks the bounds are useless. It's kind of circular in a way - we want the network to function like humans do but there isn't a clear norm for how humans judge similarity between images.
Answers.UserDisplayName: FourierFlux
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-14T01:23:09.577Z
Answers.LastActivityDate: 2020-03-14T02:07:00.837Z
Answers.LastEditDate: 2020-03-14T02:07:00.837Z
Answers.UserDisplayName: cjs
Answers.Body: <p>Large scale route optimization problems.</p>
  
  <p>The is progress made in using Deep Reinforcement learning to solve vehicle routing problems (VRP), for example in this paper: <a href="https://arxiv.org/abs/1802.04240v2" rel="nofollow noreferrer">https://arxiv.org/abs/1802.04240v2</a>. </p>
  
  <p>However, for large scale problems and overall heuristic methods, like the ones provided by Google OR tools are much easier to use.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-14T01:32:44.357Z
Answers.Text: You don't talk about neural networks in your answer, but only about deep RL. Maybe you can clarify how neural networks are related to your linked paper and the "large scale route optimization problems".
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T05:57:09.487Z
Answers.LastActivityDate: 2020-03-13T05:57:09.487Z
Answers.UserDisplayName: Akavall
Answers.Body: <p>In the case of convolutional neural networks, the features may be extracted but without taking into account their relative positions (see the concept of <em>translation invariance</em>)</p>
  
  <p>For example, you could have two eyes, a nose and a mouth be in different locations in an image and still have the image be classified as a face.</p>
  
  <p>Operations like max-pooling may also have a negative impact on retaining position information.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-16T17:24:01.420Z
Answers.Text: I have edited your answer to make it clearer. Please, can you provide a research work that proves that the extraction of features, in a CNN, can have a negative impact such that a collection of two eyes, a nose and a mouth, although not in the appropriate relative position, is classified as a face?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T15:23:19.840Z
Answers.LastActivityDate: 2020-03-16T17:22:43.483Z
Answers.LastEditDate: 2020-03-16T17:22:43.483Z
Answers.UserDisplayName: FourierFlux
Answers.Body: <p>Here's a snippet from <a href="https://medium.com/@GaryMarcus/the-deepest-problem-with-deep-learning-91c5991f5695?_branch_match_id=687598797934356895" rel="noreferrer">an article by Gary Marcus</a> </p>
  
  <blockquote>
    <p>In particular, they showed that standard deep learning nets often fall
    apart when confronted with common stimuli rotated in three dimensional
    space into unusual positions, like the top right corner of this
    figure, in which a schoolbus is mistaken for a snowplow:</p>
    
    <p><a href="https://i.sstatic.net/C4Bch.jpg" rel="noreferrer"><img src="https://i.sstatic.net/C4Bch.jpg" alt="enter image description here"></a></p>
    
    <p>.
    .
    .</p>
    
    <p>Mistaking an overturned schoolbus is
    not just a mistake, it’s a revealing mistake: it that shows not only
    that deep learning systems can get confused, but they are challenged
    in making a fundamental distinction known to all philosophers: the
    distinction between features that are merely contingent associations
    (snow is often present when there are snowplows, but not necessary)
    and features that are inherent properties of the category itself
    (snowplows ought other things being equal have plows, unless eg they
    have been dismantled). We’d already seen similar examples with
    contrived stimuli, like Anish Athalye’s carefully designed, 3-d
    printed foam covered dimensional baseball that was mistaken for an
    espresso</p>
    
    <p><a href="https://i.sstatic.net/tdE0h.jpg" rel="noreferrer"><img src="https://i.sstatic.net/tdE0h.jpg" alt="enter image description here"></a></p>
    
    <p>Alcorn’s results — some from real photos from the natural
    world — should have pushed worry about this sort of anomaly to the top
    of the stack.</p>
  </blockquote>
  
  <p>Please note that the opinions of the author are his alone and I do not necessarily share all of them with him.</p>
  
  <p>Edit: Some more fun stuff</p>
  
  <p>1) DeepMind's neural network that could play <em>Breakout</em> and <em>Starcraft</em>
  saw a dramatic dip in performance when the paddle was moved up by a few pixels.</p>
  
  <p><a href="https://www.vicarious.com/posts/general-game-playing-with-schema-networks" rel="noreferrer">See: General Game Playing With Schema Networks</a></p>
  
  <p>While in the latter, it performed well with one race of the character but not on a different map and with different characters.</p>
  
  <p><a href="https://stream.nyu.edu/channel/CS%2BFaculty%2BSearch%2B2019/111128891" rel="noreferrer">Source</a></p>
  
  <p>2) </p>
  
  <blockquote>
    <p>AlphaZero searches just 80,000 positions per second in chess and
    40,000 in shogi, compared to 70 million for Stockfish and 35 million
    for elmo.</p>
  </blockquote>
  
  <p>What the team at Deepmind did was to build a very good search algorithm. A search algorithm that includes the capability to remember facets of previous searches to apply better results to new searches. This is very clever; it undoubtedly has immense value in many areas, but it cannot be considered general intelligence.</p>
  
  <p><a href="https://medium.com/intuitionmachine/alphazero-how-intuition-demolished-logic-66a4841e6810" rel="noreferrer">See: AlphaZero: How Intuition Demolished Logic (Medium)</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-15T23:52:47.977Z
Answers.Text: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/105594/discussion-on-answer-by-anshuman-kumar-what-are-some-well-known-problems-where-n).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-03T23:47:27.757Z
Answers.Text: Just an FYI: this link seems to have moved, or requires a login: https://stream.nyu.edu/channel/CS%2BFaculty%2BSearch%2B2019/111128891
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-11T17:41:19.353Z
Answers.LastActivityDate: 2020-05-03T23:47:00.690Z
Answers.LastEditDate: 2020-05-03T23:47:00.690Z
Answers.UserDisplayName: Anshuman Kumar
Body: <p><strong>Background:</strong> It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of <a href="https://ai.stackexchange.com/questions/13317/where-can-i-find-the-proof-of-the-universal-approximation-theorem/25917#25917">their universal approximation capabilities</a>.  However, in this post I'm curious about the <em>opposite</em>:</p>
  <p><strong>Question:</strong> Namely, what are some <em>well-known</em> cases, problems or real-world applications where neural networks don't do very well?</p>
  <hr />
  <p><strong>Specification:</strong>
  I'm looking for specific <em>regression</em> tasks (with accessible data-sets) where neural networks are not the state-of-the-art.  The regression task should be &quot;naturally suitable&quot;, so no sequential or time-dependent data (in which case an RNN or reservoir computer would be more natural).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-22T14:37:59.773Z
Comments.Text: [Here](https://ai.stackexchange.com/q/247/2444) is a related question, but it's more theoretical.
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><reference-request><applications>
Title: What are some well-known problems where neural networks don't do very well?
UserDisplayName: ABIM
ThreadText: # What are some well-known problems where neural networks don't do very well?
  
  (Asked by: username_0 on 2020-03-11)
  
  **Background:** It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of [their universal approximation capabilities](https://ai.stackexchange.com/questions/13317/where-can-i-find-the-proof-of-the-universal-approximation-theorem/25917#25917). However, in this post I'm curious about the *opposite*:
  
  **Question:** Namely, what are some *well-known* cases, problems or real-world applications where neural networks don't do very well?
  
  ---
  
  **Specification:**
  I'm looking for specific *regression* tasks (with accessible data-sets) where neural networks are not the state-of-the-art. The regression task should be "naturally suitable", so no sequential or time-dependent data (in which case an RNN or reservoir computer would be more natural).
  
  **Tags:** neural-networks, deep-learning, reference-request, applications
  > **Comments:**
  > **username_1**: [Here](https://ai.stackexchange.com/q/247/2444) is a related question, but it's more theoretical.  
  
  ---
  
  ## 11 answers
  
  ### Answer by username_14 on 2020-03-11. Score: 32 | Accepted answer
  
  Here's a snippet from [an article by Gary Marcus](https://medium.com/@GaryMarcus/the-deepest-problem-with-deep-learning-91c5991f5695?_branch_match_id=687598797934356895)
  
  > In particular, they showed that standard deep learning nets often fall
  > apart when confronted with common stimuli rotated in three dimensional
  > space into unusual positions, like the top right corner of this
  > figure, in which a schoolbus is mistaken for a snowplow:
  >
  > [![enter image description here](https://i.sstatic.net/C4Bch.jpg)](https://i.sstatic.net/C4Bch.jpg)
  >
  > .
  > .
  > .
  >
  > Mistaking an overturned schoolbus is
  > not just a mistake, it’s a revealing mistake: it that shows not only
  > that deep learning systems can get confused, but they are challenged
  > in making a fundamental distinction known to all philosophers: the
  > distinction between features that are merely contingent associations
  > (snow is often present when there are snowplows, but not necessary)
  > and features that are inherent properties of the category itself
  > (snowplows ought other things being equal have plows, unless eg they
  > have been dismantled). We’d already seen similar examples with
  > contrived stimuli, like Anish Athalye’s carefully designed, 3-d
  > printed foam covered dimensional baseball that was mistaken for an
  > espresso
  >
  > [![enter image description here](https://i.sstatic.net/tdE0h.jpg)](https://i.sstatic.net/tdE0h.jpg)
  >
  > Alcorn’s results — some from real photos from the natural
  > world — should have pushed worry about this sort of anomaly to the top
  > of the stack.
  
  Please note that the opinions of the author are his alone and I do not necessarily share all of them with him.
  
  Edit: Some more fun stuff
  
  1) DeepMind's neural network that could play *Breakout* and *Starcraft*
  saw a dramatic dip in performance when the paddle was moved up by a few pixels.
  
  [See: General Game Playing With Schema Networks](https://www.vicarious.com/posts/general-game-playing-with-schema-networks)
  
  While in the latter, it performed well with one race of the character but not on a different map and with different characters.
  
  [Source](https://stream.nyu.edu/channel/CS%2BFaculty%2BSearch%2B2019/111128891)
  
  2)
  
  > AlphaZero searches just 80,000 positions per second in chess and
  > 40,000 in shogi, compared to 70 million for Stockfish and 35 million
  > for elmo.
  
  What the team at Deepmind did was to build a very good search algorithm. A search algorithm that includes the capability to remember facets of previous searches to apply better results to new searches. This is very clever; it undoubtedly has immense value in many areas, but it cannot be considered general intelligence.
  
  [See: AlphaZero: How Intuition Demolished Logic (Medium)](https://medium.com/intuitionmachine/alphazero-how-intuition-demolished-logic-66a4841e6810)
  
  > **Comments:**
  > **username_1**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/105594/discussion-on-answer-by-anshuman-kumar-what-are-some-well-known-problems-where-n).  
  > **username_6**: Just an FYI: this link seems to have moved, or requires a login: https://stream.nyu.edu/channel/CS%2BFaculty%2BSearch%2B2019/111128891  
  
  ---
  
  ### Answer by username_1 on 2020-03-11. Score: 22
  
  In theory, most neural networks can approximate any continuous function on compact subsets of $\mathbb{R}^n$, provided that the activation functions satisfy certain mild conditions. This is known as the *universal approximation theorem* (UAT), but that should *not* be called *universal*, given that there are a lot more discontinuous functions than continuous ones, although certain discontinuous functions can be approximated by continuous ones. The UAT shows the theoretical powerfulness of neural networks and their purpose. They represent and approximate functions. If you want to know more about the details of the UAT, for different neural network architectures, see [this answer](https://ai.stackexchange.com/a/13319/2444).
  
  However, in practice, neural networks trained with gradient descent and backpropagation face several issues and challenges, some of which are due to the training procedure and not just the architecture of the neural network or available data.
  
  For example, it is well known that [neural networks are prone to catastrophic forgetting (or interference)](https://ai.stackexchange.com/a/13293/2444), which means that they aren't particularly suited for **incremental learning** tasks, although some more sophisticated incremental learning algorithms based on neural networks have already been developed.
  
  Neural networks can also be [sensitive to their inputs](https://arxiv.org/abs/1712.07107), i.e. a small change in the inputs can drastically change the output (or answer) of the neural network. This is partially due to the fact that they learn a function that isn't really the function you expect them to learn. So, a system based on such a neural network can potentially be hacked or fooled, so they are probably not well suited for **safety-critical applications**. This issue is related to the [low interpretability and explainability of neural networks](https://arxiv.org/pdf/2001.02522.pdf), i.e. they are often denoted as [black-box models](https://stats.stackexchange.com/q/93705/82135).
  
  [Bayesian neural networks (BNNs)](https://arxiv.org/abs/1505.05424) can potentially mitigate these problems, but they are unlikely to be the ultimate or complete solution. Bayesian neural networks maintain a distribution for each of the units (or neurons), rather than a point estimate. In principle, this can provide more uncertainty guarantees, but, in practice, this is not yet the case.
  
  Furthermore, neural networks often require a lot of data in order to approximate the desired function accurately, so in cases where **data is scarce** neural networks may not be appropriate. Moreover, the training of neural networks (especially, deep architectures) also **requires a lot of computational resources**. Inference can also be sometimes problematic, when you need real-time predictions, as it can also be expensive.
  
  To conclude, neural networks are just function approximators, i.e. they approximate a *specific* function (or set of functions, in the case of Bayesian neural networks), given a specific configuration of the parameters. They can't do more than that. They cannot magically do something that they have not been trained to do, and it is usually the case that you don't really know the specific function the neural network is representing (hence the expression *black-box model*), apart from knowing your training dataset, which can also contain spurious information, among other issues.
  
  > **Comments:**
  > **username_1**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/105593/discussion-on-answer-by-nbro-what-are-some-well-known-problems-where-neural-netw).  
  
  ---
  
  ### Answer by username_2 on 2020-03-12. Score: 17
  
  In our deep learning lecture, we discussed the following example (from [Unmasking Clever Hans predictors and assessing what machines really learn](https://www.nature.com/articles/s41467-019-08987-4) (2019) by Lapuschkin et al.).
  
  [![enter image description here](https://i.sstatic.net/cLggJ.jpg)](https://i.sstatic.net/cLggJ.jpg)
  
  Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong "relevant components". In the sensitivity maps next to the pictures, we can see that the watermark was used to identify if there is a horse present in the picture. If we remove the watermark, the classification is no longer made. Even more worryingly, if we add the tag to a completely different picture, it gets identified as a horse!
  
  > **Comments:**
  > **username_1**: This is similar to the famous [tank story](https://www.lesswrong.com/posts/5o3CxyvZ2XKawRB5w/machine-learning-and-unintended-consequences).  
  
  ---
  
  ### Answer by username_3 on 2020-03-12. Score: 5
  
  This is more in the direction of 'what kind of problems can be solved by neural networks'. In order to train a neural network you need a large set of training data which is labelled with correct/ incorrect for the question you are interested in. So for example 'identify all pictures that have a cat on them' is very suitable for neural networks. On the other hand 'summarize the story of this toddler picture book' is very hard. Although a human can easily decide whether a given summary is any good or not it would be very difficult to build a suitable set of training data for this kind of problem. So if you can't build a large training data set with correct answers, you can't train a neural network to solve the problem.
  
  The answer of Anshuman Kumar is also an instance of that, also a potentially solvable one. The neural network that misidentified upside-down school buses presumably had very few if any upside-down school buses in its training data. Put them into the training data and the neural network will identify these as well. This is still a flaw in neural networks, a human can correctly identify an upside-down school bus the first time they see one if they know what school busses look like.
  
  > **Comments:**
  > **username_0**: Good point.  Also using a feature map which rotates and stretches immages to create more "artificial" immage samples should bypass Anshuman's problem  
  > **username_4**: @AIM_BLB you'd need a feature map that can do so in 3D. So you pretty much need the whole pipeline the brain does: edge detection -> depth map -> segmentation -> 3D feature map -> classifier search (with each stage feeding back to the edge detection (!) stage and all later ones). And that's without any depth cues, meaning that the depth map can only take input from the classifier and the segmentation map.  
  
  ---
  
  ### Answer by username_5 on 2020-03-12. Score: 5
  
  I don't know if it might be of use, but many areas of NLP are still hard to tackle, and even if deep models achieve the state of the art results, they usually beat baseline shallow models by very few percentage points.
  One example that I've had the opportunity to work on is stance classification [1](https://www.aclweb.org/anthology/S16-1075.pdf). In many datasets, the best F score achievable is around 70%.
  
  Even though it's hard to compare results since in NLP many datasets are really small and domain-specific (especially for stance detection and similar SemEval tasks), many times SVM, conditional random fields, sometimes even Naive Bayes models are able to perform almost as good as CNN or RNN. Other tasks for which this holds are argumentation mining or claim detection.
  
  See e.g. the paper [TakeLab at SemEval-2016 Task 6: Stance Classification in Tweets Using a Genetic Algorithm Based Ensemble](https://www.aclweb.org/anthology/S16-1075.pdf) (2016) by Martin Tutek et al.
  
  > **Comments:**
  > **username_6**: Thanks for this answer.  I'm interested in this subject from the standpoint of (satisficing)[https://en.wikipedia.org/wiki/Satisficing]!  (i.e. if time or space is sufficiently restricted, the "less optimal" method may prevail.)  
  
  ---
  
  ### Answer by username_7 on 2020-04-15. Score: 4
  
  My 50cents:
  [NP\_(complexity)](https://en.wikipedia.org/wiki/NP_(complexity)) - is still hard to solve, even with NeuralNets.
  
  > In computational complexity theory, NP (nondeterministic polynomial
  > time) is a complexity class used to classify decision problems. NP is
  > the set of decision problems for which the problem instances, where
  > the answer is "yes", have proofs verifiable in polynomial time by a
  > deterministic Turing machine.
  
  The easiest example, to imagine what is speech about, it is cryptography's [Integer\_factorization](https://en.wikipedia.org/wiki/Integer_factorization), which is basement of [RSA cryptosystem](https://en.wikipedia.org/wiki/RSA_(cryptosystem)).
  
  For example, we have two simple numbers:
  
  * 12123123123123123123123.....45456
  * 23412421341234124124124.....11112
  
  NeuralNetwork shall answer us exactly digit to digit both this numbers, when we will show it only multiplication of this two numbers... This is not guessing about school bus. The field of numbers much more bigger than number of words in all languages on whole Earth. Imagine, that there are billions of billion different school buses, billions of billions of different fire-hydrants and billions of such classes, and NN shall answer exactly - what is on the picture - no way. The chance to guess is so little...
  
  
  ---
  
  ### Answer by username_8 on 2020-03-12. Score: 4
  
  A checkerboard with missing squares is impossible for a neural network to learn the missing color. The more it learns on training data, the worse it does on test data.
  
  See e.g. this article [The Unlearnable Checkerboard Pattern](https://journals.blythinstitute.org/ojs/index.php/cbi/article/view/48) (which, unfortunately, is not freely accessible). In any case, it should be easy to try out yourself that this task is difficult.
  
  > **Comments:**
  > **username_9**: Impossible is not the right word here; I'm pretty sure a neural network specially trained for identifying checkerboards will have no problem, but we'll have to see. Impossible is just too strong a word.  
  
  ---
  
  ### Answer by username_10 on 2020-03-23. Score: 3
  
  From my experience in industry, a lot of data science (operating on customer information, stored in a database) is still dominated by decision trees and even SVMs. Although neural networks have seen incredible performance on "unstructured" data, like images and text, there still do not appear to be great results extending to structured, tabular data (yet).
  
  At my old company (loyalty marketing with 10 million+ members) there was a saying, "*You can try any model you like, but you must try XGBoost*". And let's just say that I did try comparing it to a neural network, and ultimately I did go with XGBoost ;)
  
  > **Comments:**
  > **username_0**: This sounds very promising.  So something like Boston housing would be a toy example data-set of this type?  
  > **username_10**: Yes exactly! Same with the classic Adult dataset: https://archive.ics.uci.edu/ml/datasets/adult  
  > **username_0**: So in that case XGBoost is the best?  
  > **username_10**: Well it's usually difficult to say in advance, and especially before you do tinkering like hyperparameter sweeps, regularization etc. But in general, XGBoost gives you good results right out of the box, with the added bonus that it is also somewhat interpretable (such as giving you feature importances), allowing you to explain things in a business setting nicely  
  
  ---
  
  ### Answer by username_11 on 2020-03-14. Score: 3
  
  Neural networks seem to have a great deal of difficulty handling *adversarial input*, i.e., inputs with certain changes (often imperceptible or nearly imperceptible by humans) designed by an attacker to fool them.
  
  This is not the same thing as just being highly sensitive to certain changes in inputs. Robustness against wrong answers in that case can be increased by reducing the probability of such inputs. (If only one in 10^15 possible images causes a problem, it's not much of a problem.) However, in the adversarial case reducing the space of problematic images doesn't reduce the probability of getting one because the images are specifically selected by the attacker.
  
  One of the more famous papers in this area is ["Synthesizing Robust Adversarial Examples"](https://arxiv.org/pdf/1707.07397.pdf), which produced not only examples where a few modified pixels or other invisible-to-humans modifications to a picture fooled a neural network-based image classifier, but also perhaps the first examples of 3D objects designed to fool similar classifiers and successfully doing so (from every angle!).
  
  (Those familiar with IT securitity will no doubt recognise this as a familiar asymmetry: roughly, a defender must defend against *all* attacks launched against a system, but an attacker need find only *one* working attack.)
  
  In ["A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance"](https://arxiv.org/pdf/1901.10861.pdf), Adi Shamir et al. propose a mathematical framework for analyzing the problem based on Hamming distances that, while currently a less practical attack than the MIT/Lab6 one, has some pretty disturbing theoretical implications, including that current approaches to preventing these attacks may be, in the end, ineffective. For example, he points out that blurring and similar techniques that have been used to try to defend against adversarial attacks can be treated mathematically as simply another layer added on top of the existing neural network, requiring no changes to the attack strategy.
  
  (I attended a talk by Shamir a few months ago that was much easier going than the paper, but unfortunately I can't find a video of that or a similar talk on-line; if anybody knows of one please feel free to edit this answer to add a link!)
  
  There's obviously still an enormous amount of research to be done in this area, but it seems possible that neural networks alone are not capable of defense against this class of attack, and other techniques will have to be employed in addition to make neural networks robust against it.
  
  > **Comments:**
  > **username_12**: The attacks aren't quite the same if you apply random blurring. In principle adding random noise should provide sufficient defense, if your network on average handles noise well. I have seen attempts to use lipschitz like constraints but for deep networks the bounds are useless. It's kind of circular in a way - we want the network to function like humans do but there isn't a clear norm for how humans judge similarity between images.  
  
  ---
  
  ### Answer by username_13 on 2020-03-13. Score: 2
  
  Large scale route optimization problems.
  
  The is progress made in using Deep Reinforcement learning to solve vehicle routing problems (VRP), for example in this paper: <https://arxiv.org/abs/1802.04240v2>.
  
  However, for large scale problems and overall heuristic methods, like the ones provided by Google OR tools are much easier to use.
  
  > **Comments:**
  > **username_1**: You don't talk about neural networks in your answer, but only about deep RL. Maybe you can clarify how neural networks are related to your linked paper and the "large scale route optimization problems".  
  
  ---
  
  ### Answer by username_12 on 2020-03-13. Score: 1
  
  In the case of convolutional neural networks, the features may be extracted but without taking into account their relative positions (see the concept of *translation invariance*)
  
  For example, you could have two eyes, a nose and a mouth be in different locations in an image and still have the image be classified as a face.
  
  Operations like max-pooling may also have a negative impact on retaining position information.
  
  > **Comments:**
  > **username_1**: I have edited your answer to make it clearer. Please, can you provide a research work that proves that the extraction of features, in a CNN, can have a negative impact such that a collection of two eyes, a nose and a mouth, although not in the appropriate relative position, is classified as a face?  
  
  ---
  
  