Answers.Body: <p>Not really.</p>
  <p>Neural networks are good for determining non-linear relationships between inputs when there are hidden variables. In the examples above, the relationships are linear, and there are no hidden variables. But even if they were non-linear, a traditional ANN design would not be well suited to accomplish this.</p>
  <p>By carefully constructing the layers and tightly supervising the training, you could get a network to consistently produce the output 4.01, say, for the inputs: 2, 1 (+), and 2, but this is not only wrong, it's an inherently unreliable application of the technology.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-20T15:30:29.253Z
Answers.Text: Well, that's probably because they are universal approximators. They'll never get an exact value for anything and we actually don't want NNs to predict an exact value (that would be over fitting). So in a way, they are working the way they designed to work.
Answers.UserDisplayName: Sarvagya Gupta
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-08T12:03:08.500Z
Answers.Text: This answer seems to be wrong. Even without reading the other answers, I know of models that have been used to solve simple addition and multiplication problems, as mentioned in the question (e.g. 2+2). So, you may want to revise/update this answer.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-26T17:42:47.750Z
Answers.Text: This answer is wrong. Or a few decades out of date. Or based on an ELI5 understanding of ML. Why would the model output a floating point number? A language model can return probabilities for tokens like "4". A graph search model can return probabilities for each edge in the proof space that should be explored. Returning "4.01" is, for lack of a better word, retarded.
Answers.UserDisplayName: Navin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-08-02T21:57:56.160Z
Answers.LastActivityDate: 2021-10-08T11:57:57.370Z
Answers.LastEditDate: 2021-10-08T11:57:57.370Z
Answers.UserDisplayName: dynrepsys
Answers.Body: <ol>
  <li><p>It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out <a href="https://github.com/keras-team/keras-io/blob/master/examples/nlp/addition_rnn.py" rel="nofollow noreferrer">this link to see the source code</a>.</p>
  </li>
  <li><p>This particular example uses a recurrent neural network (RNN) to process the problem as a sequence of characters, producing a sequence of characters which form the answer. Note that this approach is obviously different from how humans tend to think about solving simple addition problems, and probably isn't how you would ever want a computer to solve such a problem. Mostly this is an example of sequence to sequence learning using Keras. When handling sequential or time-series inputs, RNNs are a popular choice.</p>
  </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-03-09T17:01:54.670Z
Answers.LastActivityDate: 2021-09-30T13:14:57.730Z
Answers.LastEditDate: 2021-09-30T13:14:57.730Z
Answers.UserDisplayName: user3390629
Answers.Body: <p>There's the fairly well established field of <a href="https://en.wikipedia.org/wiki/Automated_theorem_proving" rel="nofollow noreferrer">automated theorem proving</a>. This most likely encompasses solving equations, but doesn't necessarily involve AI. <a href="https://stats.stackexchange.com/questions/262390/application-of-machine-learning-to-the-automated-theorem-proving">This post from the Cross Validated stackexchange</a> has some more information on the topic.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T00:22:23.530Z
Answers.LastActivityDate: 2018-11-30T00:22:23.530Z
Answers.UserDisplayName: sfmiller940
Answers.Body: <p>Yes - it would seem that it is now possible to achieve more is required from the example you've given this paper describes a DL solution to a considerably harder problem - <a href="http://arxiv.org/pdf/1510.07211.pdf" rel="nofollow">generating the source code for a program described in natural language</a>.</p>
  
  <p>Both of these can be described as regression problems (i.e. the goal is to minimize some loss function on the validation set), but the search space in the natural language case is much bigger.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-07T12:21:12.527Z
Answers.LastActivityDate: 2016-08-07T12:21:12.527Z
Answers.UserDisplayName: NietzscheanAI
Answers.Body: <p>Yes, it is possible, you would be training a network with 3 inputs and 1 output to learn the model <span class="math-container">$Y= X_1 + X_2 + X_3$</span> , although I don't see what would be the use of doing that. Furthermore, how do you plan to generalize to learn to add four or five terms?</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-25T14:10:55.127Z
Answers.LastActivityDate: 2022-10-25T14:10:55.127Z
Answers.UserDisplayName: Ken Grimes
Answers.Body: <p>Yes, it has been done!</p>
  <p>However, the applications aren't to replace calculators or anything like that. The lab I'm associated with develops neural network models of equational reasoning to better understand how humans might solve these problems. This is a part of the field known as <a href="https://web.archive.org/web/20201111232708/http://archive.is/mt4RO" rel="nofollow noreferrer">Mathematical Cognition</a>. Unfortunately, our website isn't terribly informative, but here's a <a href="https://web.archive.org/web/20200331161444/http://web.stanford.edu/%7Ekmickey/pdf/MickeyMcClelland2014.pdf" rel="nofollow noreferrer">link</a> to an example of such work.</p>
  <p>Apart from that, recent work on extending neural networks to include external memory stores (e.g. Neural Turing Machines) was used to solve math problems as a good proof of concept. This is because many arithmetic problems involve long procedures with stored intermediate results. See the sections of <a href="http://arxiv.org/pdf/1511.08228.pdf" rel="nofollow noreferrer">this paper</a> on long binary addition and multiplication.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-10T13:46:18.490Z
Answers.Text: Architectures like NTM and DNC can be used to solve problems, like the shortest path, because they do have the ability to execute the iterative process by keeping track of what was being done (no catastrophic forgetting). But for me using just supervised learning is simply wrong as mentioned in the second answer.
Answers.UserDisplayName: Shamane Siriwardhana
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-08-02T22:50:52.560Z
Answers.LastActivityDate: 2021-10-08T11:55:40.857Z
Answers.LastEditDate: 2021-10-08T11:55:40.857Z
Answers.UserDisplayName: zergylord
Body: <p>I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations?</p>
  
  <p>So given the 3 inputs: 1st number, operator sign represented by the number (1 - <code>+</code>, 2 - <code>-</code>, 3 - <code>/</code>, 4 - <code>*</code>, and so on), and the 2nd number, then after training the network should give me the valid results.</p>
  
  <p>Example 1 (<code>2+2</code>):</p>
  
  <ul>
  <li>Input 1: <code>2</code>; Input 2: <code>1</code> (<code>+</code>); Input 3: <code>2</code>; Expected output: <code>4</code></li>
  <li>Input 1: <code>10</code>; Input 2: <code>2</code> (<code>-</code>); Input 3: <code>10</code>; Expected output: <code>0</code></li>
  <li>Input 1: <code>5</code>; Input 2: <code>4</code> (<code>*</code>); Input 3: <code>5</code>; Expected output: <code>25</code></li>
  <li>and so</li>
  </ul>
  
  <p>The above can be extended to more sophisticated examples.</p>
  
  <p>Is that possible? If so, what kind of network can learn/achieve that?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-04T12:10:36.920Z
Comments.Text: A related video [NeuroSAT: An AI That Learned Solving Logic Problems](https://www.youtube.com/watch?v=luwP75lPExo).
Comments.UserDisplayName: kenorb
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-19T13:49:04.327Z
Comments.Text: This might be relevant - https://github.com/deepmind/mathematics_dataset
Comments.UserDisplayName: Ganesh Tata
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-26T05:50:18.393Z
Comments.Text: Take a look here: [https://ai.facebook.com/blog/using-neural-networks-to-solve-advanced-mathematics-equations/](https://ai.facebook.com/blog/using-neural-networks-to-solve-advanced-mathematics-equations/).
Comments.UserDisplayName: mip
Tags: <neural-networks><math><applications>
Title: Is it possible to train the neural network to solve math equations?
UserDisplayName: kenorb
ThreadText: # Is it possible to train the neural network to solve math equations?
  
  (Asked by: username_0 on 2016-08-02)
  
  I'm aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations?
  
  So given the 3 inputs: 1st number, operator sign represented by the number (1 - `+`, 2 - `-`, 3 - `/`, 4 - `*`, and so on), and the 2nd number, then after training the network should give me the valid results.
  
  Example 1 (`2+2`):
  
  * Input 1: `2`; Input 2: `1` (`+`); Input 3: `2`; Expected output: `4`
  * Input 1: `10`; Input 2: `2` (`-`); Input 3: `10`; Expected output: `0`
  * Input 1: `5`; Input 2: `4` (`*`); Input 3: `5`; Expected output: `25`
  * and so
  
  The above can be extended to more sophisticated examples.
  
  Is that possible? If so, what kind of network can learn/achieve that?
  
  **Tags:** neural-networks, math, applications
  > **Comments:**
  > **username_0**: A related video [NeuroSAT: An AI That Learned Solving Logic Problems](https://www.youtube.com/watch?v=luwP75lPExo).  
  > **username_1**: This might be relevant - https://github.com/deepmind/mathematics_dataset  
  > **username_2**: Take a look here: [https://ai.facebook.com/blog/using-neural-networks-to-solve-advanced-mathematics-equations/](https://ai.facebook.com/blog/using-neural-networks-to-solve-advanced-mathematics-equations/).  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_11 on 2016-08-02. Score: 24 | Accepted answer
  
  Yes, it has been done!
  
  However, the applications aren't to replace calculators or anything like that. The lab I'm associated with develops neural network models of equational reasoning to better understand how humans might solve these problems. This is a part of the field known as [Mathematical Cognition](https://web.archive.org/web/20201111232708/http://archive.is/mt4RO). Unfortunately, our website isn't terribly informative, but here's a [link](https://web.archive.org/web/20200331161444/http://web.stanford.edu/%7Ekmickey/pdf/MickeyMcClelland2014.pdf) to an example of such work.
  
  Apart from that, recent work on extending neural networks to include external memory stores (e.g. Neural Turing Machines) was used to solve math problems as a good proof of concept. This is because many arithmetic problems involve long procedures with stored intermediate results. See the sections of [this paper](http://arxiv.org/pdf/1511.08228.pdf) on long binary addition and multiplication.
  
  > **Comments:**
  > **username_12**: Architectures like NTM and DNC can be used to solve problems, like the shortest path, because they do have the ability to execute the iterative process by keeping track of what was being done (no catastrophic forgetting). But for me using just supervised learning is simply wrong as mentioned in the second answer.  
  
  ---
  
  ### Answer by username_3 on 2016-08-02. Score: 9
  
  Not really.
  
  Neural networks are good for determining non-linear relationships between inputs when there are hidden variables. In the examples above, the relationships are linear, and there are no hidden variables. But even if they were non-linear, a traditional ANN design would not be well suited to accomplish this.
  
  By carefully constructing the layers and tightly supervising the training, you could get a network to consistently produce the output 4.01, say, for the inputs: 2, 1 (+), and 2, but this is not only wrong, it's an inherently unreliable application of the technology.
  
  > **Comments:**
  > **username_4**: Well, that's probably because they are universal approximators. They'll never get an exact value for anything and we actually don't want NNs to predict an exact value (that would be over fitting). So in a way, they are working the way they designed to work.  
  > **username_5**: This answer seems to be wrong. Even without reading the other answers, I know of models that have been used to solve simple addition and multiplication problems, as mentioned in the question (e.g. 2+2). So, you may want to revise/update this answer.  
  > **username_6**: This answer is wrong. Or a few decades out of date. Or based on an ELI5 understanding of ML. Why would the model output a floating point number? A language model can return probabilities for tokens like "4". A graph search model can return probabilities for each edge in the proof space that should be explored. Returning "4.01" is, for lack of a better word, retarded.  
  
  ---
  
  ### Answer by username_7 on 2017-03-09. Score: 8
  
  1. It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out [this link to see the source code](https://github.com/keras-team/keras-io/blob/master/examples/nlp/addition_rnn.py).
  2. This particular example uses a recurrent neural network (RNN) to process the problem as a sequence of characters, producing a sequence of characters which form the answer. Note that this approach is obviously different from how humans tend to think about solving simple addition problems, and probably isn't how you would ever want a computer to solve such a problem. Mostly this is an example of sequence to sequence learning using Keras. When handling sequential or time-series inputs, RNNs are a popular choice.
  
  
  ---
  
  ### Answer by username_8 on 2018-11-30. Score: 2
  
  There's the fairly well established field of [automated theorem proving](https://en.wikipedia.org/wiki/Automated_theorem_proving). This most likely encompasses solving equations, but doesn't necessarily involve AI. [This post from the Cross Validated stackexchange](https://stats.stackexchange.com/questions/262390/application-of-machine-learning-to-the-automated-theorem-proving) has some more information on the topic.
  
  
  ---
  
  ### Answer by username_9 on 2016-08-07. Score: 2
  
  Yes - it would seem that it is now possible to achieve more is required from the example you've given this paper describes a DL solution to a considerably harder problem - [generating the source code for a program described in natural language](http://arxiv.org/pdf/1510.07211.pdf).
  
  Both of these can be described as regression problems (i.e. the goal is to minimize some loss function on the validation set), but the search space in the natural language case is much bigger.
  
  
  ---
  
  ### Answer by username_10 on 2022-10-25. Score: 0
  
  Yes, it is possible, you would be training a network with 3 inputs and 1 output to learn the model $Y= X\_1 + X\_2 + X\_3$ , although I don't see what would be the use of doing that. Furthermore, how do you plan to generalize to learn to add four or five terms?
  
  
  ---
  
  