Answers.Body: <p>A neural network is not good at selecting a function based on those 3 input parameters, because of the way a neuron is setup.</p>
  
  <p>What you should do is either make a neural network for each operation, or use different input neurons for each operation. E.g. 2 input neurons for the addition operation, 2 for the multiplication, and 2 for the minus. 6 inputs in total of which 4 will always be 0. </p>
  
  <p>This will make it easier for the neural network to calculate the result.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-13T22:52:05.847Z
Answers.Text: Thanks :) That seems to work to an extent. Now addition and subtraction work but multiplication doesn't work at all. Why exactly isn't a neural network not good at selecting a function the way I had it? Also any idea why multiplication wouldn't be working here?
Answers.UserDisplayName: KNejad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-16T08:14:18.647Z
Answers.Text: I looked into it and it seems a simple multiplication function is very hard to approximate for a neural network with certain activation funtions (such as relu). And this is only learning multiplication, not trying to do the other operations in the same network.
  
  link:https://www.sololearn.com/Discuss/1467062/can-a-neural-network-learn-multiplication
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-16T11:54:06.287Z
Answers.Text: Thanks Lustwelpintje, that's what I've discovered too. I'll continue looking into it and see if I can figure out a way to do it
Answers.UserDisplayName: KNejad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-13T12:39:50.940Z
Answers.LastActivityDate: 2019-09-23T07:56:46.567Z
Answers.LastEditDate: 2019-09-23T07:56:46.567Z
Answers.UserDisplayName: Lustwelpintje
Body: <p>I am playing around with neural networks in Tensorflow and I figured an interesting test would be whether I can write a calculator using a Tensorflow Neural Network.</p>
  
  <p>I started with simple addition and it kinda worked (so given 2, 4 it would get around 5.9 or 6.1).</p>
  
  <p>Then I wanted to add the ability to calculate using "+", "-", and "*".</p>
  
  <p>Here is the code I came up with in the end:</p>
  
  
  
  <pre><code>import numpy as np
  import tensorflow as tf
  from random import randrange
  
  def generate_input(size):
      nn_input = []
      for i in range(0,size):
          symbol = float(randrange(3))
          nn_input.append([
                  float(randrange(1000)),
                  float(randrange(1000)),
                  1 if symbol == 0 else 0,
                  1 if symbol == 1 else 0,
                  1 if symbol == 2 else 0,
                  ])
      return nn_input
  
  def generate_output(input_data):
      return [[generate_single_output(i)] for i in input_data]
  
  def generate_single_output(input_data):
      plus = input_data[2]
      minus = input_data[3]
      multiplication = input_data[4]
  
      if (plus):
          return input_data[0] + input_data[1]
  
      if (minus):
          return input_data[0] - input_data[1]
  
      if (multiplication):
          return input_data[0] * input_data[1]
  
  def user_input_to_nn_input(user_input):
      symbol = user_input[1]
      return np.array([[
              float(user_input[0]),
              float(user_input[2]),
              1 if symbol == '+' else 0,
              1 if symbol == '-' else 0,
              1 if symbol == '*' else 0,
              ]])
  
  
  if __name__ == '__main__':
      model = tf.keras.models.Sequential([
          tf.keras.layers.Dense(64, activation='relu', input_shape=(5,)),
          tf.keras.layers.Dense(64, activation='relu'),
          tf.keras.layers.Dense(1),
          ])
  
      model.compile(tf.keras.optimizers.RMSprop(0.001), loss=tf.keras.losses.MeanSquaredError())
  
  
      input_data = np.array(generate_input(10000))
      output_data = np.array(generate_output(input_data))
  
      model.fit(input_data, output_data, epochs=20)
  
      while True:
          user_calculation = input("Enter expression (e.g. 2 + 3):")
          user_input = user_calculation.split()
          nn_input = user_input_to_nn_input(user_input)
          print(model.predict(nn_input)[0][0])
  
  </code></pre>
  
  <p>The idea is built on this tutorial: <a href="https://www.tensorflow.org/tutorials/keras/basic_regression" rel="nofollow noreferrer">https://www.tensorflow.org/tutorials/keras/basic_regression</a></p>
  
  <p>The input is 5 fields: number 1, number 2, plus?, minus?, multiplication?</p>
  
  <p>Where the last 3 are simply 1 or 0 depending on whether that is the calculation I am trying to do.</p>
  
  <p>As an output for say [1,4,1,0,0] I would expect 1 + 4 = 5
  for [1,4,0,1,0] I would expect 1 - 4 = -3 etc.</p>
  
  <p>For some reason though the numbers I am getting are completely off and seem random.</p>
  
  <p>Basically I am trying to understand what I went wrong?
  The data being input to the NN seems correct and the model is based on the model used in the tutorial I quoted (and the problems seem fairly similar so I expect if one would work the other would too).</p>
  
Tags: <neural-networks><tensorflow>
Title: Why isn't my Neural Network based calculator working?
UserDisplayName: KNejad
ThreadText: # Why isn't my Neural Network based calculator working?
  
  (Asked by: username_0 on 2019-09-12)
  
  I am playing around with neural networks in Tensorflow and I figured an interesting test would be whether I can write a calculator using a Tensorflow Neural Network.
  I started with simple addition and it kinda worked (so given 2, 4 it would get around 5.9 or 6.1).
  Then I wanted to add the ability to calculate using "+", "-", and "*".
  Here is the code I came up with in the end:
  import numpy as np
  import tensorflow as tf
  from random import randrange
  
  def generate_input(size):
      nn_input = []
      for i in range(0,size):
          symbol = float(randrange(3))
          nn_input.append([
                  float(randrange(1000)),
                  float(randrange(1000)),
                  1 if symbol == 0 else 0,
                  1 if symbol == 1 else 0,
                  1 if symbol == 2 else 0,
                  ])
      return nn_input
  
  def generate_output(input_data):
      return [[generate_single_output(i)] for i in input_data]
  
  def generate_single_output(input_data):
      plus = input_data[2]
      minus = input_data[3]
      multiplication = input_data[4]
  
      if (plus):
          return input_data[0] + input_data[1]
  
      if (minus):
          return input_data[0] - input_data[1]
  
      if (multiplication):
          return input_data[0] * input_data[1]
  
  def user_input_to_nn_input(user_input):
      symbol = user_input[1]
      return np.array([[
              float(user_input[0]),
              float(user_input[2]),
              1 if symbol == '+' else 0,
              1 if symbol == '-' else 0,
              1 if symbol == '*' else 0,
              ]])
  
  
  if __name__ == '__main__':
      model = tf.keras.models.Sequential([
          tf.keras.layers.Dense(64, activation='relu', input_shape=(5,)),
          tf.keras.layers.Dense(64, activation='relu'),
          tf.keras.layers.Dense(1),
          ])
  
      model.compile(tf.keras.optimizers.RMSprop(0.001), loss=tf.keras.losses.MeanSquaredError())
  
  
      input_data = np.array(generate_input(10000))
      output_data = np.array(generate_output(input_data))
  
      model.fit(input_data, output_data, epochs=20)
  
      while True:
          user_calculation = input("Enter expression (e.g. 2 + 3):")
          user_input = user_calculation.split()
          nn_input = user_input_to_nn_input(user_input)
          print(model.predict(nn_input)[0][0])
  
  
  The idea is built on this tutorial: https://www.tensorflow.org/tutorials/keras/basic_regression
  The input is 5 fields: number 1, number 2, plus?, minus?, multiplication?
  Where the last 3 are simply 1 or 0 depending on whether that is the calculation I am trying to do.
  As an output for say [1,4,1,0,0] I would expect 1 + 4 = 5
  for [1,4,0,1,0] I would expect 1 - 4 = -3 etc.
  For some reason though the numbers I am getting are completely off and seem random.
  Basically I am trying to understand what I went wrong?
  The data being input to the NN seems correct and the model is based on the model used in the tutorial I quoted (and the problems seem fairly similar so I expect if one would work the other would too).
  
  **Tags:** neural-networks, tensorflow
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-09-13. Score: 6 | Accepted answer
  
  A neural network is not good at selecting a function based on those 3 input parameters, because of the way a neuron is setup.
  
  What you should do is either make a neural network for each operation, or use different input neurons for each operation. E.g. 2 input neurons for the addition operation, 2 for the multiplication, and 2 for the minus. 6 inputs in total of which 4 will always be 0.
  
  This will make it easier for the neural network to calculate the result.
  
  > **Comments:**
  > **username_0**: Thanks :) That seems to work to an extent. Now addition and subtraction work but multiplication doesn't work at all. Why exactly isn't a neural network not good at selecting a function the way I had it? Also any idea why multiplication wouldn't be working here?  
  > **username_1**: I looked into it and it seems a simple multiplication function is very hard to approximate for a neural network with certain activation funtions (such as relu). And this is only learning multiplication, not trying to do the other operations in the same network.  link:https://www.sololearn.com/Discuss/1467062/can-a-neural-network-learn-multiplication  
  > **username_0**: Thanks Lustwelpintje, that's what I've discovered too. I'll continue looking into it and see if I can figure out a way to do it  
  
  ---
  
  