Answers.Body: <p>When you represent a number in base 2 (binary), you have already divided the number by 2 many times.  If there is no remainder at the end, the number is obviously evenly divisible by 2.  This hints that your AI could test for divisibility by dividing.  Hmm-- not much gained there!  </p>
  
  <p>Unfortunately the problem is not one suited to solving via AI.  That's why factorization of large numbers is a good basis for hard encryption schemes.  I'd suggest finding a different sort of problem on which to test your AI.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-12T03:10:05.077Z
Answers.LastActivityDate: 2019-12-12T03:10:05.077Z
Answers.UserDisplayName: S. McGrew
Answers.Body: <p>There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as well as LSTMs (RNNs which are Turing-complete) are not able to model simple arithmetic operations (e.g. addition/multiplication), they designed a new logic unit which can solve all of the simple arithmetic problems.</p>
  
  <p>See: <a href="https://arxiv.org/abs/1808.00508" rel="noreferrer">Neural Arithmetic Logic Units</a></p>
  
  <p>More recently, DL can solve symbolic maths: <a href="https://arxiv.org/abs/1912.01412" rel="noreferrer">Deep Learning for Symbolic Mathematics</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T20:31:56.200Z
Answers.LastActivityDate: 2020-03-01T20:31:56.200Z
Answers.UserDisplayName: Anuar Y
Body: <p>So I’ve been working on my own little dynamic architecture for a deep neural network (any number of hidden layers with any number of nodes in every layer) and got it solving the XOR problem efficiently. I moved on to trying to see if I could train my network on how to classify a number as being divisible by another number or not while experimenting with different network structures and have noticed some odd things. I know this is a weird thing to try and train a neural network to do but I just thought it might be easy because I can simply generate the training data set and test data set programmatically.</p>
  
  <p>From what I’ve tested, it seems that my network is only really good at identifying whether or not a number is divisible by a number who is a power of 2. If you test divisibility by a power of two, it converges on a very good solution very quickly. And it generalizes well on numbers outside of the training set - which I guess it kind of makes sense, as I’m inputting the numbers into the network in binary representation, so all the network has to learn is that a number n is only divisible by 2^m if the last m digits in the binary input vector are 0 (i.e. fire the output neuron if the last m neurons on the input layer don't fire, else don't). When checking divisibility by non-powers of two, however, there does not seem to be as much of a "positional" (maybe that's the word, maybe not) relationship between the input bits and whether or not the number is divisible.  I thought though, that if I threw more neurons and layers at the problem that it might be able to solve classifying divisibility by other numbers – but that is not the case. The network seems to converge on not-so-optimal local minima on the cost function (for which I am using mean-squared-error) when dividing by numbers that are not powers of 2. I’ve tried different learning rates as well to no avail. </p>
  
  <p>Do you have any idea what would cause something like this or how to go about trying to fix it? Or are plain deep neural networks maybe just not good at solving these types of problems?</p>
  
  <p>Note: I should also add that I've tried using different activation functions for different layers (like having leaky-relu activation for your first hidden layer, then sigmoid activation for your output layer, etc.) which has also not seem to have made a difference</p>
  
  <p>Here is my code if you feel so inclined as to look at it: <a href="https://github.com/bigstronkcodeman/Deep-Neural-Network/blob/master/Neural.py" rel="noreferrer">https://github.com/bigstronkcodeman/Deep-Neural-Network/blob/master/Neural.py</a></p>
  
  <p>(beware: it was all written from scratch by me in the quest to learn so some parts (namely the back-propagation) are not very pretty - I am really new to this whole neural network thing)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-12T03:06:43.930Z
Comments.Text: When you represent a number in base 2 (binary), you have already divided the number by 2 many times.  If there is no remainder at the end, the number is obviously evenly divisible by 2.  This hints that your AI could test for divisibility by dividing.  Hmm-- nothing gained there!
Comments.UserDisplayName: S. McGrew
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-09-26T21:49:42.317Z
Comments.Text: My two cents on this problem: I believe a step that makes factorization difficult is the usual mod b representation. For instance, if we say that 9 is 121 (in "binary", 1x2^2+2x2^1+1x2^0) instead of the usual 1001 it may be easier to find that its factors are 11 and 11.
Comments.UserDisplayName: Jaume Oliver Lafont
Tags: <neural-networks><machine-learning><classification>
Title: Can a deep neural network be trained to classify an integer N1 as being divisible by another integer N2?
UserDisplayName: bigphil
ThreadText: # Can a deep neural network be trained to classify an integer N1 as being divisible by another integer N2?
  
  (Asked by: username_0 on 2019-12-11)
  
  So I’ve been working on my own little dynamic architecture for a deep neural network (any number of hidden layers with any number of nodes in every layer) and got it solving the XOR problem efficiently. I moved on to trying to see if I could train my network on how to classify a number as being divisible by another number or not while experimenting with different network structures and have noticed some odd things. I know this is a weird thing to try and train a neural network to do but I just thought it might be easy because I can simply generate the training data set and test data set programmatically.
  
  From what I’ve tested, it seems that my network is only really good at identifying whether or not a number is divisible by a number who is a power of 2. If you test divisibility by a power of two, it converges on a very good solution very quickly. And it generalizes well on numbers outside of the training set - which I guess it kind of makes sense, as I’m inputting the numbers into the network in binary representation, so all the network has to learn is that a number n is only divisible by 2^m if the last m digits in the binary input vector are 0 (i.e. fire the output neuron if the last m neurons on the input layer don't fire, else don't). When checking divisibility by non-powers of two, however, there does not seem to be as much of a "positional" (maybe that's the word, maybe not) relationship between the input bits and whether or not the number is divisible. I thought though, that if I threw more neurons and layers at the problem that it might be able to solve classifying divisibility by other numbers – but that is not the case. The network seems to converge on not-so-optimal local minima on the cost function (for which I am using mean-squared-error) when dividing by numbers that are not powers of 2. I’ve tried different learning rates as well to no avail.
  
  Do you have any idea what would cause something like this or how to go about trying to fix it? Or are plain deep neural networks maybe just not good at solving these types of problems?
  
  Note: I should also add that I've tried using different activation functions for different layers (like having leaky-relu activation for your first hidden layer, then sigmoid activation for your output layer, etc.) which has also not seem to have made a difference
  
  Here is my code if you feel so inclined as to look at it: <https://github.com/bigstronkcodeman/Deep-Neural-Network/blob/master/Neural.py>
  
  (beware: it was all written from scratch by me in the quest to learn so some parts (namely the back-propagation) are not very pretty - I am really new to this whole neural network thing)
  
  **Tags:** neural-networks, machine-learning, classification
  
  > **Comments:**
  > **username_1**: When you represent a number in base 2 (binary), you have already divided the number by 2 many times.  If there is no remainder at the end, the number is obviously evenly divisible by 2.  This hints that your AI could test for divisibility by dividing.  Hmm-- nothing gained there!  
  > **username_2**: My two cents on this problem: I believe a step that makes factorization difficult is the usual mod b representation. For instance, if we say that 9 is 121 (in "binary", 1x2^2+2x2^1+1x2^0) instead of the usual 1001 it may be easier to find that its factors are 11 and 11.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2020-03-01. Score: 5 | Accepted answer
  
  There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as well as LSTMs (RNNs which are Turing-complete) are not able to model simple arithmetic operations (e.g. addition/multiplication), they designed a new logic unit which can solve all of the simple arithmetic problems.
  
  See: [Neural Arithmetic Logic Units](https://arxiv.org/abs/1808.00508)
  
  More recently, DL can solve symbolic maths: [Deep Learning for Symbolic Mathematics](https://arxiv.org/abs/1912.01412)
  
  
  ---
  
  ### Answer by username_1 on 2019-12-12. Score: 1
  
  When you represent a number in base 2 (binary), you have already divided the number by 2 many times. If there is no remainder at the end, the number is obviously evenly divisible by 2. This hints that your AI could test for divisibility by dividing. Hmm-- not much gained there!
  
  Unfortunately the problem is not one suited to solving via AI. That's why factorization of large numbers is a good basis for hard encryption schemes. I'd suggest finding a different sort of problem on which to test your AI.
  
  
  ---
  
  