Answers.Body: <p>I don't know of any neural network that can do cryptography well, so you would have to do a little experimenting yourself. The main thing that sticks out to me is that doing operations in the elliptical curve requires the modulus operator since it works in finite field, and I think neural networks have a hard time learning the modulus operator in general. So I would focus on that first. Some things to help the network learn the modulus operator:</p>
  <ul>
  <li>I would try to increase the hidden layers to a number 4x-10x bigger than the input dimension, which maps it to a higher dimension to hopefully learn more complex behavior.</li>
  <li>I would use less layers, maybe only 2-4 hidden layers, to speed up the development time.</li>
  <li>Most importantly, I would train with a LOT of data points (25% of the total possible finite field possibilities). I don't think the neural network can learn unless the number of data is this high.</li>
  <li>For reference, <a href="http://stackoverflow.com/a/62095681/3667125">someone</a> got a network to learn the modulus operator using these points.</li>
  </ul>
  <p>For rapid iterating, I would test with a much smaller finite field. E.g. use a 8 bit security and see if the neural network can do well with that first before moving on to the full 256-bit security key (or whatever your end goal is).</p>
  <p>Taking this a step further, I would first test to see if the neural network can even perform a point addition in the elliptical curve well, because if it doesn't then it definitely can't do point multiplication which is needed to compute the public key.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-25T22:37:11.967Z
Answers.LastActivityDate: 2020-12-25T22:37:11.967Z
Answers.UserDisplayName: user3667125
Body: <p>As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression.</p>
  <p>I was interested in trying a harder problem and have been working on a network that, given a private key, could perform point multiplication along ECC curve to obtain a public key.</p>
  <p>I have tried training on a dataset of generated keypairs, but am seeing the test loss values bounce around like crazy with train loss values eventually decreasing after many epochs due to what I assume is overfitting.</p>
  <p>Is this public key generation problem even solvable with a deep learning architecture? If so, am I doing something wrong with my current approach?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-21T21:36:47.923Z
Comments.Text: Can you explain further what you mean by ResNets with Dense layers? What are the hidden sizes, # of layers, activation function, etc?
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-21T22:03:04.733Z
Comments.Text: Also, to clarify, the network was able to predict the training instances correctly right? Only the test instances have poor accuracy? What is the training accuracy?
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-24T00:43:19.923Z
Comments.Text: @user3667125 28 blocks of layers. Layers are 32 wide, and there is a skip connection every 2 layers. ReLU activation function. I am dealing with 256 byte input and 264 byte output, so I just pass these in as byte arrays.
Comments.UserDisplayName: superuser
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-24T00:44:03.370Z
Comments.Text: @user3667125 The problem is that currently the model just gives a value near 128 for every byte, since loss function is MSE.
Comments.UserDisplayName: superuser
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-24T00:45:03.267Z
Comments.Text: @user3667125 Is a residual network capable to learn to do problems like these which require understanding a geometrical abstraction such as point addition on a line?
Comments.UserDisplayName: superuser
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-24T04:50:39.590Z
Comments.Text: In general neural networks has difficulty learning the modulus operator, which is necessary for cryptography since they use finite fields. I would try to increase the hidden layers to a number bigger than the input dimension (maybe 2048), which maps to a higher dimension to hopefully learn more complex behavior. Use less layers, maybe only 2 or 4 hidden layers. I would also train with a LOT of datapoints (25% of the total possible finite field possibilities). I don't think the neural network can learn unless # of data is this high. For reference: https://stackoverflow.com/a/62095681/3667125
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-24T05:00:20.627Z
Comments.Text: I don't know of any neural network that can do cryptography well, so you would have to experiment with features like the above I listed. If it works, then grats! It might be a publish-worthy paper or at least a blog post
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-24T05:06:14.013Z
Comments.Text: For rapid iterating, I would test with a much smaller finite field. E.g. use a 8 bit security and see if the neural network can do well with that first before moving on to the full 256-bit security key (or whatever your end goal is). Taking this a step further, I would first test to see if the neural network can even perform a point addition in the elliptical curve well, because if it doesn't then it definitely can't do point multiplication which is needed to compute the public key.
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-25T19:20:21.793Z
Comments.Text: @user3667125 Ok, thanks for the ideas. I actually was thinking the same things haha. I 've been working on a simpler point addition network and was also playing around with modulus operations and actually just writing a simple network to compute x raised to a constant and mod some other constant. Will keep you updated! If you want to post an answer I'll give you the bounty.
Comments.UserDisplayName: superuser
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-25T22:38:44.490Z
Comments.Text: For sure, keep us updated! I'm sure there are others like me that would like to see the extent of the predictive power of neural networks
Comments.UserDisplayName: user3667125
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-28T23:16:44.140Z
Comments.Text: The OP is only asking if a neural net can compute the forward pass of a trapdoor function, not its inverse. It is trivial to think of an example ML model that can do this, e.g. the forward pass of a Rabin cryptography is simply the product of 2 primes and a modulo operator. So have one network learn the multiplication operator, and one network learn the modulus operator, and stack the 2 networks together, it would learn the forward pass of the Rabin cryptography. The forward pass of an elliptical curve public key is more complicated, but not too different, so its interesting to see if it work.
Comments.UserDisplayName: user3667125
Tags: <deep-learning><deep-neural-networks><residual-networks>
Title: Regression For Elliptical Curve Public Key Generation Possible?
UserDisplayName: superuser
ThreadText: # Regression For Elliptical Curve Public Key Generation Possible?
  
  (Asked by: username_0 on 2020-12-18)
  
  As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression.
  
  I was interested in trying a harder problem and have been working on a network that, given a private key, could perform point multiplication along ECC curve to obtain a public key.
  
  I have tried training on a dataset of generated keypairs, but am seeing the test loss values bounce around like crazy with train loss values eventually decreasing after many epochs due to what I assume is overfitting.
  
  Is this public key generation problem even solvable with a deep learning architecture? If so, am I doing something wrong with my current approach?
  
  **Tags:** deep-learning, deep-neural-networks, residual-networks
  
  > **Comments:**
  > **username_1**: Can you explain further what you mean by ResNets with Dense layers? What are the hidden sizes, # of layers, activation function, etc?  
  > **username_1**: Also, to clarify, the network was able to predict the training instances correctly right? Only the test instances have poor accuracy? What is the training accuracy?  
  > **username_0**: @username_1 28 blocks of layers. Layers are 32 wide, and there is a skip connection every 2 layers. ReLU activation function. I am dealing with 256 byte input and 264 byte output, so I just pass these in as byte arrays.  
  > **username_0**: @username_1 The problem is that currently the model just gives a value near 128 for every byte, since loss function is MSE.  
  > **username_0**: @username_1 Is a residual network capable to learn to do problems like these which require understanding a geometrical abstraction such as point addition on a line?  
  > **username_1**: In general neural networks has difficulty learning the modulus operator, which is necessary for cryptography since they use finite fields. I would try to increase the hidden layers to a number bigger than the input dimension (maybe 2048), which maps to a higher dimension to hopefully learn more complex behavior. Use less layers, maybe only 2 or 4 hidden layers. I would also train with a LOT of datapoints (25% of the total possible finite field possibilities). I don't think the neural network can learn unless # of data is this high. For reference: https://stackoverflow.com/a/62095681/3667125  
  > **username_1**: I don't know of any neural network that can do cryptography well, so you would have to experiment with features like the above I listed. If it works, then grats! It might be a publish-worthy paper or at least a blog post  
  > **username_1**: For rapid iterating, I would test with a much smaller finite field. E.g. use a 8 bit security and see if the neural network can do well with that first before moving on to the full 256-bit security key (or whatever your end goal is). Taking this a step further, I would first test to see if the neural network can even perform a point addition in the elliptical curve well, because if it doesn't then it definitely can't do point multiplication which is needed to compute the public key.  
  > **username_0**: @username_1 Ok, thanks for the ideas. I actually was thinking the same things haha. I 've been working on a simpler point addition network and was also playing around with modulus operations and actually just writing a simple network to compute x raised to a constant and mod some other constant. Will keep you updated! If you want to post an answer I'll give you the bounty.  
  > **username_1**: For sure, keep us updated! I'm sure there are others like me that would like to see the extent of the predictive power of neural networks  
  > **username_1**: The OP is only asking if a neural net can compute the forward pass of a trapdoor function, not its inverse. It is trivial to think of an example ML model that can do this, e.g. the forward pass of a Rabin cryptography is simply the product of 2 primes and a modulo operator. So have one network learn the multiplication operator, and one network learn the modulus operator, and stack the 2 networks together, it would learn the forward pass of the Rabin cryptography. The forward pass of an elliptical curve public key is more complicated, but not too different, so its interesting to see if it work.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-12-25. Score: 0 | Accepted answer
  
  I don't know of any neural network that can do cryptography well, so you would have to do a little experimenting yourself. The main thing that sticks out to me is that doing operations in the elliptical curve requires the modulus operator since it works in finite field, and I think neural networks have a hard time learning the modulus operator in general. So I would focus on that first. Some things to help the network learn the modulus operator:
  
  * I would try to increase the hidden layers to a number 4x-10x bigger than the input dimension, which maps it to a higher dimension to hopefully learn more complex behavior.
  * I would use less layers, maybe only 2-4 hidden layers, to speed up the development time.
  * Most importantly, I would train with a LOT of data points (25% of the total possible finite field possibilities). I don't think the neural network can learn unless the number of data is this high.
  * For reference, [someone](http://stackoverflow.com/a/62095681/3667125) got a network to learn the modulus operator using these points.
  
  For rapid iterating, I would test with a much smaller finite field. E.g. use a 8 bit security and see if the neural network can do well with that first before moving on to the full 256-bit security key (or whatever your end goal is).
  
  Taking this a step further, I would first test to see if the neural network can even perform a point addition in the elliptical curve well, because if it doesn't then it definitely can't do point multiplication which is needed to compute the public key.
  
  
  ---
  
  