Body: <p>Considering I have some neural network that, using supervised learning, transforms a string into a learned feature vector where "close" strings will result into more close vectors.</p>
  
  <p>I know that since a NN is no one-way-function there is a way to retrieve the input data from my output if I have the entire network at hand (if I know biases, weights, etc.)</p>
  
  <p>My question is if the network is not known, is there a way for me (using ie. some probabilistic distributions) to make assumptions or even reconstruct the input data?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-05T15:06:10.390Z
Comments.Text: even if you have weights and biases, unless the network falls into some reversable scheme (most used in practice do not) then you still wont be able to reconstruct the input
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-05T19:36:39.370Z
Comments.Text: In principle, if an attacker can query the network as often as they want, they can simply build a dataset of encodings and sources and learn a decoder, that should get pretty good. I also remember that there was a paper a few years ago, that could even reconstruct a model by querying it enough. They noted, that this could for example be used to reconstruct recommendation engines from amazon. 
  However, if the network is trained in a specific way (like concatenating a random key to the input), it might be possible.
Comments.UserDisplayName: Luca Thiede
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-05T19:36:43.343Z
Comments.Text: Beside that, there exists a wikipedia article concerned with this question: https://en.wikipedia.org/wiki/Neural_cryptography
Comments.UserDisplayName: Luca Thiede
Tags: <neural-networks><machine-learning>
Title: Can learned feature vectors be considered a good encryption?
UserDisplayName: Jonathan R
ThreadText: # Can learned feature vectors be considered a good encryption?
  
  (Asked by: username_0 on 2019-12-03)
  
  Considering I have some neural network that, using supervised learning, transforms a string into a learned feature vector where "close" strings will result into more close vectors.
  
  I know that since a NN is no one-way-function there is a way to retrieve the input data from my output if I have the entire network at hand (if I know biases, weights, etc.)
  
  My question is if the network is not known, is there a way for me (using ie. some probabilistic distributions) to make assumptions or even reconstruct the input data?
  
  **Tags:** neural-networks, machine-learning
  
  > **Comments:**
  > **username_1**: even if you have weights and biases, unless the network falls into some reversable scheme (most used in practice do not) then you still wont be able to reconstruct the input  
  > **username_2**: In principle, if an attacker can query the network as often as they want, they can simply build a dataset of encodings and sources and learn a decoder, that should get pretty good. I also remember that there was a paper a few years ago, that could even reconstruct a model by querying it enough. They noted, that this could for example be used to reconstruct recommendation engines from amazon.  However, if the network is trained in a specific way (like concatenating a random key to the input), it might be possible.  
  > **username_2**: Beside that, there exists a wikipedia article concerned with this question: https://en.wikipedia.org/wiki/Neural_cryptography  
  
  ---
  
  ## 0 answers
  
  