Body: <p>Suppose we trained a neural network on some training set that we call <span class="math-container">$X$</span>. </p>
  
  <p>Given the neural network and the method of training(algorithm, hyperparameters etc.) can we infer anything about <span class="math-container">$X$</span>.</p>
  
  <p>Now, instead, suppose we also have some subset of the training data <span class="math-container">$Y \in X$</span> available. Is there anything we can infer about <span class="math-container">$Y^c$</span>?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-15T18:13:28.713Z
Comments.Text: What you refer with $Y^c$?
Comments.UserDisplayName: mico
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-15T18:50:37.193Z
Comments.Text: And should this belong to DataScience SE site?
Comments.UserDisplayName: mico
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-15T20:53:05.817Z
Comments.Text: @mico I don't really think so. This is a conceptual question about neural networks.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-15T21:48:43.280Z
Comments.Text: @mico the complement of Y in X
Comments.UserDisplayName: Dilitante
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-15T23:13:02.160Z
Comments.Text: @Dilitante You're asking two questions here. I would ask just one. I think it is already pretty difficult to answer your first vague question. What do you mean by "conclusion"? We could e.g. conclude that $X$ is the training dataset, but, of course, this is not what you're looking for, so, please, try to be more specific.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-16T08:22:35.757Z
Comments.Text: I have edited the question to make it clear that $X$ is unknown
Comments.UserDisplayName: Dilitante
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-16T08:55:43.630Z
Comments.Text: The impression I got from the question is that you're curious if, for example, personally sensitive information may "leak" from training data into the trained neural network's weights, and may still partially be reconstructed by a third party with access to the NN weights but no access to the original training data? I definitely know that this is a thing, there are people doing research on this (and on how to train models such that there is no risk of leaking personal data to users of the trained model)... but I'm not familiar enough with this stuff to write a proper answer.
Comments.UserDisplayName: Dennis Soemers
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-16T16:15:24.420Z
Comments.Text: Yes, along those lines. I heard someone make a comment that GDPR could mean models must be retrained when the data they were originally trained on is no longer allowed to be kept. At first I thought they were confused.
Comments.UserDisplayName: Dilitante
Tags: <neural-networks><machine-learning><datasets>
Title: What can be inferred about the training data from a trained neural network?
UserDisplayName: Dilitante
ThreadText: # What can be inferred about the training data from a trained neural network?
  
  (Asked by: username_0 on 2019-07-15)
  
  Suppose we trained a neural network on some training set that we call $X$.
  
  Given the neural network and the method of training(algorithm, hyperparameters etc.) can we infer anything about $X$.
  
  Now, instead, suppose we also have some subset of the training data $Y \in X$ available. Is there anything we can infer about $Y^c$?
  
  **Tags:** neural-networks, machine-learning, datasets
  
  > **Comments:**
  > **username_1**: What you refer with $Y^c$?  
  > **username_1**: And should this belong to DataScience SE site?  
  > **username_2**: @username_1 I don't really think so. This is a conceptual question about neural networks.  
  > **username_0**: @username_1 the complement of Y in X  
  > **username_2**: @username_0 You're asking two questions here. I would ask just one. I think it is already pretty difficult to answer your first vague question. What do you mean by "conclusion"? We could e.g. conclude that $X$ is the training dataset, but, of course, this is not what you're looking for, so, please, try to be more specific.  
  > **username_0**: I have edited the question to make it clear that $X$ is unknown  
  > **username_3**: The impression I got from the question is that you're curious if, for example, personally sensitive information may "leak" from training data into the trained neural network's weights, and may still partially be reconstructed by a third party with access to the NN weights but no access to the original training data? I definitely know that this is a thing, there are people doing research on this (and on how to train models such that there is no risk of leaking personal data to users of the trained model)... but I'm not familiar enough with this stuff to write a proper answer.  
  > **username_0**: Yes, along those lines. I heard someone make a comment that GDPR could mean models must be retrained when the data they were originally trained on is no longer allowed to be kept. At first I thought they were confused.  
  
  ---
  
  ## 0 answers
  
  