Answers.Body: <p>You can already do this with some neural networks, such as <a href="https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf" rel="nofollow noreferrer">GANs</a> and <a href="https://arxiv.org/pdf/1312.6114.pdf?source=post_page---------------------------" rel="nofollow noreferrer">VAEs</a>, which are generative models that learn a probability distribution over the inputs, so they learn how to produce e.g. images that are similar to the images they were trained with.</p>
  <p>Now, if you're interested in whether there is a <em>black-box method</em>, i.e. a method that, for every possible neural network, would tell you the dataset a neural network was trained with, that seems to be a harder task and definitely an <a href="https://en.wikipedia.org/wiki/Well-posed_problem" rel="nofollow noreferrer">ill-posed problem</a>, but I suspect that people working on <a href="https://ai.stackexchange.com/a/25080/2444">adversarial machine learning</a> have already attempted or will attempt to do something similar.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-12T16:23:05.303Z
Answers.LastActivityDate: 2021-02-12T16:23:05.303Z
Answers.UserDisplayName: nbro
Body: <p>Let's say we have a neural network that was trained with a dataset <span class="math-container">$D$</span> to solve some task. Would it be possible to &quot;reverse-engineer&quot; this neural network and get a vague idea of the dataset <span class="math-container">$D$</span> it was trained on?</p>
  
Tags: <neural-networks><machine-learning><datasets><generative-model>
Title: Would it be possible to determine the dataset a neural network was trained on?
UserDisplayName: sahal mulki
ThreadText: # Would it be possible to determine the dataset a neural network was trained on?
  
  (Asked by: username_0 on 2021-02-12)
  
  Let's say we have a neural network that was trained with a dataset $D$ to solve some task. Would it be possible to "reverse-engineer" this neural network and get a vague idea of the dataset $D$ it was trained on?
  
  **Tags:** neural-networks, machine-learning, datasets, generative-model
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-02-12. Score: 4 | Accepted answer
  
  You can already do this with some neural networks, such as [GANs](https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf) and [VAEs](https://arxiv.org/pdf/1312.6114.pdf?source=post_page---------------------------), which are generative models that learn a probability distribution over the inputs, so they learn how to produce e.g. images that are similar to the images they were trained with.
  
  Now, if you're interested in whether there is a *black-box method*, i.e. a method that, for every possible neural network, would tell you the dataset a neural network was trained with, that seems to be a harder task and definitely an [ill-posed problem](https://en.wikipedia.org/wiki/Well-posed_problem), but I suspect that people working on [adversarial machine learning](https://ai.stackexchange.com/a/25080/2444) have already attempted or will attempt to do something similar.
  
  
  ---
  
  