Answers.Body: <p>The <a href="https://ai.stackexchange.com/a/12504/2444"><em>auto-encoder</em></a> (AE) can be used to learn a compressed representation (a vectorised hash value) of each observation in the training dataset, <span class="math-container">$z$</span>, which can then be used to later retrieve the original (or similar) observation.</p>
  
  <p>The <a href="https://arxiv.org/pdf/1312.6114.pdf" rel="nofollow noreferrer"><em>variational auto-encoder</em></a> (VAE), a <a href="https://ai.stackexchange.com/a/12354/2444">statistical</a> variation of AE, can also be used to generate objects <em>similar</em> to the observations (or inputs) in the training set.</p>
  
  <p>There are other data compressor models, for example, <a href="http://artem.sobolev.name/posts/2016-07-11-neural-variational-inference-variational-autoencoders-and-Helmholtz-machines.html" rel="nofollow noreferrer">Helmholtz machine</a>, which precedes both the AE and VAE.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-28T10:26:27.470Z
Answers.LastActivityDate: 2019-05-29T23:16:24.783Z
Answers.LastEditDate: 2019-05-29T23:16:24.783Z
Answers.UserDisplayName: nbro
Answers.Body: <p>Train a network that has large input and small output. Turn it upside down (yes, you may do that). By giving the small outputs, corresponging to input, the ideally- trained network will generate those large data. 
  But you see in all compression there will be data lost, so generated data will be <em>slightly</em> :DD different then original dataset. So its suitable for statistical data, like images, whatever, but not for structured like text or the most unsuitable example - program source code.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-28T13:42:37.613Z
Answers.LastActivityDate: 2019-05-28T13:42:37.613Z
Answers.UserDisplayName: user8426627
Answers.Body: <p>An Auto-Encoder is probably what you are looking for. AE is a very powerful Neural Network when you want to compress data and get a lower dimensional representation of the data with maximum information retained. </p>
  
  <p>If you are interested to know how it works -- Think of it as training a NN to predict the data itself. Which means your input and output layers would exactly be the same. So how does this help in compression ? The Hidden Layer -- Here is where the magic happens. The compression comes from the fact that we use lesser number of neurons in the hidden layer than in the i/p layer. Assuming that the NN we build nicely predicts the i/p at the end of the training,  the output of the hidden layers can be thought of as newly engineered features which are less in dimension yet powerful enough to represent the information in original data. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T08:16:14.677Z
Answers.LastActivityDate: 2019-06-06T08:16:14.677Z
Answers.UserDisplayName: Srikant Jayaraman
Answers.Body: <p>See <a href="https://ai.stackexchange.com/questions/4245/what-is-the-difference-between-encoders-and-auto-encoders/11808">What is the difference between encoders and auto-encoders?</a> For a working example of a neural network being used to compress (and decompress) images.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-06T11:01:52.903Z
Answers.LastActivityDate: 2019-06-06T11:01:52.903Z
Answers.UserDisplayName: DrMcCleod
Body: <p>When training a neural network, we often run into the issue of overfitting.</p>
  
  <p>However, is it possible to put overfitting to use? Basically, my idea is, instead of storing a large dataset in a database, you can just train a neural network on the <em>entire</em> dataset until it overfits as much as possible, then retrieve data "stored" in the neural network like it's a hashing function.</p>
  
Tags: <neural-networks><autoencoders><overfitting><data-compression>
Title: Is it possible for a neural network to be used to compress data?
UserDisplayName: Bryan Tan
ThreadText: # Is it possible for a neural network to be used to compress data?
  
  (Asked by: username_0 on 2019-05-28)
  
  When training a neural network, we often run into the issue of overfitting.
  
  However, is it possible to put overfitting to use? Basically, my idea is, instead of storing a large dataset in a database, you can just train a neural network on the *entire* dataset until it overfits as much as possible, then retrieve data "stored" in the neural network like it's a hashing function.
  
  **Tags:** neural-networks, autoencoders, overfitting, data-compression
  
  ---
  
  ## 4 answers
  
  ### Answer by username_1 on 2019-05-28. Score: 6
  
  The [*auto-encoder*](https://ai.stackexchange.com/a/12504/2444) (AE) can be used to learn a compressed representation (a vectorised hash value) of each observation in the training dataset, $z$, which can then be used to later retrieve the original (or similar) observation.
  
  The [*variational auto-encoder*](https://arxiv.org/pdf/1312.6114.pdf) (VAE), a [statistical](https://ai.stackexchange.com/a/12354/2444) variation of AE, can also be used to generate objects *similar* to the observations (or inputs) in the training set.
  
  There are other data compressor models, for example, [Helmholtz machine](http://artem.sobolev.name/posts/2016-07-11-neural-variational-inference-variational-autoencoders-and-Helmholtz-machines.html), which precedes both the AE and VAE.
  
  
  ---
  
  ### Answer by username_2 on 2019-05-28. Score: 1
  
  Train a network that has large input and small output. Turn it upside down (yes, you may do that). By giving the small outputs, corresponging to input, the ideally- trained network will generate those large data.
  But you see in all compression there will be data lost, so generated data will be *slightly* :DD different then original dataset. So its suitable for statistical data, like images, whatever, but not for structured like text or the most unsuitable example - program source code.
  
  
  ---
  
  ### Answer by username_3 on 2019-06-06. Score: 1
  
  An Auto-Encoder is probably what you are looking for. AE is a very powerful Neural Network when you want to compress data and get a lower dimensional representation of the data with maximum information retained.
  
  If you are interested to know how it works -- Think of it as training a NN to predict the data itself. Which means your input and output layers would exactly be the same. So how does this help in compression ? The Hidden Layer -- Here is where the magic happens. The compression comes from the fact that we use lesser number of neurons in the hidden layer than in the i/p layer. Assuming that the NN we build nicely predicts the i/p at the end of the training, the output of the hidden layers can be thought of as newly engineered features which are less in dimension yet powerful enough to represent the information in original data.
  
  
  ---
  
  ### Answer by username_4 on 2019-06-06. Score: 0
  
  See [What is the difference between encoders and auto-encoders?](https://ai.stackexchange.com/questions/4245/what-is-the-difference-between-encoders-and-auto-encoders/11808) For a working example of a neural network being used to compress (and decompress) images.
  
  
  ---
  
  