Body: <p>In <a href="https://arxiv.org/pdf/0812.4360.pdf" rel="nofollow noreferrer">Driven by Compression Progress:
  A Simple Principle Explains Essential Aspects of
  Subjective Beauty, Novelty, Surprise,
  Interestingness, Attention, Curiosity, Creativity,
  Art, Science, Music, Jokes</a> Jürgen Schmidhuber describes, using the idea of using compression progress on perceived data as an intrinsic reward signal:</p>
  <blockquote>
  <p>[Artists] create action sequences yielding interesting inputs, where interestingness is a measure of learning progress, for example [...] the saved number of bits needed to encode the data.</p>
  </blockquote>
  <p>Has such a (purely generative) system been created before? Say, one that creates 2d images?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-20T00:24:50.377Z
Comments.Text: Hello. Welcome to Artificial Intelligence Stack Exchange. Are you familiar with or heard of (variational) autoencoders?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-20T09:28:58.417Z
Comments.Text: @nbro Although variational autoencoders inherently use compression (and better compression arguably leads to better results in many cases), the loss signal is not directly related to degree of compression, but fidelity of reproduction using a fixed size compression. I think someone could answer using VAEs as an example, but it should be caveated as these don't quite reach Schmidhuber's design.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-20T09:33:23.793Z
Comments.Text: @NeilSlater Well, the KL divergence and cross-entropy (which form the ELBO objective function to train VAEs) can have an information-theoretic perspective. See [my answer](https://ai.stackexchange.com/a/20574/2444) if you are interested in more details. However, I don't know the details of Schmidhuber's design, but this answer also doesn't provide more details. That's why I was asking if the OP was aware of VAEs. If yes, maybe he had in mind something different and could provide more details about Schmidhuber's design.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-20T09:45:11.037Z
Comments.Text: @NeilSlater By the way, if I remember correctly, World Models (co-authored by Schmidhuber)  use the VAE. So, maybe that goes more in this direction, as the OP is talking about rewarding and intrinsic rewards. Anyway, I think this question is vague, so it needs more details to be answered appropriately, so I don't plan to give an answer now.
Comments.UserDisplayName: nbro
Tags: <papers><generative-model><rewards><image-generation><data-compression>
Title: Generative systems based on Schmidhuber's compression framework
UserDisplayName: 2080
ThreadText: # Generative systems based on Schmidhuber's compression framework
  
  (Asked by: username_0 on 2021-12-19)
  
  In [Driven by Compression Progress:
  A Simple Principle Explains Essential Aspects of
  Subjective Beauty, Novelty, Surprise,
  Interestingness, Attention, Curiosity, Creativity,
  Art, Science, Music, Jokes](https://arxiv.org/pdf/0812.4360.pdf) Jürgen Schmidhuber describes, using the idea of using compression progress on perceived data as an intrinsic reward signal:
  
  > [Artists] create action sequences yielding interesting inputs, where interestingness is a measure of learning progress, for example [...] the saved number of bits needed to encode the data.
  
  Has such a (purely generative) system been created before? Say, one that creates 2d images?
  
  **Tags:** papers, generative-model, rewards, image-generation, data-compression
  > **Comments:**
  > **username_1**: Hello. Welcome to Artificial Intelligence Stack Exchange. Are you familiar with or heard of (variational) autoencoders?  
  > **username_2**: @nbro Although variational autoencoders inherently use compression (and better compression arguably leads to better results in many cases), the loss signal is not directly related to degree of compression, but fidelity of reproduction using a fixed size compression. I think someone could answer using VAEs as an example, but it should be caveated as these don't quite reach Schmidhuber's design.  
  > **username_1**: @NeilSlater Well, the KL divergence and cross-entropy (which form the ELBO objective function to train VAEs) can have an information-theoretic perspective. See [my answer](https://ai.stackexchange.com/a/20574/2444) if you are interested in more details. However, I don't know the details of Schmidhuber's design, but this answer also doesn't provide more details. That's why I was asking if the OP was aware of VAEs. If yes, maybe he had in mind something different and could provide more details about Schmidhuber's design.  
  > **username_1**: @NeilSlater By the way, if I remember correctly, World Models (co-authored by Schmidhuber)  use the VAE. So, maybe that goes more in this direction, as the OP is talking about rewarding and intrinsic rewards. Anyway, I think this question is vague, so it needs more details to be answered appropriately, so I don't plan to give an answer now.  
  
  ---
  
  ## 0 answers
  
  