Answers.Body: <p>The way you would recognize images (image -> genre) would be very different from the other way around (genre -> image). For the former you are correct on what would be involved.</p>
  
  <p>For the latter, if you want large images then GANs are indeed the way to go. Currently the largest images we can generate are on the order of 220 x 220 pixels, mostly due to memory constraints on the GPU. There is no fundamental problem with having rectangular images, it just so happens that we use squares. You would be able to use identical architectures to train on rectangular data as well.</p>
  
  <p>The reason that some generated images you see (e.g. from DeepDream or NeuralStyle) are larger is that it's not a GAN. Both of these are not generative models in a standard sense, even if they do technically "generate" images. Instead, they merely modify an existing image by running backprop on a specifically designed loss function.</p>
  
  <p>TLDR: your movie genre recognition idea is sound. For generation, have a look at things like this (<a href="https://github.com/Newmu/dcgan_code" rel="nofollow noreferrer">https://github.com/Newmu/dcgan_code</a>), where Alec generated album covers instead of posters. If you have enough data it will do something half sensible.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-01-03T01:41:03.257Z
Answers.LastActivityDate: 2017-01-03T01:41:03.257Z
Answers.UserDisplayName: Andrej
Answers.Body: <p>There is a fundamental difference between Generative Models and Discriminative Models, to simplify it we could say that </p>
  
  <ul>
  <li><p>given <span class="math-container">$S$</span> Semantic Space (low dimensionality) and</p></li>
  <li><p>given <span class="math-container">$A$</span> Appearance Space (high dimensionality, e.g. space of <span class="math-container">$W \times H$</span> images)</p></li>
  </ul>
  
  <p>the Discriminator learns the function <span class="math-container">$f(A) \rightarrow S$</span> and the Generator learns the function <span class="math-container">$g(S) \rightarrow A$</span> </p>
  
  <p>According to this formalism we can say that <span class="math-container">$g \neq f^{-1}$</span> and a simple reason for this is <span class="math-container">$f$</span> is not invertible because it is not injective, in fact 2 different horror movie posters will be assigned the same label “horror movie” (assuming the network works well :) )</p>
  
  <p>So you have to find another way to define a mapping from <span class="math-container">$S$</span> to <span class="math-container">$A$</span> and this kind of job is related to the Generative Models </p>
  
  <p>Certainly GAN are state of the art in image generation but technically they are not the only choice, e.g. you could use a VAE as the Decoder Subnetwork is a generative model </p>
  
  <p>However from a practical point of view do not expected VAE results would that good: you’ll probably get blurred stuff like this one </p>
  
  <p><img src="https://jaan.io/images/variational-autoencoder-faces.jpg" alt="VAE1"></p>
  
  <p>so go for GAN but beware that training them properly is definitely not easy </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-01T11:42:58.187Z
Answers.LastActivityDate: 2019-03-01T11:42:58.187Z
Answers.UserDisplayName: Nicola Bernini
Body: <p>I've spent the past couple of months learning about neural networks, and am thinking of projects that would be fun to work on to cement my understanding of this tech.</p>
  
  <p>One thing that came to mind last night is a system that takes an image of a movie poster and predicts the genre of the movie. I think I have a good understanding of what'd be required to do this (put together a dataset, augment it, download a convnet trained on imagenet, finetune it on my dataset, and go from there).</p>
  
  <p>I also thought that it would be pretty cool to run the system backwards at the end, so that I could put in e.g. a genre like 'horror' and have the system generate a horror movie poster. I expect that it will be very bad at this because I'm not a team of expert researchers, but I think I could have some fun hacking on it even if it only ever generated incomprehensible results.</p>
  
  <p>Here's what I'm having trouble understanding: on the one hand, all the convnets whose architecture I've seen described seem to rely on being given very small, square input images (on the order of 220px by 220px iirc), and movie posters are rectangular, and a generated poster would have to be of a larger size in order for a human to make any sense of it.  I've seen several examples of papers where researchers use convnets to generate images, e.g. the adversarial system that generates pictures of birds and flowers, and a system that generates the next few frames of video when given a feed of a camera sweeping across the interior of a room, but all of those generated images seemed to be of the small square size I've been describing.</p>
  
  <p>On the other hand, I've seen lots of "deep dream" images over the past year or so that have been generated by convnets and are of a much larger size than ~220px by ~220px.</p>
  
  <p>Here's my question: is it possible for me to build the system I describe, which takes a movie genre and outputs a movie poster of a size like e.g. 400px by 600px? [I'm not asking about whether or not the resulting poster would be any <em>good</em> - I'm curious about whether or not it's possible to use a convnet to generate an image of that size.]</p>
  
  <p>If it is possible, <em>how</em> is it possible, given that these systems seem to expect small, square input images?</p>
  
Tags: <neural-networks><machine-learning><deep-learning><convolutional-neural-networks>
Title: Feasibility of generating large images with a convnet
UserDisplayName: JR Heard
ThreadText: # Feasibility of generating large images with a convnet
  
  (Asked by: username_0 on 2017-01-02)
  
  I've spent the past couple of months learning about neural networks, and am thinking of projects that would be fun to work on to cement my understanding of this tech.
  
  One thing that came to mind last night is a system that takes an image of a movie poster and predicts the genre of the movie. I think I have a good understanding of what'd be required to do this (put together a dataset, augment it, download a convnet trained on imagenet, finetune it on my dataset, and go from there).
  
  I also thought that it would be pretty cool to run the system backwards at the end, so that I could put in e.g. a genre like 'horror' and have the system generate a horror movie poster. I expect that it will be very bad at this because I'm not a team of expert researchers, but I think I could have some fun hacking on it even if it only ever generated incomprehensible results.
  
  Here's what I'm having trouble understanding: on the one hand, all the convnets whose architecture I've seen described seem to rely on being given very small, square input images (on the order of 220px by 220px iirc), and movie posters are rectangular, and a generated poster would have to be of a larger size in order for a human to make any sense of it. I've seen several examples of papers where researchers use convnets to generate images, e.g. the adversarial system that generates pictures of birds and flowers, and a system that generates the next few frames of video when given a feed of a camera sweeping across the interior of a room, but all of those generated images seemed to be of the small square size I've been describing.
  
  On the other hand, I've seen lots of "deep dream" images over the past year or so that have been generated by convnets and are of a much larger size than ~220px by ~220px.
  
  Here's my question: is it possible for me to build the system I describe, which takes a movie genre and outputs a movie poster of a size like e.g. 400px by 600px? [I'm not asking about whether or not the resulting poster would be any *good* - I'm curious about whether or not it's possible to use a convnet to generate an image of that size.]
  
  If it is possible, *how* is it possible, given that these systems seem to expect small, square input images?
  
  **Tags:** neural-networks, machine-learning, deep-learning, convolutional-neural-networks
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-01-03. Score: 2 | Accepted answer
  
  The way you would recognize images (image -> genre) would be very different from the other way around (genre -> image). For the former you are correct on what would be involved.
  
  For the latter, if you want large images then GANs are indeed the way to go. Currently the largest images we can generate are on the order of 220 x 220 pixels, mostly due to memory constraints on the GPU. There is no fundamental problem with having rectangular images, it just so happens that we use squares. You would be able to use identical architectures to train on rectangular data as well.
  
  The reason that some generated images you see (e.g. from DeepDream or NeuralStyle) are larger is that it's not a GAN. Both of these are not generative models in a standard sense, even if they do technically "generate" images. Instead, they merely modify an existing image by running backprop on a specifically designed loss function.
  
  TLDR: your movie genre recognition idea is sound. For generation, have a look at things like this (<https://github.com/Newmu/dcgan_code>), where Alec generated album covers instead of posters. If you have enough data it will do something half sensible.
  
  
  ---
  
  ### Answer by username_2 on 2019-03-01. Score: 1
  
  There is a fundamental difference between Generative Models and Discriminative Models, to simplify it we could say that
  
  * given $S$ Semantic Space (low dimensionality) and
  * given $A$ Appearance Space (high dimensionality, e.g. space of $W \times H$ images)
  
  the Discriminator learns the function $f(A) \rightarrow S$ and the Generator learns the function $g(S) \rightarrow A$
  
  According to this formalism we can say that $g \neq f^{-1}$ and a simple reason for this is $f$ is not invertible because it is not injective, in fact 2 different horror movie posters will be assigned the same label “horror movie” (assuming the network works well :) )
  
  So you have to find another way to define a mapping from $S$ to $A$ and this kind of job is related to the Generative Models
  
  Certainly GAN are state of the art in image generation but technically they are not the only choice, e.g. you could use a VAE as the Decoder Subnetwork is a generative model
  
  However from a practical point of view do not expected VAE results would that good: you’ll probably get blurred stuff like this one
  
  ![VAE1](https://jaan.io/images/variational-autoencoder-faces.jpg)
  
  so go for GAN but beware that training them properly is definitely not easy
  
  
  ---
  
  