title: Using pretrained convolutional network as a GAN discriminator
body: <p>I've pulled some code from TF2.0 documentation to generate images from a custom dataset. The code is <a href="https://www.tensorflow.org/beta/tutorials/generative/dcgan" rel="nofollow noreferrer">here</a> </p>
  
  <p>Since the documentation uses Keras i figured i might change the discriminator network to a pretrained network e.g InceptionV3, and only train the top layers. I've found <a href="https://keras.io/applications/" rel="nofollow noreferrer">this</a> code (Fine-tune InceptionV3 on a new set of classes). I cant seem to figure out how to replace the the one with the other. I understand that im trying to replace Sequential mode with the Functional API. But i guess they are somehow interconnected. However, im not a frequent Keras user.</p>
  
  <p>My questions is: How do i replace a custom CNN in Sequential mode with a pretrained one from the Functional API to use as a discriminator?</p>
  
  <p>EDIT: I would be happy if anyone has examples of doing it with the GANEstimator instead as im more used to TF.</p>
  
  <p><strong>Use the generator to generate a random image</strong></p>
  
  <pre><code>def make_generator_model():
      model = tf.keras.Sequential()
      model.add(layers.Dense(7*7*256, use_bias=False, input_shape=(100,)))
      model.add(layers.BatchNormalization())
      model.add(layers.LeakyReLU())
  
      model.add(layers.Reshape((7, 7, 256)))
      assert model.output_shape == (None, 7, 7, 256) # Note: None is the batch size
  
      model.add(layers.Conv2DTranspose(128, (5, 5), strides=(1, 1), padding='same', use_bias=False))
      assert model.output_shape == (None, 7, 7, 128)
      model.add(layers.BatchNormalization())
      model.add(layers.LeakyReLU())
  
      model.add(layers.Conv2DTranspose(64, (5, 5), strides=(2, 2), padding='same', use_bias=False))
      assert model.output_shape == (None, 14, 14, 64)
      model.add(layers.BatchNormalization())
      model.add(layers.LeakyReLU())
  
      model.add(layers.Conv2DTranspose(3, (5, 5), strides=(2, 2), padding='same', use_bias=False, activation='tanh'))
      assert model.output_shape == (None, 28, 28, 3)
  
      return model
  
  generator = make_generator_model()
  noise = tf.random.normal([1, 100])
  generated_image = generator(noise, training=False)
  </code></pre>
  
  <p><strong>The current discriminator and helpers (Outputs tf.Tensor([[-0.0003378]], shape=(1, 1), dtype=float32))</strong></p>
  
  <pre><code>def make_discriminator_model():
      model = tf.keras.Sequential()
      model.add(layers.Conv2D(64, (5, 5), strides=(2, 2), padding='same',
                                       input_shape=[28, 28, 1]))
      model.add(layers.LeakyReLU())
      model.add(layers.Dropout(0.3))
  
      model.add(layers.Conv2D(128, (5, 5), strides=(2, 2), padding='same'))
      model.add(layers.LeakyReLU())
      model.add(layers.Dropout(0.3))
  
      model.add(layers.Flatten())
      model.add(layers.Dense(1))
  
      return model
  
  cross_entropy = tf.keras.losses.BinaryCrossentropy(from_logits=True)
  
  def discriminator_loss(real_output, fake_output):
      real_loss = cross_entropy(tf.ones_like(real_output), real_output)
      fake_loss = cross_entropy(tf.zeros_like(fake_output), fake_output)
      total_loss = real_loss + fake_loss
      return total_loss
  
  discriminator = make_discriminator_model()
  decision = discriminator(generated_image)
  print (decision)
  </code></pre>
  
  <p><strong>The desired discriminator</strong></p>
  
  <pre><code>def make_discriminator_model():
      # create the base pre-trained model
      model = InceptionV3(weights='imagenet', include_top=False)
  
      # ADD TOP LAYERS
  
      # FREEZE ALL LAYERS EXCEPT TOP LAYERS
  
      return model
  
  # COMPILE
  
  def discriminator_loss(real_output, fake_output):
      real_loss = ??? # Real Loss
      fake_loss = ??? # Fake loss
      total_loss = real_loss + fake_loss
      return total_loss
  
  noise = tf.random.normal([1, 100])
  generated_image = generator(noise, training=False)
  
  discriminator = make_discriminator_model()
  decision = discriminator(generated_image)
  print (decision)
  </code></pre>
  
  <p><strong>All imports</strong></p>
  
  <pre><code>  from __future__ import absolute_import, division, print_function, unicode_literals
  
  try:
    # %tensorflow_version only exists in Colab.
    %tensorflow_version 2.x
  except Exception:
    pass
  import tensorflow as tf
  print('TF version: {}'.format(tf.__version__))
  
  import glob
  import imageio
  import matplotlib.pyplot as plt
  import numpy as np
  import os
  import PIL
  from PIL import Image
  from tensorflow.keras import layers
  import time
  
  from IPython import display
  from tensorflow.keras.preprocessing import image
  from tensorflow.keras.applications import vgg16
  import os.path
  from tensorflow.keras.applications.inception_v3 import InceptionV3
  from tensorflow.keras.preprocessing import image
  from tensorflow.keras.models import Model
  from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
  from tensorflow.keras import backend as K
  </code></pre>
  
  <p><strong>EDIT:
  This was the discriminator i ended up with! Thanks to @pandrey</strong></p>
  
  <pre><code>def make_discriminator_model():
      pre_trained = tf.keras.applications.InceptionV3(
          weights='imagenet', include_top=False, input_shape=IMG_SHAPE
      )
      pre_trained.trainable = False  # mark all weights as non-trainable
      model = tf.keras.Sequential([pre_trained])
      model.add(layers.GlobalAveragePooling2D())
      model.add(layers.Dense(1))   
      return model
  </code></pre>