title: What layers are affected by dropout layer in Tensorflow?
body: <p>Consider transfer learning in order to use a pretrained model in keras/tensorflow. For each old layer, <code>trained</code> parameter is set to <code>false</code> so that its weights are not updated during training whereas the last layer(s) have been substituted with new layers and these must be trained. Particularly two fully connected hidden layers with <code>512</code> and <code>1024</code> neurons and and relu activation function have been added. After these layers a Dropout layer is used with <code>rate</code> <code>0.2</code>. This means that during each epoch of training <code>20%</code> of the neurons are randomly discarded.</p>
  <p>What layers does this dropout layer affect? Does it affect all the network including also the pretrained layers for which <code>layer.trainable=false</code> has been set or does it affect only the newly added layers? Or does it affect only the previous layer (i.e., the one with <code>1024</code> neurons)?</p>
  <p>In other words, which layer(s) do the neurons that are turned off during each epoch by the dropout belong to?</p>
  <pre><code>import os
  
  from tensorflow.keras import layers
  from tensorflow.keras import Model
    
  from tensorflow.keras.applications.inception_v3 import InceptionV3
  
  local_weights_file = 'weights.h5'
  
  pre_trained_model = InceptionV3(input_shape = (150, 150, 3), 
                                  include_top = False, 
                                  weights = None)
  
  pre_trained_model.load_weights(local_weights_file)
  
  for layer in pre_trained_model.layers:
    layer.trainable = False
    
  # pre_trained_model.summary()
  
  last_layer = pre_trained_model.get_layer('mixed7')
  last_output = last_layer.output
  
  # Flatten the output layer to 1 dimension
  x = layers.Flatten()(last_output)
  # Add two fully connected layers with 512 and 1,024 hidden units and ReLU activation
  x = layers.Dense(512, activation='relu')(x)
  x = layers.Dense(1024, activation='relu')(x)
  # Add a dropout rate of 0.2
  x = layers.Dropout(0.2)(x)                  
  # Add a final sigmoid layer for classification
  x = layers.Dense  (1, activation='sigmoid')(x)           
  
  model = Model( pre_trained_model.input, x) 
  
  model.compile(optimizer = RMSprop(lr=0.0001), 
                loss = 'binary_crossentropy', 
                metrics = ['accuracy'])
  </code></pre>