title: Changing batch_size parameter in keras leads to broadcast error
body: <p>I am running a simple encoder-decoder setup to train a representation for a one dimensional image. In this sample the input are lines with varying slopes and in the encoded layer we would expect something that resembles the slope. My setup is keras with a tensorflow backend. I am very new to this as well.
  It all works fine, at least until I move away from steps_per_epoch to batch_size in the model.fit() method. Certain values of the batch_size, such as 1,2,3, 8 and 16 do work, for others I get a value error. My initial guess was 2^n, but that did not work.
  The error I get for batch_size = 5 
  <code>ValueError: operands could not be broadcast together with shapes (5,50) (3,50) (5,50)</code></p>
  
  <p>I am trying to understand which relation between batch_size and training data is valid such that it always passes. I assumed that the training set would be simply divided into floor(N/batch_size) batches and the remainder would be processed as such.</p>
  
  <p>My questions are:</p>
  
  <ol>
  <li>What is the relation between size of data set and batch_size that are allowed.
  
  <ol start="2">
  <li>What exactly is the keras/tensorflow trying to do such that the batch_size is important? </li>
  </ol></li>
  </ol>
  
  <p>Thank you very much for the help.</p>
  
  <p>The code to reproduce this is</p>
  
  <pre><code>import numpy as np
  from keras.models import Model
  from keras.layers import Input, Dense, Conv1D, Concatenate
  from keras.losses import mse
  from keras.optimizers import Adam
  
  INPUT_DIM = 50
  INTER_DIM = 15
  LATENT_DIM = 1
  
  # Prepare Sample Data
  one_line = np.linspace(1, 30, INPUT_DIM).reshape(1, INPUT_DIM)
  test_array = np.repeat(one_line, 1000, axis=0)
  slopes = np.linspace(0, 1, 1000).reshape(1000, 1)
  data = test_array * slopes
  # Train test split
  train_mask = np.where(np.random.sample(1000) &lt; 0.8, 1, 0).astype('bool')
  x_train = data[train_mask].reshape(-1, INPUT_DIM, 1)
  x_test = data[~train_mask].reshape(-1, INPUT_DIM, 1)
  
  # Define Model
  input = Input(shape=(INPUT_DIM, 1), name='input')
  conv_layer_small = Conv1D(filters=1, kernel_size=[3], padding='same')(input)
  conv_layer_medium = Conv1D(filters=1, kernel_size=[5], padding='same')(input)
  merged_convs = Concatenate()(
      [conv_layer_small, conv_layer_medium])
  
  latent = Dense(LATENT_DIM, name='latent_layer',
                 activation='relu')(merged_convs)
  encoder = Model(input, latent)
  
  decoder_int = Dense(INTER_DIM, name='dec_int_layer', activation='relu')(latent)
  output = Dense(INPUT_DIM, name='output', activation='linear')(decoder_int)
  
  encoder_decoder = Model(input, output, name='encoder_decoder')
  
  
  # Add Loss
  reconstruction_loss = mse(input, output)
  encoder_decoder.add_loss(reconstruction_loss)
  encoder_decoder.compile(optimizer='adam')
  
  
  if __name__ == '__main__':
      epochs = 100
      encoder_decoder.fit(
          x_train,
          epochs=epochs,
          batch_size=4,
          verbose=2
      )
  </code></pre>