title: ValueError: You are trying to load a weight file containing 16 layers into a model with 0 layers
body: <p>I came across some answers that said that the pretrained model weight files get automatically downloaded when we declare it in the <strong>.keras/models/</strong> directory. <code>vgg=VGG16(weights='imagenet')</code> I managed to locate the file from the directory and copied it to my working directory. When i try to load the model, the script returns with the error</p>
  
  <pre><code>ValueError: You are trying to load a weight file containing 16 layers into 
  a model with 0 layers.
  </code></pre>
  
  <p>What should I do?</p>
  
  <p>My source code is as follows</p>
  
  <pre><code>model=Sequential()
  model.add(Concatenate([image_model, language_model]))
  model.add(LSTM(1000, return_sequences=False))
  model.add(Dense(vocab_size))
  model.add(Activation('softmax'))
  model.load_weights('./models/vgg16_weights.h5')
  model.compile(loss='categorical_crossentropy', optimizer=Nadam(), 
  metrics=['accuracy'])
  model.summary()
  
  model.fit([images, captions], next_words, batch_size=512, epochs=50)
  </code></pre>