EDIT - If you come across this answer just see @Mona_Jalal 's answer below (use model.load_state_dict(state_dict, strict=False)).
@Mona_Jalal it’s not quite the same. It might help if you include a snippet showing exactly how you try to load your checkpoint but my guess is you’re doing something like:
state_dict = torch.load('checkpoint.pth')
model.load_state_dict(state_dict)

In which case the fix is to put the missing keys into state_dict. So do this:
state_dict = torch.load('checkpoint.pth')
state_dict['embeddingnet.embeddingnet.layer_attend1.0.weight'] = model.embeddingnet.embeddingnet.layer_attend1.0.weight
state_dict['embeddingnet.embeddingnet.layer_attend1.0.bias'] = model.embeddingnet.embeddingnet.layer_attend1.0.bias
model.load_state_dict(state_dict)

… or something like that, just making sure you’re pointing to the correct submodule. More generally you could do:
loaded_state_dict = torch.load('checkpoint.pth')
model_state_dict = model.state_dict()
for k, v in model_state_dict.items():
    if k not in loaded_state_dict:
        loaded_state_dict[k] = v
model.load_state_dict(state_dict)

I haven’t tested any of this just now so you may need to fiddle.
PS: Just a side note. You don’t need pretrained=True when creating the model, as you are just going to load up another set of weights anyway.