I think my approach was wrong.
This is how I make this work:
old_embed = decoder.embed.weight.data
old_weight = decoder.linear.weight.data
old_bias = decoder.linear.bias.data


 decoder.embed = nn.Embedding(vocab_size, args.embed_size)
 decoder.linear = nn.Linear(args.hidden_size, vocab_size)

 decoder.embed.weight.data[:tmp_dim, :] = old_embed
 decoder.linear.weight.data[:tmp_dim] = old_weight
 decoder.linear.bias.data[:tmp_dim] = old_bias

much more easy and elegant!