They are definitely not same values:

>>> lstm1 = nn.LSTM(input_dim, hidden_dim, num_layers=1)
>>> lstm2 = nn.LSTM(input_dim, hidden_dim, num_layers=2)
>>> lstm1.weight_ih_l0
Parameter containing:
-0.3027 -0.2689 -0.3551
 0.5509  0.1728  0.0360
-0.1964  0.1770  0.2209
-0.4915  0.3696  0.5712
 0.2401  0.0593 -0.4117
 0.4066  0.3684  0.3482
 0.2870 -0.0531  0.1953
 0.0928 -0.4165  0.5613
-0.4697  0.4112  0.1346
 0.3438 -0.1885  0.5242
 0.3756  0.2288  0.2949
-0.1401  0.0173 -0.0247
[torch.FloatTensor of size 12x3]

>>> lstm2.weight_ih_l0
Parameter containing:
-0.3672 -0.0299  0.1597
 0.0828 -0.2755  0.4451
 0.1861  0.1213 -0.5596
-0.2776 -0.4791 -0.2322
-0.5063  0.0437  0.1145
-0.2652 -0.0932  0.0865
-0.3323  0.4274 -0.3038
-0.1449 -0.1430  0.5393
 0.5589  0.1293 -0.5174
-0.4502  0.5351  0.2430
-0.5448 -0.4007 -0.2560
 0.5424 -0.1821 -0.0779
[torch.FloatTensor of size 12x3]


No, they are computed by different LSTMs with different parameters. They are different.