@isalirezag
Did @rasbt 's suggestion work for you?
Because for me, it doesnt. Torch 0.4.
In [22]: emb.weight
Out[22]: 
Parameter containing:
tensor([[-0.5733,  0.8611,  0.0699],
        [ 1.7558,  0.7413, -0.4029],
        [-0.7413, -1.0902,  0.0550],
        [-0.9363, -0.6691, -0.4870],
        [ 0.0080, -0.4598,  0.9987]])

In [23]: K = torch.Tensor([[1 ,0, -1],[2, 0 ,-2], [1, 0 ,-1],[3,4,5],[6,7,8]])

In [24]: K
Out[24]: 
tensor([[ 1.,  0., -1.],
        [ 2.,  0., -2.],
        [ 1.,  0., -1.],
        [ 3.,  4.,  5.],
        [ 6.,  7.,  8.]])

In [25]: emb.weight = emb.weight+ nn.Parameter(K)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-25-9f658db7039f> in <module>()
----> 1 emb.weight = emb.weight+ nn.Parameter(K)

~/anaconda3/envs/san/lib/python3.6/site-packages/torch/nn/modules/module.py in __setattr__(self, name, value)
    549                 raise TypeError("cannot assign '{}' as parameter '{}' "
    550                                 "(torch.nn.Parameter or None expected)"
--> 551                                 .format(torch.typename(value), name))
    552             self.register_parameter(name, value)
    553         else:

TypeError: cannot assign 'torch.FloatTensor' as parameter 'weight' (torch.nn.Parameter or None expected)

How do I add arbitrary tensor to embedding’s weight?