As suggested in https://github.com/pytorch/pytorch/issues/1479 2, I tried doing
f_embed = Variable(f_embed.view(f_embed.shape[0], f_embed.shape[1]), requires_grad=True)
s_embed = Variable(s_embed.view(s_embed.shape[0], s_embed.shape[1]), requires_grad=True)
but still my network is not learning