Looks like you are putting it as a list in the case where its not working.
self.s = [torch.nn.Parameter(torch.arange(1, embedding_dim+1, dtype=torch.float),  requires_grad=True)]    #The box bracket is for list initialisation.
That could be the problem

list is a variable and I think only tensors can be stored as parameters