Did you initialize both models with the same values?
Note that torch.Tensor creates a tensor with uninitialized memory, so that you might get arbitrary values (including Infs and NaNs).
I would recommend to initialize your custom model with some initialization method from torch.nn.init to make sure your parameters are in a reasonable range.
Also, if you compare the outputs of both approaches, make sure to use the same values (copy the parameter values from the module model to your custom model via .copy_).