Well, I don’t really catch your drift. Maybe you could point me to the doc you’re referring to. Adding the bias to hidden layer would mean changing the bias definition to:
bias = register_parameter("bias", torch::zeros({ numHid }));

and then
hidden_layer += bias;

Linear does basically what you’re doing …
output = torch::mm(input, weight) + bias