ptrblck:

Could you check the dtype of the layer’s parameters as well, please?


i did following manipulation and now it works. (convert to float then back to double after training). It probably not the cleanest way to do it so I will look up later to see what i can improve. Now, loss and train is working ok.
    # forward propagate input
    def forward(self, X):
        X = self.hidden1(X.float())
        X = self.act1(X)
        X = self.hidden2(X)
        X = self.act2(X)
        return X.double()