I thought you might want to see some code-snippets which shows the neural network.
The linear layers are implemented by myself (instead of using nn.Linear()) in the forward_pass() function. I also tried to use the pre-defined nn.Linear but, unfortunately, it did not speed up the training.
The code snippet below shows the forward pass, which represents the neural network which has as output the solution ‘u’ to the PDE u_{xx}  = f(x), and a vector x as input.
Then we have the function net_u()  which returns the solution u from the forward_pass. The function net_r()  returns the residual of the PDE, which should in the end be equal to zero (since residual = u_xx - f(x)).
 # Evaluate the forward pass
    def forward_pass(self, H):
        num_layers = len(self.layers)
        for l in range(0,num_layers-2): 
            W = self.weights[l]
            b = self.biases[l]
            H = torch.tanh(torch.add(torch.matmul(H,W), b))
        W = self.weights[-1]
        b = self.biases[-1]
        H = torch.add(torch.matmul(H,W), b)
        return H

    # Forward pass for u
    def net_u(self, x):
        u = self.forward_pass(x)
        return u

    # Forward pass for f
    def net_r(self, x):
        # LHS
        x.requires_grad = True 
        u = self.net_u(x)
        u_x = torch.autograd.grad(outputs = u.sum(), inputs = x, create_graph = True)[0]/self.sigma_x
        u_xx = torch.autograd.grad(outputs = u_x.sum(), inputs = x, create_graph = True)[0]/self.sigma_x
        x.requires_grad = False
        # RHS
        f = self.f(x*self.sigma_x + self.mu_x)

        residual = u_xx - f
        return residual

With this in mind, we can write our loss function (mean squared error) as:
    def loss_r(self):
        r_pred = self.net_r(self.X_r)
        loss_r = torch.mean(torch.square(r_pred))
        return loss_r

    def loss_u(self):
        u_pred = self.net_u(self.X_u)
        loss_u = torch.mean(torch.square(self.Y_u - u_pred))
        return loss_u

    def loss(self):
        loss = self.loss_u() + self.loss_r()
        return loss

In the training, the loss() will be minimized.