Thanks for the response and the notes. Although this really does help as far as clarity (I was getting really uncertain with my method) it still does not solve me problem. This leads me to believe that they error my lie else where. Perhaps take a look at my loss function:
def loss_function(u_0, u_0_target, v_0, v_0_target, u_lb, u_rb, v_lb, v_rb, u_x_lb, u_x_rb,
v_x_lb, v_x_rb, f_u, f_v):
loss = nn.MSELoss()
return loss(u_0, u_0_target) + loss(v_0, v_0_target) + loss(u_lb, u_rb) + \
          loss(v_lb, v_rb) + loss(u_x_lb, u_x_rb) + loss(v_x_lb, v_x_rb) + \
          loss(f_u, torch.zeros((f_u.shape[0], 1))) + loss(f_v, torch.zeros((f_v.shape[0], 1)))

The NNs task is to minimize each of these individual pairs (the f_u and f_v are desired to be 0 hence the torch.zeros). I have previously plotted the progress of each individual loss, and it could clearly be seen that every loss converged to zero, except the loss of the two f’s which stay relatively constant. Do you have any idea how this might be possible?
Another thing that might be worth noting is that to find the individual inputs to the loss function I had to run the NN four times:
u_0, v_0, _, _, _, _ = pinn(batch_input_0)
u_lb, v_lb, u_x_lb, v_x_lb, _, _ = pinn(batch_input_lb)
u_rb, v_rb, u_x_rb, v_x_rb, _, _ = pinn(batch_input_rb)
_, _, _, _, f_u, f_v = pinn(batch_input_f)
though I doubt pytorch has issues with this.
Your help is very much appreciated.