Thanks, but it still doesn’t work. I think I should point out, that when I print the loss, most of the values are fine (somewhere under 1), but in some specific conditions, the loss can even be around 60.
(My PyCharm console)

Also, I noticed that most of the people gather the state-action values from the prediction and target and calculate the loss with these. Instead I use the whole tensor, so I wonder if this is not the problem.
(Code from PyTorch tutorial on DQN)
state_action_values = policy_net(state_batch).gather(1, action_batch)

next_state_values = torch.zeros(BATCH_SIZE, device=device)
next_state_values[non_final_mask] = target_net(non_final_next_states).max(1)[0].detach()