Solved by TheUnnamed22 in post #16 
              
              
                No wait that’s totally messed up. Why do you even clone the prediction for the opt prediction? 
pred = self.net.predict(sample.state)[sample.action]

            
            opt_pred = sample.reward
            if sample.next_state is not None:
                opt_pred += t.max(self.target_net.pred…