Solved by dys129 in post #2 
              
              
                Yes, they are the same. By default all the modules are initialized to train mode (self.training = True). Also be aware that some layers have different behavior during train/and evaluation (like BatchNorm, Dropout) so setting it matters. 
Also as a rule of thumb for programming in general, try to exp…