There is a bug in version 1.4

  
      github.com/pytorch/pytorch
  
  
    
  
    
  

  
    
      fix handling of replica parameters in DataParallel 3
    

    
      pytorch:master ← ngimel:data_parallel
    

    
      
        opened 
        
          
        
        Feb 27, 2020
      
      

      
        
          
          ngimel
        
      

      
        
          +9
          -9
        
      
    

  

  
  
    
    
  
  


After using nightly version, the issue is solved.