The torchvision reference script (train_quantization.py) has not been tested for multiGPU support yet. Recently, we landed fixes to the code that should solve this issue:

  
      github.com/pytorch/pytorch
  
  
    
  
    
  

  
    
      [quant] Regsiter fake_quant and observer attributes as buffers 8
    

    
      pytorch:gh/supriyar/56/base ← pytorch:gh/supriyar/56/head
    

    
      
        opened 
        
          
        
        Feb 21, 2020
      
      

      
        
          
          supriyar
        
      

      
        
          +18
          -13
        
      
    

  

  
  
    
    
  
  


Note that syncBN is not yet supported for quantization aware training.