Hi,
Would anyone happen to know where IBM hosts the Pytorch sources for the IBM PowerAI versions which feature LMS (Large Model Support) support for Power9 systems?
https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.0/navigation/pai_getstarted_pytorch.html 73

Large Model Support is a feature provided in PowerAI PyTorch that allows the successful training of deep learning models that would otherwise exhaust GPU memory and abort with “out of memory” errors. LMS manages this oversubscription of GPU memory by temporarily swapping tensors to host memory when they are not needed.One or more elements of a deep learning model can lead to GPU memory exhaustion. These include:
Model depth and complexity
Base data size (for example, high-resolution images)
Batch size
Traditionally, the solution to this problem has been to modify the model until it fits in GPU memory. This approach, however, can negatively impact accuracy – especially if concessions are made by reducing data fidelity or model complexity.
With LMS, deep learning models can scale significantly beyond what was previously possible and, ultimately, generate more accurate results.


  
      
      Linux on IBM Power
  
  
    

PowerAI FAQ - Linux on IBM Power 17

IBM PowerAI developer portal Learn about deep learning and PowerAI. Create something amazing. Get started PowerAI Watson ML Accelerator PowerAI Vision Use cases Code patterns Try PowerAI FAQ Frequently asked questions Find answers to some of the most...


  
  
    
    
  
  



  
      
      IBM IT Infrastructure Blog – 11 May 17
  
  
    

NVIDIA GTC deep learning dispatch: Day 3 - IBM IT Infrastructure Blog 3

NVIDIA Volta will feature NVIDIA NVLink 2.0 and IBM will continue to support the high-bandwidth interface between our Power family...


  
  
    
    
  
  


There is a Conda channel with ppc64le support, but those are all pre-built binaries.

  
      
      IBM Developer
  
  
    

Install PyTorch on Power systems 51

Learn how to install PyTorch from source on Power platforms.


  
  
    
    
  
  


I’d prefer if their sources with the LMS patches for PyTorch are available publically.
Are these patches included in the current PyTorch mainline? A quick search for PRs with “large model support” or “LMS” didn’t yield any results.