For anyone else who finds this question, I managed to get something like what I wanted by modding the code on my own. You’ll need to build your own libpytorch if you want to use it (for now):

  
      github.com/neonbjb/pytorch
  
  
    
  
    
  

  
    
      Add ability to access saved variables from Python 8
    

    
      
        committed 
        
          
        
        Mar 6, 2020
      
      

      
        
          
          neonbjb
        
        
      

      
        
          +40
          -1
        
      
    

  



  
    Capability to restore them efficiently still needed. You can
do this in C++ by messing with the data_ field directly,
but a nice...
  


  
  
    
    
  
  


I’ve filed a feature request in case the mods are interested in bringing something like this into mainline:

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      Allow accessing or save/restore of SavedVariables 5
    

    
      
        opened 
        
          
        
        Mar 5, 2020
      
      


      
        
          
          neonbjb
        
      
    
  



  🚀 Feature
Allow users to save and restore the state of the intermediate variables used to compute the backward pass of a...



    needs research
    triaged