Hey all,
I’m finding that our native file buffer reader can load data about 300x faster if we can skip a numpy ndarray copy. Unfortunately, all attempts to go directly to PyTorch seem to be orders of magnitude slower than going through numpy.
Is there any way I can memcpy (from Python) my data directly into a Tensor efficiently?
Stats:
Tensor:  90 seconds
torch.tensor(buffer)

Tensor Set_ + FloatStorage: 48 seconds
tensor.set_(FloatStorage(buffer))

TorchVision loader: 7 seconds

Numpy conversion: 0.7 seconds
torch.from_numpy(numpy.asarray(buffer)) # uses numpy protocol

Native loader: 0.002 seconds

While the Numpy version is quite a nice improvement, you can see the motivation to get our raw data into tensors directly and also skip Numpy.
I’m looking for something like this C++ code, but in Python:
image772×383 20.7 KB

  
      blog.christianperone.com
  
  
    

PyTorch – Internal Architecture Tour | Terra Incognita 2



  
  
    
    
  
  


A similar request here:

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      Support creating a CPU tensor from ctypes pointer in Python 9
    

    
      
        opened 
        
          
        
        Mar 12, 2020
      
      


      
        
          
          vadimkantorov
        
      
    
  



  NumPy has utilities that allow to create an array from a ctypes pointer in pure Python (without C++ extensions):
https://docs.scipy.org/doc/numpy/reference/routines.ctypeslib.html#module-numpy.ctypeslib
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.ctypes.html
In C++ land,...



    enhancement
    module: docs
    module: numpy
    topic: memory usage
    triaged


  
  
    
    
  
  


Following that last post, this code is the fastest pure-Torch implementation I can muster, however I’m not sure if it’s leaking memory and it’s still slower than numpy:
from_buffer: 1.1 seconds

t = torch.Tensor()
s = torch.Storage.from_buffer(buffer, byte_order="native")
t.set_(s)

Thanks,
Jeremy