Torch.transpose is too slow in GPU,slower than CPU
<SCODE>import torch
import time
from torch.autograd import Variable

x = Variable(torch.randn(100,500))
cputimes = []
for sampl in (1000, 10000, 100000, 1000000):
    start = time.time()
    for i in range(sampl):
        y = torch.transpose(x,0,1)
    end = time.time()
    cputimes.append(end-start)
print(cputimes)

x = x.cuda(device_id=2)
gputimes = []
for sampl in (1000, 10000, 100000, 1000000):
    start = time.time()
    for i in range(sampl):
        y = torch.transpose(x,0,1)
    end = time.time()
    gputimes.append(end-start)
print(gputimes)
<ECODE> test2: <SCODE>In [16]: torch.cuda.set_device(2)

In [17]: %timeit torch.transpose(torch.FloatTensor(20,100),1,0)
The slowest run took 26.26 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 1.72 µs per loop

In [18]: %timeit torch.transpose(torch.cuda.FloatTensor(20,100),1,0)
The slowest run took 21.21 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 3.05 µs per loop<ECODE>
Kyle
It case my model run very slowly,how can I solve it
Kyle
Hi, This code sample is slow only because of the python loop which calls c functions. To make it faster, you need to find a way to remove this loop. If in your case, you want to transpose a bunch of matrices, you could for example stack them in a single tensor and then call transpose on this tensor.
albanD
<SCODE>class NoOp(torch.autograd.Function):
    def __init__(self):
        super().__init__()
        
    def forward(self, x):
        return x

def print_times(x, func, msg):
    start = time.time()
    for i in range(1000000):
        _ = func(x)
    t = time.time() - start
    print("{}: {:.5f}".format(msg, t))

tensor   = torch.randn(100, 500)
ndarray  = tensor.numpy()
variable = Variable(tensor)
    
print_times(tensor,   lambda x: x,             "Python noop")
print_times(ndarray,  lambda x: x.transpose(), "Numpy transpose")
print_times(tensor,   lambda x: x.t(),         "Torch transpose")
print_times(variable, lambda x: NoOp()(x),     "Autograd noop")
print_times(variable, lambda x: x.t(),         "Autograd transpose")

# output:
#
# Python noop: 0.07554
# Numpy transpose: 0.23783
# Torch transpose: 0.49813
# Autograd noop: 1.95098
# Autograd transpose: 3.72835
<ECODE>
elanmart
Actually I changed my code and remove the transpose op, but the model still run slower on GPU than CPU, here is some of my model code . <SCODE>x = x.unsqueeze(1)
x0 = F.relu(self.conv0_0(x)).squeeze(3)
x1 = F.relu(self.conv0_1(x)).squeeze(3)
x = torch.cat((x0,x1),1)

x = x.unsqueeze(1)
x0 = F.relu(self.conv1_0(x)).squeeze(2)
x1 = F.relu(self.conv1_1(x)).squeeze(2)

x0 = F.max_pool1d(x0, x0.size(2)).squeeze(2)
x1 = F.max_pool1d(x1, x1.size(2)).squeeze(2)
x = torch.cat((x0,x1),1)
<ECODE>
Kyle
albanD