In the following graph, you can see the execution time in seconds taken for Torch.svd in both CPU and GPU. As you can see the time taken for GPU is larger than that time taken for CPU and keeps increasing as we increase the number of records. Please note that the GPU memory has a size of 12 GB and the maximum number of records that can be run with svd is 3.25 million.

chart (4).png997×616 7.59 KB

Following is the code I used.
import torch
from datetime import datetime
import sys

n_records = sys.argv[1];
x = torch.zeros(n_records, 300).cuda()
s = datetime.now()
u, _, _ = torch.svd(x, some=True)
e = datetime.now()
del x
el = (e-s).total_seconds()
print(el)

Following is the output from NVIDIA visual profiler for 1000 and 1 million records respectively. As you can see after the ‘kernelPointWiseApply2’ cuda kernel invocation there is a certain amount of idle time that is introduced. This idle time between the first and second kernel functions in the “compute” section has also increased.

svd_1K_time.png1866×1033 111 KB


svd_1M_time.png1861×1051 128 KB

Could anyone help me in understanding why this idle time occurs and why it keeps increasing as we increase the size of the dataset?