Hi,
i have the following two settings for one sequence of length 1200
setting 1:
input size: batch size by sequence length (1, 1200)
for i in range(3):
transformer(input[i*512: (i+1)*512])
setting 2:
input size: batch size by sequence length-(3, 512)(this includes padding)
transformer(input)
i assumed setting2 would be faster than setting1, and it is true if i use GPU, but if im using CPU, setting 2 is slower than setting1
can someone please explain why this is the case?
thank you!