I face a strange problem, I have 2 models, model1 and model2, model1 is bigger than model 2.
when I test forward time of different model with same input, i need copy to gpu memory.
just like
if self.args.cuda:
device = torch.device(“cuda”)
else:
device = torch.device(“cpu”)
input = input.to(device)
when i collect time cost for “input = input.to(device)”, i found for same input, 2 models cost different time in GPU case:
for model1
input = input.to(device)  prepareData Time: 0.12544s
for model2
input = input.to(device)  prepareData Time: 0.01899s
when i switch to cpu model, they are same:
for model1
input = input.to(device)  prepareData Time: 0.00001s
for model2
input = input.to(device)  prepareData Time: 0.00001s
as i know that is just copy to gpu memory, so i think that should be same. that time cost puzzled me.
anyone know why?