Hi! I defined a network and put it into cuda.I entered a picture and got the result.I found that the time to take the results is much longer than the time spent by the model prediction.0.007s vs 0.139s!!
I searched for the relevant questions. I know that this is the time consumed by GPU synchronization, but I defined the same network with tensorflow. I can easily get the result. It seems that there is no time consumed by synchronization. I want to ask whether there is a mistake in my way of using pytorch or a mistake in pytorch strategy.
After all, 0.139 seconds of printing results are much more than the 0.007 seconds predicted by the model? Here’s my code.
def load_model(self):
    network.load_net(self.model_path, self.net)
    if self.cuda:
        self.net.cuda()
    self.net.eval()
def predict(self, data):
    # load a image
    img = cv2.imread(data, 0)
    img = img.astype(np.float32, copy=False)
    img = img.reshape((1, 1, img.shape[0], img.shape[1]))
    # inference
    t1 = time.time()
    density_map = self.net(img)
    t2 = time.time()
    # torch.cuda.synchronize()
    t3 = time.time()
    density_map = density_map.data.cpu().numpy()
    t4 = time.time()
    print(t2 - t1)
    print(t4 - t3)
    et_count = np.sum(density_map)
0.00782918930053711
0.13913774490356445
Thank you for your advice !