I am using an Inference class and the following code is one of its member function. Every batch is of size 128.
def predict(image_objs_batches):
    for batch in image_objs_batches:
        with torch.no_grad():
            _, outputs = self.model(batch)