Thanks for the shape information.
Your code snippet works fine in my setup:
device = torch.device("cuda")
model = DriverNet().to(device)
model = model.eval()

image_tensor = torch.randn(1,3,218,291).to(device)
output = model(image_tensor)
print(output)
> tensor([[ 0.0792, -0.0424]], device='cuda:0', grad_fn=<AddmmBackward>)

Could you remove the model = torch.load("model.pt") operation, as it could load any other stored model?