I would like to elaborate a little on this. First, I don’t think it is as simple as 730 ms - 20 ms = 710 ms for backward pass, unless those 730 ms you measure only GPU part of the work. Second, you almost always have two phases of the work:

You prepare data (reading, resizing, augmenting images goes here), this work usually done by CPU
You train a neural network (those are forward and backward pass), this work usually done by GPU

To measure time spent throughout those task, you would probably better measure “data” time and “model” time separately. These separate measurements will help you to understand there the most of the work is taking place actually.
Consider my example. I am training a regular convnet classifier for kaggle competition with the backbone of ResNet200d. My setup is Google Colab V100 16 GB GPU. I use image size 768x768. Training takes ~17 images per second, validation ~ 37 images per second. I am actually not measuring the “data” part separately, so I can not tell you exact amount of time spent on GPU.
And you still can compare those numbers only then GPU utilization is the same, and you don’t have data processing as a bottleneck.
A valid solution on how to measure the time used by GPU will be to try it on synthetic data. Create tensors in memory and measure GPU time.