Hi,
What is expensive depends a lot on what you are running.

If you perform operations on big things (like images or larger tensors) or with big batch sizes, then you can consider that the runtime will be 40% forward, 60% backward (for operations that are usually used for NN like conv/linear/relu…).
If you perform small operation most of the time, then the runtime is quite hard to predict as it will depend a lot on which operations you’re doing. If your total forward pass is very fast (in the order of the microsecond) then you will start to see the overhead of the autograd engine/python binding. And if you use gpu, you will have a noticeable overhead from just launching the kernels on top of the rest.

If the GPU usage is around 35% then I guess you’re either performing many small operations or you have a lot of other stuff happening in your code on top of just the "forward/backward"s.
Which one is it? Could you describe in more details what kind of things you’re running?