A model with a 2x throughput for an accuracy loss of 5% sounds like a really interesting use case.
Depending, how your method works, would it be possible to “restore the better accuracy” by disabling your method towards the end of the training?