Hi guys, and more specifically @mcarilli
What is the current status of AMP using libtorch for inference?
I tried a similar approach to your suggestion in this thread:
at::autocast::set_enabled(true);
torch::Tensor result = model(source[b]);
at::autocast::clear_cache();
at::autocast::set_enabled(false);
This get me no speed difference at all, so I’m wondering if it’s actually used or not. Any other pointers here?
Cheers,
David