Pinterest, Engineering, Monetization.
a lot of the smaller layers didn’t affect inference time enough to make a difference. For those larger layers, we tried both data types. The main pitfall of float16 is that due to the reduced range, it’s easy for the model to overflow to “infinity.” We found that one of our main layers, the DCNv2