Sorry @ptrblck I had to gather some more data.
The data I am seeing for workloads that are run at FP32 are slower than FP16.
See runtime column below. Additional information: my group is interested in the per-layer performance of these workloads, and based off the information I’ve collected below I’m struggling to see what the advantage of half precision is: even if there is reduced bandwidth, it seems like the FP16 layers generally take longer to run, so the throughput isn’t as great
Could you shed some light on that? Thanks a lot!
(BTW the rows below are taken as an average over ~30 trials)
Screenshot from 2020-07-06 14-48-401602×259 61.7 KB