Yes, apex makes sure mixed precision models work fine, i.e. potentially unsafe ops are performed in FP32, while other operations are performed using FP16.
I wanted to post this as the next suggestion, but you were faster. 
Were you able to run your script with apex?
I’m not sure, how easy that would be using the fast.ai wrapper.