Hi!
I was wondering if it’s possible to train partially with fp16.
I’m dealing with a very big network. I would like to use fp16 in a submodule of the network.
Does autograd deal with casting from float16 to float32 during the backprop?
In other post i saw someone who recommends to use fp32 on batch normalization not to have convergence issues. So there should be no problem right?