A quick note on reduced precision training:
Params are typically still stored in float32. But some intermediate computations can be done in bfloat16. Most standard Flax modules have a dtype  attribute, that you can set to bfloat16 to compute the safe parts in reduced precision.