Solved by mcarilli in post #3 
              
              
                Anything that runs under autocast in a particular thread will have autocast enabled. 
MyModel.forward is what DP runs in a side thread.  If MyModel.forward is decorated with @autocast(), that takes care of enabling autocast for the side thread. 
If subModel.forward runs within MyModel’s forward, you…