Okay I think I could just encapsulate it in a module and use the is_training method to help me decide which of the two forward versions I choose, this assumes I’ll have to call .train() and .eval() manually but at least it’s more “standard” than introducing my own flags. Probably something similar to BatchNorm.