Not that I’m aware of. Did the work around mentioned in the blog post not work for you?

I implemented everything as mentioned in the blog post but GPU #1 still runs out of memory when increasing the batch_size. I notice a significant increase in memory inside the loss forward method.
So I think initialization is fine but at runtime it just uses more memory with every computation added to the graph inside the loss forward method.
DataParallel has to have a mechanism to predict how much memory will be used by a model as well as the loss(criterion), right? So I should expose that information to DataParallelCriterion when initializing or change my loss function.
What’s the correct way of doing this?