che85:

When using DataParallel, is there is a way to set maximum allocated memory for single GPU?


Not that I’m aware of. Did the work around mentioned in the blog post not work for you?



 che85:

Other than that, how does DataParallel decide how to split data?


The batch will be chunked in dim0 based on the number of available GPUs.