Hi guys, currently I have a model with a lot of classes on the output layer (20k classes) and I’m having some difficulties to use DataParallel, mainly because the first GPU is getting OOM.
Is it possible to have Data parallel, but doing the aggregation on the CPU instead of GPU? If not there is a way to have some sort of Mix between Data/Model parallel?