I read the paper yet, but still feel confused.
Is that means “model.features = torch.nn.DataParallel(model.features)” corresponding to Model parallelism, and “torch.nn.DataParallel(model)” corresonding to Data parallelism?
But why “args.arch.startswith(‘alexnet’) or args.arch.startswith(‘vgg’)” ,then do Model Parallelism, else do Data parallelism?