torch.zeros isn’t implemented for sparse tensors yet, though that isn’t hard to do. You could open a feature request here: https://github.com/pytorch/pytorch/issues 29 about getting sparse tensors to work with DataParallel if you feel strongly about it.
Out of curiosity, what are you using sparse tensors for?