Sounds like what you’ve got is more commonly called “trivially” parallel (or “embarrassingly parallel” https://en.wikipedia.org/wiki/Embarrassingly_parallel 2): You want to have different models work on the same data without talking to each other.  That’s easy. Just start the run 8 different times, and preface each command-line call with CUDA_VISIBLE_DEVICES=X, where X is the GPU for that run.
In “data parallel” applications (https://en.wikipedia.org/wiki/Data_parallelism 2), different subsets of the dataset are sent to different nodes, which work on their subset of the dataset, and the results are typically combined later.
If you’re worried about 8 copies of the dataset taking up too much RAM on your machine so you want the 8 GPU threads to all share the same CPU  data then… yea, that’s a neat idea.  Not sure what the name for that computing paradigm is, maybe “model parallelism”; I doubt PyTorch has a pre-made method for this, but you should be able to do it using the .to(device) method in PyTorch 0.4.  Thus you could send the different models to different GPUs, and you’ll also have to explicitly send each (copy of) part of the dataset to each GPU as well, e.g. in the midst of a batch training loop.   Let me know if that works.  …I might try this myself!