i think it should be reasonably efficent to call .cpu() on pred and put it in a list.
prediction_list = []
def predict(self, dataloader):
    for i, batch in enumerate(dataloader):
        pred, output = self.step(batch)
        prediction_list.append(pred.cpu())

A more extreme case is to use CUDA pinned memory on the CPU, http://pytorch.org/docs/master/notes/cuda.html?highlight=pinned#best-practices 131
However, in your use-case i’m not sure you’ll gain much with that.