I guess you meant dataset.train_data and dataset.train_labels
Using dataset.train_labels is a great idea. It’s exaltly what I wanted.
But as you  know, dataset.train_data is not transformed. MNIST images are transformed in getitem()  and though I could develop my own dataset, I like the simplicity of torchvision MNIST.
So my updated solution is:
x_all_ts = torch.tensor([mnist_ds[i][0].numpy() for i in range(0, len(mnist_ds))])
t_all_ts = mnist_ds.train_labels

Regarding your first sentence , I am a bit confused. How to push the MNIST dataset onto the GPU? did you mean push dataset.train_data ?