It is hard to get a definite answer on that. It will depends a lot on the GPU you have and the ops you’re doing.
Linear layers going from 48 to 20 dimensions definitly are (but the cost of moving everything back to the cpu just to do this op is even more).
A Linear layer with 5M inputs should be ok for most recent GPUs.