Hi Timo_v,
Is your data set images? If so, you can use ImageDataGenerator 1 to load and shuffle the data for you.
If it is not images, you can assign each piece of data a unique identifier and shuffle the array of identifiers. You can then lazily load each piece of data as you iterate the the array.
On top of that, Colab has pro versions that may allow you to have increased memory.