@BramVanroy it really was. You got me at “After a weekend of debugging”. I can totally relate 
I have pre-processed my model input data and saved it as python lists. When loading the training data, I am setting format to torch. From your discussion with @lhoestq this conversion is as fast because loading converts numpy arrays to torch tensors, right? Do I need to save my data as numpy arrays as well? Doesn’t seem to be needed. What is the best format to save your training data so it can be loaded as fast as possible into torch tensors?