Okaay, so apparently filenames do matter here…
I tried the exact same code, but called my file weights.pt in stead of weights.pkl and now it works like a charm!
Not too sure if this naming convention is mentioned somewhere, but I did not find anything about it on my searches through the web…