Hello @ptrblck,
Apologies for the late response. I was trying to put code together to replicate the issue, and it seems the mystery is finally resolved.
Apparently the issue was coming from my own code - I’d written a custom Dataset class which relied heavily on dicts. The behavior of dicts between Python 3.5 and 3.6 changed drastically, which inadvertently caused some of my training data to bleed into my testing data on Python 3.5, while this wasn’t happening on Python 3.6. Therefore the performance gain reported was completely spurious and I can confirm now that the results can now be replicated across Python releases.
Really sorry for wasting your time with this. Thank you very much for your input. I appreciate it alot.
Thanks
Ps: I will change the title of this thread to reflect this turn of events