Solved by RowanG1 in post #2 
              
              
                Resolved: it was due to the FastAI normalize config for props in: 

dls = TabularDataLoaders.from_csv(props = [normalize]) 

I removed the normalize, and now the pytorch result matches FastAI.