You can still load the data with pickle and depending on your data type transform it somehow to a valid format.
Once it’s transformed to a tensor you could save it with pytorch using torch.save and load it again if needed with torch.load, which will probably be faster than using pickle and the transformations.
Could you explain, what kind of data you’ve saved in the .p files?