I have a huge text data and at certain amount of data (usually more than 700MB), I keep getting this message error.
Cpython…
Python multiprocessing struct.error: ‘i’ format requires -2147483648 <= number <= 2147483647
But I don’t use any of that and it’s probably because of pytorch features such as Dataset and so on.
Any idea how to solve this?