Disk memory management
I am wondering if you guys could add support for disk memory management when running large transformer models. At least when running on my laptop with limited DRAM, it is not feasible to fully materialize some of the larger models (T5-3b e.g. or even T5-large) in DRAM, especially if there are other memory intensive tasks running (like the IDE). I'm wondering if it's possible for the Huggingface library to not materialize the entire model in DRAM as a Python object for these larger models and instead re-materialize them layer by layer from the disk.
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.