Hi @sgugger, any updates on the possbility to reduce the RAM footprint on this model?
Eager to test it out for our application, but our 11GB of GPU RAM doesn’t seem to be enough to run the float16 model.
I cannot find a way to load the model into 2 separate GPU devices either (this would also be a solution for many of us) and so hoping for your solution to allow us to move forward!