should find gcc-7 (and g++7) and then the build will succeed.
As always make sure to edit the paths in the example to match your situation.
FairScale
By integrating FairScale the [Trainer]
provides support for the following features from the ZeRO paper:
Optimizer State Sharding
Gradient Sharding
Model Parameters Sharding (new and very experimental)
CPU offload (new and very experimental)
You will need at least two GPUs to use this feature.
Installation:
Install the library via pypi:
bash
pip install fairscale
or via transformers' extras:
bash
pip install transformers[fairscale]
(available starting from transformers==4.6.0) or find more details on the FairScale's GitHub page.
If you're still struggling with the build, first make sure to read CUDA Extension Installation Notes.
If it's still not resolved the build issue, here are a few more ideas.
