Hey @thecity2, I am currently using HF on a EC2 g4dn.xlarge with Deep Learning Base AMI (Ubuntu 18.04) Version 32.0.
The good part about the AMI is that you have CUDA ready. What I do is to create a new conda environment for HF, then I install PyTorch 8 (checking the CUDA version with the command nvidia-smi) and HuggingFace in the new env simply by using pip.
Let me know if you have any more questions.