I don’t think the GPUs will be more than T4s on AWS SageMaker either (and only one GPU not several), which is why we recommend projects that involve fine-tuning pretrained models more than pretraining.