Solved by philschmid in post #2 
              
              
                Hey @kmfoda, 
If you want to use private models you also need to define HF_API_TOKEN in the hub dictionary. More here GitHub - aws/sagemaker-huggingface-inference-toolkit.