google/gemma-3-4b-it · Deployment issues on sagemaker

Apr 10

I am trying to deploy on AWS sagemaker using the above example.
But after running it I get a failure message
Cloudwatch said that the model type gemma3 is Unsupported
Is there a solution?

lkv

Google org 8 days ago

Hi @tychen6677 , Apologies for late reply,

Gemma 3 models on SageMaker has been a known problem due to compatibility issues with earlier versions of the Hugging Face transformers library, which SageMaker relies on.

To use Gemma-3 models, you need the latest development version of the Transformers library (4.50.0 or higher).. A new stable version of Transformers is now available which is compatible to Gemma3. Please update it using pip install -U transformers and try again. Let us know if this helps!

Thank you.

pagezyhf

8 days ago

Hello @tychen6677 , @juanjucm tested with the latest TGI DLC and it should work fine now! make sure you upgrade sagemakersdk to get the latest dlc uri.
Have a good day,
Simon