Use the model as a sequence classifier
Hello, I was wondering if you intend on supporting sequence classification like you do for Gemma 2 using "Gemma2ForSequenceClassification".
Do you intent to release "Gemma3ForSequenceClassification"? Because as it stands now, Gemma 2's sequence classifier won't work with Gemma 3 when I try to finetune a Gemma 3 model for sequence classification. Is there any workaround that I can do to allow it to work?
I did this temporary workaround, in the "modeling_gemma2.py", I import the Gemma 3 text model class as follows:
from ..gemma3.modeling_gemma3 import Gemma3TextModel
and then in "Gemma2ForSequenceClassification" class in the same file, I change this line:
self.model = Gemma2Model(config)
to
self.model = Gemma3TextModel(config)
I did this temporary workaround, in the "modeling_gemma2.py", I import the Gemma 3 text model class as follows:
from ..gemma3.modeling_gemma3 import Gemma3TextModeland then in "Gemma2ForSequenceClassification" class in the same file, I change this line:
self.model = Gemma2Model(config)
to
self.model = Gemma3TextModel(config)
It's good.
But is the result of Gemma2ForSequenceClassification normal as a reward model?
Hi ,
You're right—until recently, there wasn't official support for sequence classification with Gemma 3, and workarounds like the one you mentioned (swapping Gemma2Model with Gemma3TextModel) were necessary.
The model will be mostly same as Gemma2 Sequence Classification, with the only difference that it will accept pixel values as input. That means we need a base model for multimodal Gemma3, which adding here (#37033). After that can add Gemma3ForSequenceClassification.
Kindly follow this reference issue . if you have concerns let us know will assist on this.
Thank you.