I write this for someone who is going to do experience in future.
I have tested unfreezing on my dataset but it seems going to be overfitted.
Epoch 4:
trainning loss = .0003 , acc ~ 95%
validation loss = 4.3 , acc ~ 40%
So, I am going to try next option, train a BERT model.