Solved by Rocketknight1 in post #2 
              
              
                Hi! The problem here is that your config object has default layer numbers and sizes that are totally different from the ones in albert-base-v2. If you’d like to train a sequence classification model on top of Albert, you can just do: 
model = TFAlbertForSequenceClassification.from_pretrained('albert…