In this video, we're going to see how to load and fine-tune a pre-trained model. It's very quick, and if you've watched our pipeline videos, which I'll link below, the process is very similar. This time, though, we're going to be using transfer learning and doing some training ourselves, rather than just loading a model and using it as-is. To learn more about transfer learning, head to the 'What is transfer learning?' video, which we'll link below too! So now let's look at this code. To start, we pick which model we want to start with - in this case we're going to use the famous, the original BERT. But what does this monstrosity, 'TFAutoModelForSequenceClassification' mean? Well, the TF stands for TensorFlow, and the rest means "take a language model, and stick a sequence classification head onto it if it doesn't have one already". So what we're going to do here is load BERT, a general language model, and then do some transfer learning to use it on our task of interest. We load the language model with this one line of code here, using the "from_pretrained" method. That method needs to know two things: Firstly the name of the model you want it to load, and secondly how many classes your problem has. If you want to follow along with the data from our datasets videos, which I'll link below, then you'll have two classes, positive and negative, and thus num_labels equals two. What about this "compile" thing? If you're familiar with Keras, you've probably seen this already, but if not, this is one of its core methods - you always need to "compile" your model before you train it. Compile needs to know two things: Firstly, the loss function - what are we trying to optimize? Here, we import the sparse categorical crossentropy loss function - that's a mouthful, but it's the standard loss function for any neural network that's doing a classification task. It basically encourages the network to output large values for the right class, and low values for the wrong classes. Note that you can specify the loss function as a string, like we did with the optimizer, but there's a very common pitfall there - by default, this loss assumes the output is probabilities after a softmax layer, but what our model has actually output is the values before the softmax, often called the "logits" - you saw these before in the videos about pipelines. If you get this wrong, your model won't train and it'll be very annoying to figure out why. In fact, if you remember absolutely nothing else from this video, remember to always check whether your model is outputting logits or probabilities, and to make sure your loss is set up to match that. It'll save you a lot of debugging headaches in your career! The second thing compile needs to know is the optimizer you want. In our case, we use Adam, which is sort of the standard optimizer for deep learning these days. The one thing you might want to change is the learning rate, and to do that we'll need to import the actual optimizer rather than just calling it by string, but we'll talk about that in another video, which I'll link below. For now, let's just try training the model! So how do you train a model? Well, if you’ve used Keras before, this will all be very familiar to you - but if not, let's look at what we're doing here. Fit() is pretty much the central method for Keras models - it tells the model to break the data into batches and train on it.  So the first input is tokenized text - you will almost always be getting this from a tokenizer, and if you want to learn more about that process, and what exactly the outputs look like, please check out our videos on tokenizers - there'll be links below for those too! So that's our inputs, and then the second input is our labels - this is just a one-dimensional Numpy or Tensorflow array of integers, corresponding to the classes for our examples, and that’s it. If you're following along with the data from our datasets video, there'll only be two classes, so this will just be zeroes and ones. Once we have our inputs and our labels, we do the same thing with the validation data, we pass the validation inputs and the validation labels in a tuple, then we can, if we want, specify details like the batch_size for training, and then you just pass it all to model.fit() and let it rip. If everything works out, you should see a little training progress bar as your loss goes down. And while that's running you call your boss and tell him you’re a senior NLP machine learning engineer now and you’re going to want a salary review next quarter. This is really all it takes to apply the power of a massive pretrained language model to your NLP problem. Could we do better, though? We certainly could, with a few more advanced Keras features like a tuned, scheduled learning rate we can get an even lower loss, and an even more accurate model. And what do we do with our model once it's trained? I'll cover this and more in the videos linked below!