scheduler. The default optimizer used by the [Trainer] is [AdamW]:
```python
from transformers import AdamW
optimizer = AdamW(model.parameters(), lr=5e-5)
```
Finally, the learning rate scheduler used by default is just a linear decay from the maximum value (5e-5 here) to 0:
```python
from transformers import get_scheduler
num_epochs = 3
num_training_steps = num_epochs * len(train_dataloader)
lr_scheduler = get_scheduler("linear", optimizer=optimizer, num_warmup_steps=0, num_training_steps=num_training_steps)
```
One last thing, we will want to use the GPU if we have access to one (otherwise training might take several hours
instead of a couple of minutes). To do this, we define a device we will put our model and our batches on.
```python
import torch
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
