AutoModelForSequenceClassification not learning if model is initialized inside a function scope
Hey,

I've made a simple function to initialize a bert model for sequence classification, an optimizer and a scheduler, but my model wasn't learning anything. After a lot of debugging, I found out that if I execute the same piece of code OUT of the function scope, the model learns.

```
def initialize_model(pretrained_bert_name, device, num_labels, epochs, train_dataloader):
    """Initialize the Bert Classifier, the optimizer and the learning rate scheduler.
    """
    # Instantiate Bert Classifier
    model = AutoModelForSequenceClassification.from_pretrained(
        pretrained_bert_name,
        num_labels=num_labels)

    # Sends model to GPU if inabled
    model.to(device)

    # Create the optimizer
    optimizer = torch.optim.AdamW(model.parameters(), lr=5e-5)

    # Total number of training steps
    total_steps = len(train_dataloader) * epochs

    # Set up the learning rate scheduler
    scheduler = get_scheduler(
        "linear",
        optimizer=optimizer,
        num_warmup_steps=10000,
        num_training_steps=total_steps
    )
    return model, optimizer, scheduler
```


I'm thinking that maybe sending the model to device, or maybe passing its weights to the optimizer may create a different copy when the function returns, idk. Thoughts?

Hey @JAugusto97, can you provide a full script or a reproducible Colab notebook? Reading the code I don't see why it shouldn't work. <|||||>It might have been global variables interfering somewhere, when I tried to reproduce in a new notebook it worked 🤔 
Thanks anyway!