num_warmup_steps = 100
warmup_proportion = float(num_warmup_steps) / float(num_training_steps)  # 0.1
Previously BertAdam optimizer was instantiated like this:
optimizer = BertAdam(
    model.parameters(),
    lr=lr,
    schedule="warmup_linear",
    warmup=warmup_proportion,
    num_training_steps=num_training_steps,
)
and used like this:
for batch in train_data:
    loss = model(batch)
    loss.backward()
    optimizer.step()
In 🤗 Transformers, optimizer and schedules are split and instantiated like this:
optimizer = AdamW(
    model.parameters(), lr=lr, correct_bias=False
)  # To reproduce BertAdam specific behavior set correct_bias=False
scheduler = get_linear_schedule_with_warmup(
    optimizer, num_warmup_steps=num_warmup_steps, num_training_steps=num_training_steps
)  # PyTorch scheduler
