I’m using lr_scheduler on 28540 training data. For an experiment I’m training the data twice and testing it once per epoch.
optimizer = optim.AdamW(model.parameters(), hparams['learning_rate'])
    criterion = nn.CTCLoss(blank=28).to(device)
    scheduler = optim.lr_scheduler.OneCycleLR(optimizer, max_lr=hparams['learning_rate'],
                                              steps_per_epoch=int(len(train_loader)),
                                              epochs=hparams['epochs'],
                                              anneal_strategy='linear')

for epoch in range(0, epochs):        
        train(model, device, train_loader, criterion, optimizer, scheduler, text_transform, epoch)
        FeaScatter_Train(model, device, train_loader, criterion, optimizer, scheduler, 0.008,1,0.008, text_transform, epoch)
        best_wer = test(model, device, test_loader, criterion, epoch, text_transform, 'standard_')

In side the train method I used the scheduler as follows for the both the method. Here, I am showing only one of the train method
model.train()
    train_loss = 0
    iterator = tqdm(train_loader)
    
    for batch_idx, _data in enumerate(iterator):
        spectrograms, labels, input_lengths, label_lengths = _data 
        spectrograms, labels, model = spectrograms.to(device), labels.to(device), model.to(device)

        optimizer.zero_grad()

        output = model(spectrograms)  # (batch, time, n_class)
        output = F.log_softmax(output, dim=2)
        output = output.transpose(0, 1) # (time, batch, n_class)
        
        loss = criterion(output, labels, input_lengths, label_lengths).to(device)
        loss.backward()
        
        optimizer.step()
        scheduler.step()
        train_loss += loss.item()

I can see there is a relation between the training sample and the scheduler but I can’t seem to solve it.