update:
I replace pl.Trainer with raw code and here is my trainer code:

model = SpeechRecognitionModel(vocab_size=tokenizer.vocab_size)
optimizer = optim.AdamW(model.parameters(), 5e-4)
NAN = lambda x: torch.isnan(x).any()
loss = nn.CTCLoss(zero_infinity=True)
scheduler = optim.lr_scheduler.OneCycleLR(optimizer, max_lr=5e-4, 
                                            steps_per_epoch=int(len(speech_recognition_dataloader)),
                                            epochs=1,
                                            anneal_strategy='linear')
    

for batch_idx, batch in enumerate(speech_recognition_dataloader):
    mfccs, mfccs_lengths, labels, label_lengths = batch
    
    optimizer.zero_grad()

    output = model(mfccs)
    output = output.permute((-1, 0, 1))
    probs = nn.functional.log_softmax(output, dim=-1)
    _loss = loss(probs, labels, mfccs_lengths, label_lengths)
    _loss.backward()
    optimizer.step()
    scheduler.step()

    print(f"iter: {batch_idx}, input_data is nan: {NAN(mfccs)}, loss: {_loss}, output is nan: {NAN(output)}")

here is what is get:
iter: 0, input_data is nan: False, loss: 19.476131439208984, output is nan: False
iter: 1, input_data is nan: False, loss: nan, output is nan: True
iter: 2, input_data is nan: False, loss: nan, output is nan: True
iter: 3, input_data is nan: False, loss: nan, output is nan: True
iter: 4, input_data is nan: False, loss: nan, output is nan: True

so right now, it’s clear loss computed for first iteration but weights became nan for next iteration, is this optimizer problem?
Thanks