I might have misunderstood the code, as you are using a loop over target_len (apparently for the RNN) as well as gradient accumulation (for multiple batches).
I’m not sure if code is missing, but note that inside step it seems you are using the same decoder_input for all steps. This might of course fit your use case, as I’m not familiar with it.
Looking at the amp-specific part: it looks correct, assuming that your gradient accumulation code works fine for the RNN use case.