Just to be sure before I write something long that is not what you asked:

You have an nn.Module (lets call it one_step_module) that does one step given a current state, an input and produce an output
You have another nn.Module (lets call it loss_module) that given the output and a target gives you the loss for this output.

And in pseudo code, what you want is:
state = init_state
for i, (inp, target) in enumerate(my_very_long_sequence_of_inputs):
    output, state = one_step_module(inp, state)
    if (i+1)%k1 == 0:
        loss = loss_module(output, target)
        # You want the function below
        loss.backward_only_k2_last_calls_to_one_step_module()