The parameters() in the per-parameter options example 227 be any iterables for the parameters, so if you had a list of parameters_per_step and a list of learning_rates_per_step, you can do optim.SGD([{'params': p, 'lr': l} for p,l in zip(parameters_per_timestep, learning_rates_per_step)]) or somesuch.
Best regards
Thomas