You can custom a function to change the parameters of optimizer as the training goes.
def adjust_optim(optimizer, n_iter):
    if n_iter == 1000:
        optimizer.param_groups[0]['betas'] = (0.3, optimizer.param_groups[0]['betas'][1])
    if n_iter > 1000:  
        optimizer.param_groups[0]['lr'] *= 0.9999

If you are using multiple params groups, change 0 to i for ith group.
Best,