Check out the function exp_lr_scheduler in my fine tuning tutorial linked below. That lets you decay the LR. I just multiply it by a constant but you can do anything fancy you want using the same code structure.

  
      github.com
  
  
    Spandan-Madan/Pytorch_fine_tuning_Tutorial/blob/master/main_fine_tuning.py 299
### Section 1 - First, let's import everything we will be needing.

from __future__ import print_function, division
import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
import numpy as np
import torchvision
from torchvision import datasets, models, transforms
import matplotlib.pyplot as plt
import time
import copy
import os
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
from fine_tuning_config_file import *

## If you want to keep a track of your network on tensorboard, set USE_TENSORBOARD TO 1 in config file.



  This file has been truncated. show original