Hi everyone! Currently, I’m doing image segmentation task with U-net architecture, but the result not that promising, so I came across cyclical learning rates which gives high performance and more efficient. I found that CLR is built-in pytorch but I’m not sure that I used it in a correct way? Here is my code:
optimizer= torch.optim.SGD(filter(lambda p: p.requires_grad ,model.parameters()),lr=1e-1,momentum =0.9)
scheduler = torch.optim.lr_scheduler.CyclicLR(optimizer,base_lr =1e-3,max_lr = 0.1)
criterion= nn.CrossEntropyLoss()
Training loop:
epochs = 0
for i in range(30):
scheduler.step()
for img,mask in loader:
model.train()
optimizer.zero_grad()
img = img.to(device,dtype=torch.float)
mask = mask.to(device,dtype=torch.float)
preds = model(img)
loss =criterion(preds,mask.long())
loss.backward()
optimizer.step()