I was going through the following information 4 on reducing learning rates in PyTorch to really low value like 1e-9.
And I am amazed why doing  loss = loss/100 is equivalent to reducing learning rate by 100? The full snippet is below.
outputs = model(batch)
loss = criterion(outputs, targets)

# Equivalent to lowering the learning rate by a factor of 100
loss = loss / 100

self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()

Can someone please help me on this.
Thanks a lot