Hello,
I use Adam Optimizer for training my network but when I print learning rate I realized that learning rate is constant and doesn’t change. Adam optimizer is adaptive learning rate algorithm but I don’t see such behavior in results.
    if optimizer_name == 'SGD':
        # optimizer
        optimizer = torch.optim.SGD(
            params=params,
            lr=optimizer_arguments['lr'],
            momentum=optimizer_arguments['momentum'],
            weight_decay=optimizer_arguments['weight_decay'])
    elif optimizer_name == 'Adam':
        # Adam optimizer
        optimizer = torch.optim.Adam(
            params=params,
            lr=optimizer_arguments['lr'],
            weight_decay=optimizer_arguments['weight_decay'])
    else:
        raise ValueError('Optimizer name is incorrect!')