Dear Forum,
Is it possible to create a scalar that is only trained during n-amount of batches? I would create my scalar alpha (init. at 1) through:
alpha = nn.Parameter(torch.ones(1, requires_grad=True))
Am I correct to assume that the value of alpha then changes during training, e.g. becomes 0.9? Or did I set this up incorrectly.
How can I avoid alpha being trained during .step(), and only train it every nth Batch? Intuitively I would model.alpha.requires_grad = False during other batches. Then, every nth Batch.
for param in model.parameters():
    param.requires_grad = False
model.alpha.required_grad = True

Thanks in advance for your help.