Hi,
Yes it will work.
Be careful though that if you do bias=nn.Parameter(torch.zeros(512,)).repeat(batch,1), the python vairable bias will not contain the original Parameter object and is not the tensor that will be optimized.
You would need to do:
bias_for_optimizer = nn.Parameter(torch.zeros(512,))
bias = bias_for_optimizer.repeat(batch, 1)