No, in this case, you could go for the workaround described above (multiply the .grad with something (say torch.exp(-delta*torch.arange(0,4)).unsqueeze(1) if you have the example from your other thread). For SGD this will be the same as having learning rates smaller for the later rows of the weight.
Or you could first define separate tensors and torch.cat/torch.stack them.
Best regards
Thomas