If I understand your use case correctly, you would like to use a Toeplitz matrix as the weight parameter in all linear layers and would like to keep the constraint during the training?
Would each layer use a square matrix via in_features=out_features so that the trainable parameters would be n+1 or any shape?
I think one valid approach could be to initialize the weight as an nn.Parameter in your custom linear layer implementation, create the Toeplitz matrix during the forward pass, and apply the operation using the functional API via F.linear(input, weight).