Hello everyone,
Hope you are having a productive day.
I am trying to build a model which has

sparse connections between layers
some weights that should not be updated

For example,
self.L_vc = nn.Linear(E, E, bias=False)
torch.nn.init.ones_(self.L_vc.weight)
for p in self.L_vc.parameters():
    p.requires_grad = False
prune.custom_from_mask(self.L_vc, name='weight', mask=vc_mask)

Here, I specify the connectivity between the neurons using the tensor vc_mask and preventing the weights from being updated by setting p.requires_grad = False. I also set bias = False since I just want L_vc to specify the connectivity just like an adjacency matrix. I am not sure if this is the correct way to do that.
Am I interrupting the gradients from propagating backwards by setting a constant layer like this? What should I expect if I have a layer like this in the middle of the model? Any help would be highly appreciated.