How are you changing the model architecture based on this adj_matrix?
Are you using this matrix directly or are you at some point resetting the model parameters?
PS: You can add code snippets using three backticks ```