I don’t think ### 2 is a valid operation as you are manipulating the weigths inplace after using them, which would update the weights and make the forward activations stale. Could you create a new parameter after ### 2 which is used in the needed operations?