Hi,
Won’t the non-linearities prevent you from doing it?
Even without non-linearities, you could do it for evaluation, but the behavior for training would be different as you would have different degrees of freedom for your weights.