The lines self.binary = binary_activate, and then the usage later, x = self.binary(x), is what is causing the problem. You are saving one instance of your binary_activate class and reusing that one instance every time you forward through Model1. Instead, create a new instance of binary_activate on each forward, i.e. get rid of self.binary and replace the line using it in forward with x = binary_activate()(x).