Solved by Nikronic in post #2 
              
              
                Hi, 
About the last line, you need to initialize it like the way you have initialized your weights. In other words, use self.pi.bias instead of self.pi which is the module. 
Also, nn.init.constant_(self.pi, 0) initialized the entire bias tensor with zeroes while the TF counterpart has initialized it…