I is possible that when you unpack your network outputs, you create multiple tensors that share the same underlying storage and so doing inplace operations become forbidden.
For example when doing x_dot_wT, phi_theta = output.