I would have to do something like this:

def forward(self, x):
x = np.heaviside(self.fc1(x),1)
x = x.detach()
x = F.tanh(self.fc2(x))
x = self.fc3(x)
return x

This error appears:
RuntimeError: Can’t call numpy() on Variable that requires grad. Use var.detach().numpy() instead.