You could probably indeed detach the output of fc1, since you won’t calculate gradients for it.
x = np.heaviside(self.fc1(x).detach().numpy(), 0)
x = torch.from_numpy(x)

Alternatively, you could also implement np.heaviside in PyTorch.