Plotting the decision boundary of a ReLU 2-layer network I have a 2-layer fully connected ReLU network with weights and biases $W_1 = [[-4, 1], [2, 3]], b_1 = [3, -2], W_2 = [[2, 3]], b_2 = -2$. The ReLU non-linearity is only between the layers, while the output is in the range $(-\infty, \infty)$.
Now, I would like to draw the decision boundary of this network. After applying all the weights, biases and the activation function on an input $(x_1, x_2)$, I end up with the following expression:
$2max(0, -4x_1+ x_2 + 3) + 3max(0, 2x_1 + 3x_2 -2) -2$
After setting the expression to zero, I try to come up with a way to plot the decision boundary in the $x_1, x_2$ plane, but I am stuck. Apart from trying to guess numbers $x_1 $ and $x_2$ that evaluate the expression to zero, is there a systematic way to plot the decision boundary?

There are four regions depending on the +/- status of the lines $-4x_1+x_2+3=0$ and $2x_1+3x_2-2$. You'll evaluate the expression for these four regions separately. For example, for the region where both lines give positive values, you'll end up with the boundary line:
$$2(-4x_1+x_2+3)+3(2x_1+3x_2-2)-2=-2x_1+11x_2-2$$
Evaluate and draw the boundaries with these four regions.
