Example 1 Given that $p$ is a prime number, find all integer pairs $(x, y)$ such that $|x+y|+(x-y)^{2}=p$.
Notice that, $x+y$ and $x-y$ are either both odd or both even, so $|x+y|+(x-y)^{2}$ is even, which implies $p=2$. This indicates that $|x+y|+(x-y)^{2}=2$.

Since $|x+y|$ and $(x-y)^{2}$ have the same parity, and $(x-y)^{2}$ is a perfect square, we have $\left(|x+y|,(x-y)^{2}\right)=(2,0),(1,1)$. Solving these respectively, we find
$$(x, y)=(1,1),(-1,-1),(0,1),(0,-1),(1,0),(-1,0) .$$