Question: Let $S=\mathbb{R}\setminus \{-1\}$. Define $\*$ on $S$ by $a\*b=a+b+ab$.

This feels fairly straightforward but it is just the details of which approach is valid that is throwing me. My attempt:

We first check for associativity by considering $a\*(b\*c)$

$a\*(b\*c)=a+(b+c+bc)+a(b+c+bc)=a+b+c+bc+ab+c+abc=(a+b+ab)+c+(a+b+ab)c=(a\*b)\*c$

We now look for an inverse. Consider $(a\*b)=a \Rightarrow a+b+ab=a \Rightarrow b=0$ and so $0$ is the identity element.

Now the trouble starts for me. We search for an inverse giving $(a\*a^{-1})=0=a+a^{-1}+aa^{-1}$ Coming at this I am always defining an inverse in terms of itself. I have written out some integer cases but I am not sure how to apply them in general.

Searching for closure is also problematic for me. We essentially want to show that taking $x,y \in S$ then $x+y+xy\neq -1$ but I run into similar problems as above. Thanks for the help!

Answer: 
Suppose $a\*b=0=ab+a+b$ then $$(a+1)b=-a$$ so that $$b=-\frac a{a+1}$$is uniquely defined, and you can see why you need to exclude the value $1$ in the definition.
About the closure:
\begin{align}
x+y+xy = -1 &\iff 1+x+y+xy = 0 \\
& \iff (1+x)(1+y)=0 \\
& \iff 1+x=0\lor 1+y=0 \\
& \iff x=-1\lor y=-1
\end{align}