What is the determinant of the matrix $A^2+B^2$ 
Let $A$ and $B$ be two $n\times n $ matrices such that $A\neq B$, $A^3=B^3$, and $A^2B=B^2A,$  then what is the value of $\det(A^2+B^2)$ ? 

My attempt: $A^3=B^3 \Rightarrow (A-B)(A^2+AB+B^2)=0 \Rightarrow A^2+AB+B^2=0$
but I can't get any idea

First a couple of notes on your attempt:
$$(A-B)(A^2+AB+B^2) = A^3+A^2B+AB^2-BA^2-BAB-B^3$$
and if $A$ and $B$ do not commute, it doesn't need to be $A^3-B^3$. You have to be careful about factorization in non-commutative rings, familiar formulas do not always work.
Even it were true, as I already said in comments, $$((A-B)(A^2+AB+B^2) = 0\ \wedge\ A\neq B)\ \nRightarrow\ A^2 +AB + B^2 = 0$$
This is because there are zero divisors in matrix algebra (there are examples in the link).

Now, to the solution:
$$(A^2+B^2)A = A^3 + B^2A = B^3 +A^2B = (A^2 + B^2)B \implies (A^2 + B^2)(A-B) = 0$$
Now, if $\det(A^2 + B^2) \neq 0$, then $A^2 + B^2$ is invertible, but then $A-B = 0$ which is contradiction. Hence, $\det (A^2 + B^2) = 0$.
