Windows are slightly "transparent" in such a way I can see underlying windows show through slightly to the current window I don't know how to explain it too well. Here are some places I see this happen:
On the Minecraft main menu, I can see through to the desktop,
While watching a youtube video (in full screen) I can see remnants of the previous page,
etc.

How the heck do I fix this? It is really bugging me.

EDIT: It appears to be because of the hardware, since this also happens under Ubuntu.
That's one of the "features" of the Mac O/S that I always thought, "what were they thinking?" Unfortunately I don't have an answer to that but to your question:

Go to System Preferences -> Accessibility -> Display

look for two check boxes: Increase Contrast and Reduce Transparency.

Adjust either or both to get your desired effect, that is not seeing the stuff behind a window.