if he had realized anything today , it was the fact that his brother had his own secrets , a woman in his past -- very likely maddie , judging by sam 's strange reaction to the curvy redhead -- who still haunted him .