Growing up with a black father and white mother (from Glasgow, Scotland), Butler at times had difficulty fitting in with a lot of the white children in his grade school, as they would sometimes ask why his mother was white and father was not.