Doyle was an amateur boxer.[53] In 1909, he was invited to referee the James Jeffries–Jack Johnson heavyweight championship fight in Reno, Nevada. Doyle wrote: "I was much inclined to accept ... though my friends pictured me as winding up with a revolver at one ear and a razor at the other.