according to his father boxers held such an advantage over a non-boxer opponent in a street brawl that his father absolutely forbade him to use his fists to settle disputes , unless in extreme cases of self-defense .