He was the son of a wealthy landowner, but joined Chiang Kai-shek's National Revolutionary Army, against his parents' wishes, in 1924. When the Kuomintang began to fight the Communists in 1927, Xu left Chiang's forces and led a Communist army based in Sichuan under the political authority of Zhang Guotao. After Zhang was purged in the early 1930s, Xu survived politically and rejoined the Red Army, in a less senior position, under the leadership of Mao Zedong.