The decade-long campaign eventually wore Phan down, and he died from dysentery as the French surrounded his forces. Phan was born in the village of Đông Thái in the northern central coast province of Hà Tĩnh. Đông Thái was famous for producing high-ranking mandarins and had been the home of senior imperial officials since the time of the Lê Dynasty. Twelve consecutive generations of the Phan family had been successful mandarinate graduates. All three of Phan's brothers who lived to adulthood passed the imperial examinations and became mandarins. Early on, Phan indicated his distaste for the classical curriculum required of an aspiring mandarin. He nevertheless persevered with his studies, passing the regional exams in 1876 and then topping the metropolitan exams the following year. In his exam response, Phan cited Japan as an example of how an Asian country could make rapid military progress given sufficient willpower. Phan was never known for his scholarly abilities; it was his reputation for principled integrity that led to his quick rise through the ranks under the reign of Emperor Tự Đức. He was first appointed as a district mandarin in Ninh Bình Province, where he punished a Vietnamese Roman Catholic priest, who, with the tacit support of French missionaries, had harassed local non-Catholics. Amid the diplomatic controversy that followed, he avoided blaming the unpopular alliance between Vietnamese Catholics and the French on Catholicism itself, stating that the partnership had arisen out of the military and political vulnerabilities of Vietnam's imperial government. Despite this, the Huế court eventually removed Phan from this post. Phan was transferred to the Huế court as a member of the censorate, a watchdog body that monitored the work of the mandarinate. He earned the ire of many of his colleagues, but the trust of the emperor, by revealing that the vast majority of the court mandarins were making a mockery of a royal edict to engage in regular rifle practice. Tự Đức later dispatched Phan on an inspection trip to northern Vietnam. His report led to the ousting of many officials who were deemed corrupt or incompetent, including the viceroy of the northern region. He rose to become the Ngu Su, or Imperial Censor, a position which allowed him to criticise other high officials and even the emperor for misconduct. Phan openly criticised Tôn Thất Thuyết, the foremost mandarin of the court, believing him to be rash and dishonest.