He generally practiced as a lawyer's lawyer, almost entirely before the U.S. Supreme Court, and appeared in over 400 cases. Phillips died January 14, 1884 in Washington, D.C. and is buried in Laurel Grove Cemetery in Savannah, Georgia. He achieved a solid reputation as a thoughtful moderate among the leading national figures of his day. Being a Southern Unionist, he had the opportunity, according to at least one biographer, to have had a career comparable to the career of U.S. President Andrew Johnson. This was complicated by his religion, and it was made impossible by the unrestrained activities of his wife.