 Anderson and his wife, Isabel, spent the next twenty-five years traveling extensively at home and abroad; collecting memorabilia and decorative arts; expanding the mansion and gardens of their summer home "Weld" in Brookline, MA, now the Larz Anderson Park; funding the construction of the Anderson Memorial Bridge across the Charles River in Boston and Cambridge, MA; and funding the construction and interior decoration of the Lady Chapel of the Washington National Cathedral.According to sources cited by his biographer, Anderson's diplomatic record was an embarrassment to President William Howard Taft; the Senate Foreign Relations Committee refused to confirm him as U.S.Ambassador to Japan in 1913 even after he had assumed his post there.