both of waller's ancestral families enjoyed wealth and political distinction in england and america. the wallers were high sheriffs of kent, where the family owned groombridge place, and judiciaries in buckinghamshire. littleton waller's ancestor col. john waller came to virginia about 1635. he trained as an attorney at the college of william and mary and founded a family that included several members of the virginia house of burgesses, a justice of the virginia supreme court, and a member of virginia's delegation to the committee that adopted the declaration of independence. littleton waller's ancestor benjamin waller was a noted colonial attorney of williamsburg, virginia.