 Although Scott's family held considerable wealth, most of the family fortune went to his older brother, James.In 1805, Scott began attending the College of William and Mary, but he soon left in order to study law in the office of attorney David Robinson, where his contemporaries included Thomas Ruffin.While apprenticing under Robinson, he attended the trial of Aaron Burr, who had been accused of treason for his role in events now known as the Burr conspiracy.