He graduated from Carmel High School in Carmel, Maine in 1955 and attended the same college as his father, Bates College. After graduating, he returned to Penobscot County and began teaching at Hermon High School. He continued teaching in the district until 1971. In 1977, while serving as Commissioner of Education, Millett and his wife bought a home in Manchester, Maine to cut down on his daily commute for work in Augusta, but the couple disliked living in a subdivision so much that he and his wife bought 25 acres of land next to his parents farm in Waterford and built a house to live in. He could barely stand to address the Maine Senate during his 2011 confirmation hearing due to an infection after knee replacement surgery. Shortly after being appointed commissioner, Millett had to miss work due to a second infection in his knee; he was temporarily replaced by Ryan Low, who had served as Commissioner of Finance under Democrat John Baldacci. Low said of Millett "I would do just about anything for him...There's no one more of an expert on budgeting and legislative process; but even more than that, he's a genuinely nice person."