Leicester-born Smith, 28, has only played in four first-team matches so far this season and has decided to pursue a career outside of cricket.
Former England Under-19 international Smith, who joined from Leicestershire prior to the 2015 season, said: "It feels like the right time for me to move on from cricket."
Smith scored almost 5,000 first-class runs with 2,000 more in one-day games.
Director of cricket Mick Newell said he was sad to see Smith leave.
"He's found it a struggle to get into the first team this season and we respect that he now considers he is ready to begin a new chapter in his life," Newell said.
"He has played some important innings in his time with us, particularly in white-ball cricket. We wish him every success in whatever he chooses to do next."
Nottinghamshire batsman Greg Smith has retired from professional cricket.