Emperor Ruizong did, and he made Zhong the minister of census (戶部尚書, "Hubu Shangshu"), and later the prefect of Shu Prefecture (蜀州, in modern Chengdu, Sichuan). In 712, Emperor Ruizong passed the throne to Li Longji, who took the throne as Emperor Xuanzong. Sometime after, Zhong Shaojing was recalled to again serve as minister of census, and later the head of the crown prince's household (太子詹事, "Taizi Zhanshi"). (As there was no crown prince at that time, the title was entirely honorary.) However, the chancellor Yao Chong disliked Zhong, and he accused Zhong and Liu Youqiu of complaining that they were not given important positions despite their contributions. Despite their denials, they were still demoted—in Zhong's case, to be the prefect of Guo Prefecture (果州, roughly modern Nanchong, Sichuan). At a later point, he was further accused of offenses and demoted to be the sheriff of Huai'en County (懷恩, in modern Zhangzhou, Fujian), and his titles were stripped. He subsequently served as the secretary general of Wen Prefecture (溫州, roughly modern Wenzhou, Zhejiang). In 727, Zhong visited the capital Chang'an, and had the opportunity to meet Emperor Xuanzong. He wept and stated:  Emperor Xuanzong was saddened, and immediately made Zhong "Yinqing Guanglu Daifu" and "Taizi You Yude" (), a member of the staff of the crown prince Li Hong. Some time thereafter, he was made "Taizi Shao Zhanshi" (), the deputy head of the crown prince's household. He died in his 80s while still serving in that position, although the year is not known. It was said that Zhong, in addition to being a famed calligrapher, also favored collecting calligraphy. Among his collections were several hundred works of Wang Xizhi, Wang Xianzhi, and Chu Suiliang. Three of his sons, Zhong Jiabi (), Zhong Jia'e (), and Zhong Jiawei () served as officials.