In 617, the Sui general Li Yuan the Duke of Tang rebelled at Taiyuan and advanced toward Chang'an. When his army arrived at Jiang Commandery, Chen Shuda surrendered the commandery to his forces, and he was made a secretary on Li Yuan's staff and created the Duke of Handong, sharing important secretarial duties with Wen Daya (). It was said that Li Yuan's military correspondences, as well as edicts later issued in the name of Yang You (the grandson of Emperor Yang, whom Li Yuan declared emperor after capturing Chang'an later in 617) announcing general pardons and, later, passing the throne to Li Yuan in 618, were largely written by Chen Shuda. After Li Yuan assumed the throne in 618, establishing Tang Dynasty as its Emperor Gaozu, Chen Shuda was made the "Huangmen Shilang" (), the deputy head of the government's examination bureau (門下省, "Menxia Sheng"). In 619, Emperor Gaozu made him "Nayan" () -- the head of the examination bureau, a post considered one for a chancellor. In 621, after the title of the head of the examination bureau was changed to "Shizhong" (), he continued to serve as "Shizhong". It was said that Chen was well-spoken and handsome in appearance, and whenever he made an appearance before the emperor, the gentlemen at the capital were all looking at him. When famed people from south of the Yangtze River (i.e., the former Chen territory) visited Chang'an, he often recommended them for government service. In 630, Emperor Gaozu created him the greater title of Duke of Jiang. At one imperial feast, when grapes were served, Chen, instead of eating the grapes, held them in his hand, and when Emperor Gaozu asked him why, he responded, "My mother has a mouth ailment, and I have wanted to get her grapes but could not get them. I want to hold them and take them back to my mother." Emperor Gaozu, who thus was reminded of "his" mother, stated, "It is a good thing you have a mother to take things back to." He rewarded Chen with silk. Late in Emperor Gaozu's reign, an intense rivalry developed between his two oldest sons, Li Jiancheng the Crown Prince and Li Shimin the Prince of Qin, as Li Jiancheng, while a capable general himself, was overshadowed by Li Shimin, who had destroyed Tang's major enemies Xue Rengao the Emperor of Qin, Liu Wuzhou the Dingyang Khan, Wang Shichong the Emperor of Zheng, and Dou Jiande the Prince of Xia.