I got the answer. There is a formula which describes the output shape:
Lout=floor((Lin+2∗padding−dilation∗(kernel_size−1)−1)/stride+1)
Therefore, in order to keep the dimension, we need to add paddings which equals the dilation size.