A semiconductor memory device having a three-dimensional structure has been proposed in which memory holes are made in a stacked body in which multiple electrode layers are stacked, and a charge storage film and a semiconductor film are provided to extend in the stacking direction of the stacked body inside the memory hole. The memory device includes a plurality of memory cells connected in series between a drain-side selection transistor and a source-side selection transistor. A drain of a memory cell is electrically connected via the drain-side selection transistor to a bit line. A source of the memory cell is electrically connected via the source-side selection transistor to a source line. The source line is generally provided in a slit which is formed in the stacked body. In contrast, a buried source line has been studied, which is buried under the stacked body. However, in the case where the buried source line is implemented, it is necessary to provide a local interconnection, which supplies a bias to a substrate, in every finger, for example. Thereby, the reduction of a plane size of a memory cell array is interfered. Thus, the plane size of the memory cell array is desired to be reduced, even if the buried source line is implemented.