Roughness is an important parameter to describe the microtopography of target surface. In the field of roughness detection, constraints on traditional methods are significant. Meanwhile, polarization imaging technology is gradually mature in recent years. In this paper, a method of roughness estimation with Mueller matrix is presented. Battery of lenses with fixed orientation have been introduced to produce a facula on the measured surface. Polarized information of each pixel can be obtained with the lenses of known position. According to the polarized information and Lambertian model, Stocks vector, Mueller matrix, and reflected Mueller matrix of each pixel can be acquired. Therefore, the roughness information of target surface can be obtained according to the relationship between roughness information and elements of matrix. Experimental results show that with the proposed method, efficiency of roughness detection can be improved without precision deducing. It can lay a foundation for extending the application of roughness into the field of object identification.