his doctor may not be concerned (he sees lots of penises) but you are so that is sufficient to do a bit of research. a little boy's penis can just seem smaller due to an abundance of fat pads in this area or he could have a true "hidden penis". a hidden penis does have part inside so may need to be surgically freed around puberty. puberty is a time where hormonal influence causes a penis to grow to an adult size; the maximum size occurs around age 17. you can easily find information on hidden penis with an internet search. little boys who are on the chubby side will often havea penis that looks disproportionately smaller. since you are concerned a consultation with a pediatric urologist may offer you the reassurance you desire. the specialist may pronounce him "normal" or if he has a true hidden penis provide you with the information you need.
my 5yr son's penis is very small. seem to be grow in and not out. his doctor tell my not to worry but i'm worried