I don’t know how and where you’ve downloaded the ImageNet dataset, but on my system each class uses a separate subfolder, which is needed in order to find all classes.
E.g. the val folder contains:
n01440764  n01824575  n02093754  n02115641  n02457408  n02877765  n03272010  n03710193  n03958227  n04296562  n04597913
n01443537  n01828970  n02093859  n02115913  n02480495  n02879718  n03272562  n03710637  n03961711  n04310018  n04599235
n01484850 ...

where each folder contains the images for this particular class.