just wanted to put a quick update,there is a bug in the dataloader,the nested for loop in the init causes one image to be associated with 10 class labels,which is not correct,because each image should only be associated with one label.
For example
for img_path in img_list:runs 1000 times
            for file in self.pose_files: runs for all the pose files#1000
                self.data.append([img_path,file])
total number of times the loop runs 1000*1000 which causes the no of entries in the dataloader to blow up,and also causes one image to be associated with 1000 labels.

correct way
for img_path in img_list:
            for file in self.pose_files:
             if pose file_idx==img_path_idx:
        then append label and img to data else ignore

Because of the previous problem,i think the data distribution gets affected,which  in turn causes the net to overfit.Let me know if I am on the right track,plus I solved the issue of encoding labels by using class_to_idx and idx_to_class methods.