IMO, yes you can do it by passing in another flag in your dataset (csv file) and selecting images based on that make, make sure that in a batch you have similar no of images of all classes; And then in the forward you can select what you want to-do accordingly easily;