I decided to come up with a model have classes 0, 1 or 2 (2 for 2 or more people) and moved my images to data folder using this code.It took seconds
i = 0
import shutil
from pathlib import Path

for i in range(len(filenames)):
    if train_test[i] != '0':
        my_file = Path('images/'+ filenames[i])
        if my_file.is_file():
            if len(sps[i]) == 0:
                shutil.copyfile('images/'+ filenames[i], 'data/0/'+filenames[i]) 
            elif len(sps[i]) == 1:
                shutil.copyfile('images/'+ filenames[i], 'data/1/'+filenames[i]) 
            else :
                shutil.copyfile('images/'+ filenames[i], 'data/2/'+filenames[i]) 

removed couple of things so my data in .csv file looks like this
015601864.jpg;1;sports;curling;12
015599452.jpg;1;sports;curling;3

Writing this if it would help someone else too and marking this reply as solution.