Use os.walk() to traverse your main folder, then write for each image 1 line containing the image path (you can choose absolute or relative, depending on your approach and preferences) followed by the target label.
Don’t forget the CSV header, so you can use Pandas for example to load the file directly in a practical database.