364 images are not that many and your model might overfit quickly, especially since you would need to split this dataset into a training, validation, and test set.
You could try to use an aggressive data augmentation and observe the validation loss to make sure the model still generalizes well.
I don’t think that a custom model trained from scratch would be easier, so your best bet might be to try to fine tune a pretrianed model, add data augmentation, maybe increase the regularization, or in the best case collect more data.