Hello,
I’ve got a question - is there possibility to train model with samples (batch size = 1) and inference this model using batches?
When I changed to batch_size = 1, I had to remove Batchnorm1D layers from my model because of errors. But I think that I will need them when I want to inference my model with batches after training.
I can’t train with batches because I’m training with incomplete data - so I’m not always “turning on” all heads. What do you suggest?
Best regards