kontrabas:

is there possibility to train model with samples (batch size = 1) and inference this model using batches?


This should be possible, as the batch size during inference could be arbitrary, if you call model.eval() and don’t use custom layers, which might depend on the batch size.



 kontrabas:

When I changed to batch_size = 1, I had to remove Batchnorm1D layers from my model because of errors. But I think that I will need them when I want to inference my model with batches after training.


I assume the batchnorm layers raise errors, as they cannot calculate the batch statistics using a scalar value?
Why would you need the batchnorm layers during inference, if they weren’t used during training?



 kontrabas:

I can’t train with batches because I’m training with incomplete data - so I’m not always “turning on” all heads. What do you suggest?


Could you explain this issue a bit? I’m not sure what the heads refer to and maybe it would be possible to increase the batch size somehow.