Hey buddy,
Thanks for your reply. From the research I did, it would still train the neural network but as expected it will take even more epochs and with strong fluctuations in loss decay. In the book DL with pytorch, they breifly touch this topic in section 7.2.4
Best