This is an old post and it’s a bit hard to read the code in this format, but usually oscillations in simple models come from a learning rate that is too large. You may also want to use a learning rate scheduler to decrease the learning rate over time.