Hi,
The usual things to look at here are to make sure that your inputs are properly normalized so that they have similar scale with the weights and between each other.
You also need to make sure that your learning rate is appropriate for your task to avoid too large steps that would make your network diverge.