Could you post some information about:

model architecture
input, output, target shapes
loss function used
pre-processing steps?

This would make it a bit easier to spot any potential bugs in your code.