Hi,
Could you give more details on the loss value evalution? Both for training and validation sets?
Also you should try with a much simpler model just to make sure that you are able to learn something on your data that is better than random.
You might have some bugs in your code that will be easier to catch with a simpler model with a more predictable behavior.