I’m not familiar with either of the mentioned libraries, but would still recommend to make sure the model can overfit the small dataset. E.g. did you try to replace the activation functions with a relu?