I tried printing the loss, it was a series of values, so decided to reduce the loss using the reduction='sum' parameter in this function binary_cross_entropy.
It started training though the loss is quite high.