Thanks @Bhack!
I guess this is my workaround solution for now:
Not every keras.metrics.* accept from_logits=True · Issue #42182 · tensorflow/tensorflow · GitHub 15.
I will call this a bug since BinaryCrossentropy suggests using from_logits=True.
In the issue you had posted, they state this is fixed but I guess this is not the case. I will post there.