I’d recommend trying out using softmax with cross entropy with each target class set to 1/num_target_labels. That has usually worked better for me than binary cross entropy and sigmoid outputs.
There are a few papers from facebook that also mention doing that (https://arxiv.org/abs/1805.00932 38)