At the last layer, I should not use out = F.relu(out) but out = torch.sigmoid(out), then the model can output [0, 1]. (so can predict class [0, 0.5] and [0.5, 1])
    def forward(self, input_16, input_32, input_48):
        # CNN part
        ...
        out = torch.cat([out_16, out_32, out_48], dim=1)
        out = F.relu(self.fc_last_1(out))
        out = self.fc_last_2(out) 
        out = torch.sigmoid(out)
        return out

# the model output part
output = model(batch_z_16, batch_z_32, batch_z_48) # model prediction
loss = criterion(output, batch_label)

Is this the correct answer?