If you want to define your model from scratch, use this tutorial 140. But based on your sample code, it seems you are using transfer learning. Here is the tutorial for transfer learning 31.
For case of binary, BCELoss 59 is a good choice.



 alx:

if the model only sees the class I’m trying to predict,


Why should model see only class 1 samples? In the first post, you have mentioned that you are using two separate folders, one for class 1 and the latter for class 0, 2, 3, 4. So, the model will see all samples and learns class 1 as 1 and 0, 2, 3, 4 as class not 1 or zero.
You can define binary models to learn each separate class and combine them. This approach called OneVsAll.