Hi,
By looking at your architecture, first question: Is there any specific reason to branch tag_net for every tag? Your tag_net takes input tensor of 1024 values and gives output tensor of only 1 value so instead of branching tag_net for every tag, you can just feed the output of resnet18 to a single tag_net and get output tensor of same values as your number of tags. I am guessing your image contains multiple tags so you have to use nn.BCEWithLogitsLoss function for calculating loss and pass hot-encoded vector as your target.
Take a look at this: https://gombru.github.io/2018/05/23/cross_entropy_loss/
Hope this helps.