Another tangentially related occasion where just position doesn’t work is the loss function, where one might naively use mean squared error, but doesn’t whenever one has a credible alternative (like cross-entropy).

mean squared loss, L2 loss and L1 loss are most suited for regression right? Why would you want to use cross-entropy which is otherwise used for binary classification?