In BertForSequenceClassification, why is loss initialised in every forward?
Looking at [the source](https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_bert.py#L902-L910) I can see that the correct loss function is initialized in each call to forward.

https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_bert.py#L902-L910

Can you explain why? Why isn't the loss function set up as part of `init()`? Is there any advantage of always re-initialising it on each forward? 

Edit: I see that you do this in other parts as well, e.g. the ReLU layer in distilbert: https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_distilbert.py#L598
Also it will be nice if the user gets to use the loss_func itself, Like currently i am using that class with slight modifications to match the pipeline with different losses rather than only CrossEntropy loss. (plus add class_weights etc as well to it)

Though this is what i did actually to use a different loss function, just grab the logits from the model and apply your own..<|||||>You can always subclass the class, to make it your own.

Some extra information for this issue: in an issue over at pytorch, it came to light that loss functions are actually meant to be imported as functions (from nn.functional) rather than modules (from nn). <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Unstale. <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
