should loss_scale be multiplied to the loss explicitly?
I noticed that in the run_swag.py, the following code is included

    if args.fp16 and args.loss_scale != 1.0:
        # rescale loss for fp16 training
        # see https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html
        loss = loss * args.loss_scale

and in run_squad.py,  this is not included.
the optimizer in them are identity, so which is right?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
