You must specify an aggregation method to update a MirroredVariable in Replica Context.
# You must specify an aggregation method to update a MirroredVariable in Replica Context.

##  
   <ipython-input-28-7cf32baaf070>:52 step_fn  *
        gradient_accumulator(grads)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/distribute/distribute_lib.py:763 experimental_run_v2
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/transformers/optimization_tf.py:229 __call__  *
        accum_gradient.assign_add(gradient)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/distribute/values.py:1124 assign_add
        return self._assign_func(f=assign_add_fn, *args, **kwargs)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/distribute/values.py:1108 _assign_func
        variable_type="MirroredVariable"))

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

using GPU: Yes 

The problem arises when using:
 Training on multiple gpus and accumulating gradient as given in run_tf_ner.py


Tried executing the test case: **test_optimization_tf.py**
The test case also fails when on GPU.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
