text: Title: How to create a tensor variable on the chief worker?
  Body: I need a scalar variable to count something. In parameter server mode, I created it on the first ps node and all the workers can run `add_op` to update it. It works fine.
  ```
  with tf.device('/job:ps/task:0/cpu:0'):
    var_for_count = tf.get_variable('count_variable', (), tf.int32, initializer=tf.zeros_initializer)
    add_op = var_for_count.assign_add(1, use_locking=True)
  ```
  In horovod mode, there exists just the worker nodes. So, I created the scalar variable on the chief worker only and expected all the workers can also use `add_op` to update it like this.
  
  ```
  with tf.device('/job:worker/task:0/cpu:0'):
    var_for_count = tf.get_variable('count_variable', (), tf.int32, initializer=tf.zeros_initializer)
    add_op = var_for_count.assign_add(1, use_locking=True)
  ```
  
  However, it caused an error.
  
  ```
  tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation count_variable: node count_variable was explicitly assigned to /job:worker/task:0/device:CPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0, /job:localhost/replica:0/task:0/device:GPU:0]
  ```