Here Jeff Dean explains why there is no central parameter server in Tensorflow: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;90-S1M7Ny_o?t=28m59s" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;90-S1M7Ny_o?t=28m59s</a>