&gt; Do the multi-dimensional matrices that exchange data between the nodes transform like tensors?<p>Yes, if you design the moeel&#x2F;graph that way.<p>&gt; If so, when does the need arise to transform them?<p>The need arises whenever tensors are needed. For deep learning, most people treat them like multidimensional arrays. TensorFlow is an excellent name.<p>Multidimensional arrays are a thing of the past. Now we call them tensors. Get with the program or become an aging, forgotten physicist not involved in deep learning.