ZiyiZhu:

Take a nn.con2d as an example(nn.Conv2d). If this is the one at the RPC boundary and in the forward pass, the total tensor size to be transmitted to the following node should be equal to Batchsize x Channels x W x H , IIUC. So in the backward path, should it be the same size as this? Or it is accumulated as Channels x W x H ?


My bad. If there is a batch dimension in the RPC argument tensor, then yes, the batch size will have an impact on the data size sent over wire. It will be B * C * W * H.