As of now, I plan to measure the time taken for gradient accumulation, and the number of bits communicated (for each iteration). I might need to even find the bits communicated for each layer in the future to explore layer wise compression.