Yes, the memory usage could be large since you are broadcasting the tensors and need to calculate the intermediates. Using loops would have a lower memory footprint, but could be slower. Your best bet might be to write a custom C++/CUDA operation for your use case and check if you could get a proper speedup without a large memory requirement.