How can the following figure be obtained, and is there a way to tag the name of each tensor during profiling?

#83
by ll922 - opened

image.png

Nanotron Research org

It's using the pytorch memory profiler: https://pytorch.org/memory_viz
good blog on how to use it here: https://huggingface.co/blog/train_memory

Sign up or log in to comment