I would like to be able to approach GPU out of memory issues more systematically

Are there some resources that explain roughly when allocations happen and when memory is released?
Are there tools that can show which tensors are alive at a given time?

As a concrete example, I am building a network that operates on 3d medical images and GPU memory is an issue. My model contains dense net like building blocks that look like
tmp = torch.cat([lots, of, inputs], 1)
small_output = conv(tmp)

I suspect that tmp eats up a lot of memory and is computationally cheap.

How to tell pytorch to recompute tmp whenever it is needed instead of storing it for the backward pass?
If this is solved, how profile the better memory footprint?