Hi , below I have two versions of the same function. In the first, the tensors for “flipped” and “salted” are created on the gpu. In the second, they are created first on the cpu and then pushed to the gpu. When I run my experiment with the second function, since I use up alot of memory in different parts of my experiment too, I run out of memory. When I use the first function, I don’t. I’m wondering - where is the extra memory coming from? Thanks!
def add_s_and_p_noise(s_and_p, p, gpu):
    q = 0.5
    s_and_p_cloned = s_and_p.clone()

    with torch.cuda.device(gpu):
        flipped = torch.cuda.FloatTensor(s_and_p.shape).uniform_() < p
        salted = torch.cuda.FloatTensor(s_and_p.shape).uniform_() > q
    peppered = ~salted
    s_and_p_cloned[flipped & salted] = 1.0
    s_and_p_cloned[flipped & peppered] = 0.0
    with torch.cuda.device(gpu):
        del flipped
        del salted
        del peppered
        torch.cuda.empty_cache()

    return s_and_p_cloned

def add_s_and_p_noise_two(s_and_p, p, gpu):

    s_and_p_cloned = s_and_p.clone()
    q = 0.5
    flipped = (torch.rand(s_and_p.shape) < p).to(gpu)
    salted = (torch.rand(s_and_p.shape) > q).to(gpu)
    peppered = ~salted
    s_and_p_cloned[flipped & salted] = 1.0
    s_and_p_cloned[flipped & peppered] = 0.0
    with torch.cuda.device(gpu):
        del flipped
        del salted
        del peppered
        torch.cuda.empty_cache()

    return  s_and_p_cloned