Broadcasting should not increase the memory usage, but you would of course need to store the result.
Here is a small code snippet with similar shapes:
x = torch.randn(30000, 1, 10).cuda()
y = torch.randn(20, 10).cuda()
#pprint.pprint(torch.cuda.memory_stats())
print('before sub')
print('mem expected in MB: ', (x.nelement() + y.nelement()) * 4 / 1024**2)
print('mem allocated in MB: ', torch.cuda.memory_allocated() / 1024**2)
print('max mem allocated in MR: ', torch.cuda.max_memory_allocated() / 1024**2)

res = x - y
print('after sub')
print('mem expected in MB: ', (x.nelement() + y.nelement() + res.nelement()) * 4 / 1024**2)
print('mem allocated in MB: ', torch.cuda.memory_allocated() / 1024**2)
print('max mem allocated in MR: ', torch.cuda.max_memory_allocated() / 1024**2)

Output:
before sub
mem expected in MB:  1.145172119140625
mem allocated in MB:  1.1455078125
max mem allocated in MR:  1.1455078125
after sub
mem expected in MB:  24.033355712890625
mem allocated in MB:  24.03369140625
max mem allocated in MR:  24.03369140625

As you can see, the expected memory matches the allocated and max. allocated memory closely.



 PytorchBeginner:

Will the required memory be released in next epoch or increased epoch by epoch?


The memory usage should not increase over time, if you are using the same shapes.
If you see an increase in memory usage while your model is training, you might accidentally store some tensors with the attached computation graph, e.g. by doing losses.append(loss) during training.
If that’s the case, you should either .detach() the tensors or call .item() on them to get a Python literal.