Your current code is trying to allocate too much memory and is thus raising this error.
Did you change anything in the model, such as the spatial size of the input or the batch size?
A workaround is to reduce the memory usage so that the workload can fit in your 6GB GPU e.g. by reducing the batch size or by using torch.utils.checkpoint to trade compute for memory.