Would intermediate backprops work?
For example
optimizer.zero_grad() 
x1 = self.cnn(x1)
... # calculate your loss
loss.backward()
optimizer.step()
optimizer.zero_grad() 
x2 = self.cnn(x2)
... # calculate your loss
loss.backward()
optimizer.step()
optimizer.zero_grad() 
x3 = self.cnn(x3)
... # calculate your loss
loss.backward()
optimizer.step()

Uses much less memory. Your effective batchsize will be smaller, but reducing the batchsize is a common way to prevent OOM errors