Thanks, I understand it now. Yes, I really want to use it for the sequential like BN-ReLU-Conv. As the backward of convolutional layer  does not need the input data, I can make the input data stored in a shared memory to save memory usage. For BN-ReLU backward, I just need to recompute the output first. By the way, does the backward of the group convolution (namely groups>1) also not need the input?