For the newcomers, the implementation of DenseNets uses checkpoint 19, where intermediate features (concatenation) are recovered by re-forwarding for each segment while back-warding.

Checkpointing is implemented by rerunning a forward-pass segment for each checkpointed segment during backward. This can cause persistent states like the RNG state to be advanced than they would without checkpointing