Well, you can multiply intermediate layer outputs with binary mask, that would zero values computed from partially available inputs. You would start this mask shaped like input, with one channel, and do “min pooling” whenever data shape is changed by convolution or pooling operation. It is actually easier to do max pooling with inverse mask (1 = bad location), and invert it for multiplication.
correction: when doing convolution and pooling, different mask pooling modes should be used (i.e. data max-pooling output is good if ANY input is good, convolution - needs ALL inputs)