Your architecture search would have to check, if some activations could become empty tensors e.g. if the pooling or conv layers are too aggressively downsampling these tensors.
I don’t know how your workflow is used and what determines, if a layer should be added or which config should be used.