This work was partly inspired by @nicolaskeriven.bsky.social's paper "Not too little, not too much". He gives an analytic bound on the improvement brought by one convolutional layer, and I propose a way to compute it numerically (with all of the caveats that you expect).

arxiv.org/abs/2205.12156