If I have a 100x100 grid and I apply a 3x3 convolution to output another 100x100 grid, I am propagating information to each cell from its neighbors. If I want to propagate information from each cell’s neighbors’ neighbors, I would apply another 3x3 convolution after the first one. If I want to propagate from N cells away, I would apply N 3x3 convolutional layers one after the other.
But what if I want to propagate information from every cell in the grid, and the grid has variable size? I can’t have a variable number of layers.
What if instead of N 3x3 convolutional layers, I applied the same 3x3 convolutional layer to its own output N times? That would have the same effect of propagating information to each cell from N cells away, and it can be done for variable N.
How would learning work with this setup? What could the drawbacks be to this approach? Should I somehow use a hidden state between the applications like in an RNN?