Given enough time, can a fully connected layer approximate a causal convolution layer? In the paper WaveNet: A Generative Model for Raw Audio, the authors try to capture spatial data as follows:

They do this by limiting the scope of the hidden layers to particular sections of the input, this approach improves performance as supported by experimental evidence. However, is it not possible for the same structure to arise naturally through the use of a straightforward fully connected layer, or am I overlooking something?
Is the purpose of the new layer just to artificially "accelerate training" by having humans narrow down the search space to smaller and smaller subsets, or is there something distinct about it?
Would a fully connected layer have eventually uncovered the same results if given enough time and computation resources (but the exact same dataset)?

Probably not.
As you allow the model more flexibility to fit the (in-sample) data, you let it use that flexibility to get a smaller value of the loss function.
The genius of conventional neural networks is not just that they limit this flexibility but that they do so in a clever way that makes sense from the standpoint of how humans see (or how we think we see).
