VQ-VAE: is the order of latents similar to the order of the signal samples? I'm reading a blogpost about VQ-VAE, and it says there that:

If we are working with a 1-d signal like audio, casting the problem into an autoregressive form is very simple: just predict the next latent in the 1-d sequence encoded from the audio. This approach assumes that the encoder is configured such that later latents in the sequence are representative of later sections of audio (this property should be expected if using convolutions to encode)

two questions:

*

*Is this property is to be expected? Why would there be a relation between the order of latents and the order of raw signal samples? and why is that due to using convolutions in the encoder?

*Is this property needed for doing autoregressive prediction? why does it matter how do the latents map into the raw signal?


*

*I believe this is due to convolution having a limited receptive field. Hence the latent #i depends on the (simplifying here, assuming that there are no strides): [i - rf, i + rf]. Hence for example latent #i + 2*rf depends on later section of audio than #i.


*An autoregressive model of order p only depends on p previous "steps". Hence having some kind of sequentiality is vital to predict sequences of length greater than p.
