What is the meaning of the variance in a prior that represents L2 regularization? Why is the L2 regularization equivalent to Gaussian prior? does a nice job in explaining the relationship between L2 regularization and Gaussian priors.
One answer explains that the variance in the Gaussian likelihood is a measure of the noise in the data (which makes intuitive sense).
What is the intuitive understanding of the variance in the Gaussian prior (which is related, by algebraic computations, to the "strength of the regularization")? Based on the above, the variance in the prior is a measure of the "noise" of the "weights of the parametrized function," but that seems an odd and unuseful way to think about it. Is there a better/clear way to understand this variance?

The basic idea of the prior is, that it describes your knowledge about the weights before you get to know the observations (the data). Thus, a prior with a mean equal to zero and a small prior variance keeps the parameters near zero, i.e. it is used as regularization. Alternatively, a large prior variance means that you don't have much knowledge about the weights and the main source of your information are your data. If there is no prior knowledge about the weights, people use uninformative priors, which don't contain information. This could go as far as using improper priors, which are not even proper densities anymore but can still be computed with as far as the purposes of Bayesian inference are concerned.
Note, that the priors are not the only source of the "noise of the weights", i.e. the variance of the posterior, but it is also very much determined by the data.
Furthermore, what is really useful about priors, is that you can learn them, too. You can make (some of) the hyperparameters, that describe the prior, random variables as well, creating a hierarchical model, and also infer them from the data. That way, you could e.g. even create priors that help you learn sparse models, as in the Relevance Vector Machine.
Thus, while the basic purpose of the prior is to convey prior knowledge about your weights, they can have additional purpose if they are learned, too.
