Frequentist perspective of regression coefficients and significance (coming from Bayesian background)? I come from a primarily Bayesian background when using performing statistical analysis. In the context of linear regression, I would look at the posterior distributions for each regression coefficient. If a given coefficient was distributed ~N(1,1), I might not be terribly convinced of the associated variable's influence on the dependent variable.
From a frequentist perspective, regression coefficients often have p-values assigned. And I'm not quite sure what to make of this coming from a bayesian, full posterior interpretation. Is the idea that there's only an x% chance that we would have observed this specific regression coefficient inferred by accident?

These numbers are unreliable if your predictors are correlated.
Suppose for example that the value to be predicted is the sum of two hidden N(0,1) random variables $Y=Z_1+Z_2$ and $Z_2$ is unobserved, and you have observations of $X_1 = Z_1+N(0, 0.01), X_2 = Z_1+N(0, 0.01)$
Then the linear regression is degenerate. Any solution is equally good, in which the coefficients of $X_1, X_2$ sum to one. No magic with p-values can help you choose.
Perfect degeneracy like this is rare, but some level of correlation is common. Take home lesson: correlation is not causation.
