Frequentist perspective of regression coefficients and significance (coming from Bayesian background)? I come from a primarily Bayesian background when using performing statistical analysis. In the context of linear regression, I would look at the posterior distributions for each regression coefficient. If a given coefficient was distributed ~N(1,1), I might not be terribly convinced of the associated variable's influence on the dependent variable.
From a frequentist perspective, regression coefficients often have p-values assigned. And I'm not quite sure what to make of this coming from a bayesian, full posterior interpretation. Is the idea that there's only an x% chance that we would have observed this specific regression coefficient inferred by accident?

In most regression software you can also request confidence intervals for the regression coefficients.  The confidence interval is a set of plausible true values of the parameter, given the observed data.  A 90% confidence interval contains all the hypotheses for the parameter under which the statistic is not significant at the two-sided 10% level.  While this does not have the same interpretation as a 90% credible interval, it serves the same purpose.  If the 90% confidence interval contains zero (two-sided p-value testing $H_0:\beta=0$ is greater than 0.10) you would not be convinced at the two-sided 10% level of the association between the covariate and the dependent variable.  If the 90% confidence interval excludes zero (two-sided p-value testing $H_0:\beta=0$ is less than 0.10) this could be interpreted as evidence (not proof) of an association at the two-sided 10% level (or one-sided 5% level).
If you use something like Proc MCMC you are used to getting kernel density estimates for the posterior distribution of a parameter.  You can analogously produce confidence curves using the point estimate and standard error.  These curves show p-values and confidence intervals of all levels for hypotheses concerning the parameter.
The upper-tailed p-value for a standard issue Wald test as a function of the hypothesis would take the form
$$H(\beta,\boldsymbol{x})=1-\Phi\bigg(\frac{\hat{\beta}-\beta}{\hat{\text{se}}}\bigg).$$
Analogously, the lower-tailed p-value is
$$H^{-}(\beta,\boldsymbol{x})=\Phi\bigg(\frac{\hat{\beta}-\beta}{\hat{\text{se}}}\bigg).$$
The corresponding confidence curve, $C(\beta,\boldsymbol{x})$, can then be defined as
$$C(\beta,\boldsymbol{x})=H(\beta,\boldsymbol{x})\text{, if $\beta < \hat{\beta}$  }$$
$$C(\beta,\boldsymbol{x})=H^{-}(\beta,\boldsymbol{x})\text{, if $\beta > \hat{\beta}$  }$$.
The confidence curve will look like a teepee or a pyramid on the parameter space depicting ex-post sampling probability of the experiment (frequentist confidence).
If, say, you have an estimated beta coefficient of 0.5 and an estimated standard error of 0.18, the figure below depicts the corresponding confidence curve for inference on the unknown fixed true $\beta$.

data norm;
do beta=-0.6 to 1.6 by 0.001;
beta_hat=0.5;
se=0.18;

C_lower=1-cdf('normal',(beta_hat-beta)/se,0,1); if beta gt beta_hat then C_lower=.;
C_upper=cdf('normal',(beta_hat-beta)/se,0,1); if beta lt beta_hat then C_upper=.;

output;
end;
run;

proc sql noprint;
select max(beta)
into: lower
from norm
where . lt C_lower le 0.05;

select min(beta)
into: upper
from norm
where . lt C_upper le 0.05;
quit;

data norm;
set norm;
lower=&lower.;
upper=&upper.;
yscatter=-0.035;
run;

ods escapechar="^";
ods graphics / height=3in width=6in border=no;
proc sgplot data=norm noautolegend;
series x=beta y=C_lower / lineattrs=(color=darkblue) name="cc" legendlabel="Confidence curve (one-sided p-value) with 90% confidence interval";
series x=beta y=C_upper / lineattrs=(color=darkblue);
scatter x=beta_hat y=yscatter / xerrorlower=lower xerrorupper=upper errorbarattrs=(color=darkblue) markerattrs=(color=darkblue);
refline 0 / axis=y;
yaxis max=0.6 label="p-value" min=0 offsetmin=0.1;
xaxis label="^{unicode beta}" min=-0.5 max=1.5;
keylegend "cc";
run;

