Hypothesis testing with p-values from similar tests as prior I hope to work on a hypothesis testing problem to test $\beta$ against the null hypothesis ($H_{null}$) $\beta=0$, and we can consider Wald test with the test statistics
$$
W = \dfrac{\beta^2}{var(\beta)}
$$
Then, with $W$, I can get p-values, which is essentially "getting a value of the test statistic as extreme as or even more extreme than what is observed by chance alone", so
$p(t > W | H_{null})$.
However, the sample size I have to calculate $\beta$ and $var(\beta)$ is fairly small. The results, although p-values for some covariates are small, are not very convincing.
On the other hand, I have some p-values calculated for the same set of covariates, from a larger body of samples. Basically, I have a prior $p_0(t>W_0|H_{null})$. In addition, sometimes I also have $\beta_0$.
I wonder whether/how I can calibrate $p(t > W | H_{null})$ with $p_0(t>W_0|H_{null})$.
Some of my current efforts

*

*I learned a way to do this is to simulate a large amount of data that leads to $p_0$ if we conduct the test, and then combine the simulated data and my real ones to do the hypothesis testing, but I hope there are more efficient ways to do it.

*Bayesian hypothesis testing sounds relevant, but it assumes $p_0(H_{null})$ instead of $p_0(t>W_0|H_{null})$, and it leads to a MAP result. I hope we can still have a p-value at the end.


It sounds like what you are after is a meta-analysis.  This is the frequentist analog to a Bayesian analysis using an informative prior.
A meta-analysis can come in many forms.  One option is to combine the subject-level observations into a single data set and run a single model to produce p-values and confidence intervals.  This is the ideal approach.
Another option is to combine summary-level statistics or estimators into a single value and estimate its standard error.  One can then perform a hypothesis test to produce a p-value and invert a hypothesis test to construct a confidence interval.  Another option is in line with what you proposed, which is to use simulated data that mimics the observed results from a historical study and use option one.
Here is a related paper on meta-analysis and transfer learning.
