Wilcoxon signed-ranks test with small samples The editors for my article in publication want us to report the Wilcoxon signed rank test for our sample.  I did not because I was taught not to run stats on pilot studies and small samples. the sample consists of 7 participants and the data shows their mean for mindfulness pre and post intervention. I ran the Wilcoxon and am getting a p =.04.  Can I really call this significant?

Answer composed of comments to date:

*

*Assuming your standard of significance exceeds 0.04 and you obtain a p-value of 0.04, by definition it is significant, regardless of sample size. (whuber)

*Depending somewhat on exactly how the test is conducted and the specifics of the data, if the difference for at least 6 out of 7 pairs is either all greater than zero or all less than zero, the p-value for the signed-rank test will be less than 0.05. ... In your narrative, you might emphasize the practical importance of such a finding for a small dataset. And also how great the change was pre- and post-. But honestly, if six out of seven participants all change in the same direction, that's an interesting result, though probably not particularly convincing for seven people. (Sal Mangiafico)

*Even though @whuber's Comment is precisely true, SalMangiafico is correct to say that one should be cautious reporting preliminary results without appropriate explanation. First, you couldn't possibly have a significant Wilcoxon RS test at the 5% level with only 5 subjects even if all 5 subjects changed in the same direction: 2/25=0.0625>0.05. Also, people might be suspicious of a significant result from only 7 subjects. (BruceET)

*... In addition to @SalMangiafico comment, the sequential analysis of an experiment in progress as the sample size increases, may call into question the criterion for rejection of H0. If a preliminary power and sample size procedure showed that 20 subjects should be used, then quitting after only 7 because of a preliminary "significant" result may be inappropriate. // If forced to reveal partial information from what you regard as a pilot study, you should raise cautions about making early conclusions. (BruceET)

