content_image/295745.jpg
## Appendix: Simulation comparing the power of the omnibus F-test to I c 

In this appendix, we compare the power of the omnibus F-test to the power of our version method to detect departures from Fisher’s sharp null hypothesis under varying degrees of “versioning.” The results can be found in Table 2. When the degree of versioning is modest, the version method is more powerful than the F-test. 

## Simulation settings 

<img src='content_image/295745.jpg'>

Let there be I = 100 matched sets, with m$_{i}$ = 1 treated and n$_{i}$ − m$_{i}$ = 4 controls in each set. The design is balanced over versions, i.e., there are two controls of each version in each matched set. In each set we generate outcomes as follows: Y$_{ij}$ = τ + X$_{i}$ + ϵ$_{ij}$ if the j -th subject in set i receives treatment, Y$_{ij}$ = X$_{i}$ + ϵ$_{ij}$ if the j -th subject receives control version b , and Y$_{ij}$ = δ + X$_{i}$ + ϵ$_{ij}$ if the j -th subject receives control version. We let the individual and matched-set level terms, ϵ$_{ij}$ and X$_{i}$ , be distributed as independent standard normals. Finally, let τ b = τ and τ a = τ − δ . If there are no versions, then δ = 0. When conducting the F-test of the null of no treatment effect, we model X$_{i}$ as a linear effect. 

