In an experiment, why can you select your own alpha level for significance testing? From what I have learned, the alpha level tells you that if you repeat the experiment a bunch of times, then (alpha x 100)% of the time you will have a false positive. I'm confused though on if you're the one who gets to pick the alpha level, then how is this always the case? Like if my alpha is 0.05, 5% of the time I will have a false positive (FP), but then if I want to use alpha = 0.5, then 50% of the time I will get a FP. Why is this?

The premise of your question is not quite right. Alpha is the fraction of the time you'll have a false positive only if:

*

*The null hypothesis is assumed to be true. Usually, this means assuming (for the sake of argument while interpreting alpha or p) that the two populations have the same mean (or proportion, or survival curves...).

*All the assumptions of the analysis are correct. Often this means assuming the population follows a normal distribution and the deviation of each value from the mean is due to independent random reasons.

Your overall question needs clarifying. Alpha is a cutoff for making a conclusion, so it has to be set to a value. If not chosen by the experimenter, then by who?
Addendum Feb. 25, 2023: Maybe this will help. If you choose a large value for alpha, you'll have a larger chance of obtaining a false positive but a smaller chance of a false negative. If you choose a small value for alpha, you'll have a smaller chance of obtaining a false positive but a larger chance of false negative. It's a tradeoff. That is why there is a choice.
