What is the best way to evaluate the efficacy of an intervention that only a non-random subset of the participants complies with? I want to evaluate the efficacy of an intervention that only about 50% of participants comply with. The intervention consists of giving people an app that they are asked to use over an extended period of time. The problem is that only about 50% of the participants who are asked to use the app do so consistently. We have reasons to believe that the people who comply with the intervention may be systematically different from those who do not. The outcome measures are self-report measures that might suffer from placebo effects or demand characteristics. Our question is how beneficial the app is for people who actually use it.
I can choose both the experimental design and the statistical methods used to analyze the data. What is the best combination of an experimental design and data analysis strategy?


We have reasons to believe that the people who comply with the intervention may be systematically different from those who do not.

This is a good assumption and is known as "selection bias".  There is usually a reason that people choose to do something (e.g. take a drug, pursue a career, etc) and that reason could be correlated with some outcome (e.g. risk of death, future earnings, etc).

What is the best combination of an experimental design and data analysis strategy?

There is a lot of existing work on how to account for selection bias.  Pearl lists out some criteria for recovering causal estimates from S biased data (S for selection) in his 2012 paper "Controlling Selection Bias in Causal Inference".  These criteria require you to draw a dag for the problem, but this can be simplified considerably since you're running an experiment.
Since you're running an experiment, I will assume you have the ability to randomize users to exposure (use of app or control).  Hence your dag might look like this

Here, conditioning the estimate on S (selecting to use the app) opens a backdoor through $W$ (factors which determine the use of the app) which produces a correlation the outcome.  One strategy would be to determine which factors you think will drive use of the app (that is, determine $W$) and adjust for them in a regression analysis.
Important to this approach is the assumption that $S \to \text{Outcome}$ and not $\text{Outcome} \to S$.  The latter would yield the problem unsolvable from a causal inference perspective (according to Pearl in that paper I mentioned).
