how standard is R model formula notation I like using R model formula notation such as $Y \sim X_1 + X_2$ when thinking of regression relationships at a high level. The same for lmer notation such as  $Y \sim X | group$. I am wondering how standard it is. Does it show up in journal articles? If it is R or S's or lmer's innovation, I probably would not put it in published work, but maybe R/lmer just borrowed it from older statistical practice.

You should not assume it's standard. Users of S-plus and R would be familiar with it, Julia’s packages adopted it, Python’s statsmodels has it as well, but not all the software uses it. Unless you are writing a paper about the software you should rather use the mathematical notation for the same reason we don't commonly use notation like [i] for subscripts, etc.
Also keep in mind that $\sim$ already has several meanings in mathematical notation e.g. “is approximately equal”, “is distributed as”, “is asymptotically equivalent”, neither of them fits regression model definition.
