Custom contrasts in R: why should I take the generalized inverse of transpose of the original contrast matrix? Let's say there is a continuous variable y and a grouping (factor) variable x with 3 different levels: a, b, and c.
x = sample(letters[1:3], size = 300, replace = T)
y = rnorm(300)

I want to test two contrasts: (1) y for group a versus the mean of y for group b and c and (2) y for group a versus y for group c.
Below is how I do this:
contr_mat = matrix(c(1, -0.5, -0.5,
                     1, 0, -1),
                   nrow = 3, ncol = 2)

lm(y ~ x, contrasts = list(x = contr_mat)) %>% summary

However, according to here and here, the correct way to do this is:
library(MASS)
lm(y ~ x, contrasts = list(x = ginv(t(contr_mat)))) %>% summary

which produces different results in terms of both the regression coefficients and P values. But none of the above links explains "why" it is the correct way. The ?lm documentation also does not mention anything about ginv. Could someone explain why I should take the generalized inverse (ginv) of transpose (t) of the original contrast matrix to get the correct contrast matrix?

What you want to specify as x in the list passed to the contrasts argument of lm is the matrix $X_\star$ which relates the regression coefficients $\left(\beta_0, \beta_1, \beta_2\right)^\top$ to the group means $\left(\mu_a, \mu_b, \mu_c\right)^\top$ via
$$
\begin{pmatrix}
\mathbf{1}_3 & X_\star
\end{pmatrix}
\begin{pmatrix}
\beta_0\\
\beta_1\\
\beta_2
\end{pmatrix}
=
\begin{pmatrix}
\mu_a\\
\mu_b\\
\mu_c
\end{pmatrix}.
$$
To make $\left(\beta_1, \beta_2\right)^\top$ correspond to the two contrasts of the group means you want to test, $X_\star$ must be chosen in such a way that
$$
\begin{pmatrix}
\beta_1\\
\beta_2
\end{pmatrix}
=
C
\begin{pmatrix}
\mu_a\\
\mu_b\\
\mu_c
\end{pmatrix}
=
C
\begin{pmatrix}
\mathbf{1}_3 & X_\star
\end{pmatrix}
\begin{pmatrix}
\beta_0\\
\beta_1\\
\beta_2
\end{pmatrix}
$$
with the contrast matrix
$$
C = 
\begin{pmatrix}
1 & -0.5 & -0.5\\
1 & 0 & -1
\end{pmatrix}
$$
holds.
If $X_\star=C^+$, where $C^+$ denotes the Moore–Penrose inverse of $C$ and $C$ has linearly independent rows (which it should have), then $C^+$ is a right inverse of $C$ and we have
$$
C
\begin{pmatrix}
\mu_a\\
\mu_b\\
\mu_c
\end{pmatrix}
=
C
\begin{pmatrix}
\mathbf{1}_3 & C^+
\end{pmatrix}
\begin{pmatrix}
\beta_0\\
\beta_1\\
\beta_2
\end{pmatrix}
=
\begin{pmatrix}
\mathbf{0}_2 & I_2
\end{pmatrix}
\begin{pmatrix}
\beta_0\\
\beta_1\\
\beta_2
\end{pmatrix}
=
\begin{pmatrix}
\beta_1\\
\beta_2
\end{pmatrix}
$$
as desired.

In your $\mathsf{R}$ code contr_mat is the transpose of $C$. You can thus calculate x $\equiv X_\star \equiv C^+$ as the Moore–Penrose inverse of the transpose of contr_mat via x = MASS::ginv(t(contr_mat)).
