Why is the Median Less Sensitive to Extreme Values Compared to the Mean? I am sure we have all heard the following argument stated in some way or the other:

*

*For a given set of measurements (e.g. heights of students), the mean of these measurements is more "prone" to be influenced by outliers compared to the median of these same measurements.
Conceptually, the above argument is straightforward to understand. The median is not directly calculated using the "value" of any of the measurements, but only using the "ranked position" of the measurements. On the other hand, the mean is directly calculated using the "values" of the measurements, and not by using the "ranked position" of the measurements. Therefore, a statistically larger number of outlier points should be required to influence the median of these measurements - compared to influence of fewer outlier points on the mean. For example: the average weight of a blue whale and 100 squirrels will be closer to the blue whale's weight, but the median weight of a blue whale and 100 squirrels will be closer to the squirrels.
Using the R programming language, we can see this argument manifest itself on simulated data:
library(plotly)
    set.seed(123)


d = data.frame(data = rnorm(20, 5,50), col = "non outlier")

dd = data.frame(data = rnorm(5,150, 10), col = "outlier")

my_data = rbind(d,dd)

> mean(d$data)
[1] 10.08877

> median(d$data)
[1] 17.11447

We can also plot this to get a better idea:
d1 = data.frame(data = mean(my_data$data), col = "mean")
# add "1" to the median so that it becomes visible in the plot
d2 = data.frame(data = median(my_data$data) +1 , col = "median")

new_data = rbind(my_data, d1, d2)


plot_ly(type = "scatter", mode = "markers", data = new_data, x = ~data, y = " ", color = ~col ) %>% layout(title = 'Effect of Outliers on Median vs Mean')


My Question:  In the above example, we can see that the median is less influenced by the outliers compared to the mean - but in general, are there any "statistical proofs" that shed light on this inherent "vulnerability" of the mean compared to the median?
Apart from the logical argument of measurement "values" vs. "ranked positions" of measurements - are there any theoretical arguments behind why the median requires larger valued and a larger number of outliers to be influenced towards the extremas of the data compared to the mean?
I am aware of related concepts such as Cooke's Distance (https://en.wikipedia.org/wiki/Cook%27s_distance) which can be used to estimate the effect of removing an individual data point on a regression model - but are there any formulas which show some relation between the number/values of outliers on the mean vs. the median?
Are there any theoretical statistical arguments that can be made to justify this logical argument regarding the number/values of outliers on the mean vs. the median?

Mathematical description/proof/viewpoint for special case
There is a short mathematical description/proof in the special case of

*

*Comparing the sensitivity in terms of the variance of the sample statistic. (this can be generalized to the 3rd central moment, and possibly other cost functions)

*Symmetric distributions in which case we can express the distribution of the median and mean in terms of integrals of the quantile functions in a similar way.

Let's assume that the distribution is centered at $0$ and the sample size $n$ is odd (such that the median is easier to express as a beta distribution).
Then in terms of the quantile function $Q_X(p)$ we can express
$$\begin{array}{rcrr}
Var[mean(X_n)] &=& \frac{1}{n}\int_0^1& 1 \cdot Q_X(p)^2 \, dp \\
Var[median(X_n)] &=&  \frac{1}{n}\int_0^1& f_n(p) \cdot Q_X(p)^2 \, dp
\end{array}$$
where $f(p) = \frac{n}{Beta(\frac{n+1}{2}, \frac{n+1}{2})} p^{\frac{n-1}{2}}(1-p)^{\frac{n-1}{2}}$
Below is a plot of $f_n(p)$ when $n = 9$ and it is compared to the constant value of $1$ that is used to compute the variance of the sample mean.

What the plot shows is that the contribution of the squared quantile function to the variance of the sample statistics (mean/median) is for the median larger in the center and lower at the edges.

Adding outliers versus changing outliers.
When we change outliers, then the quantile function $Q_X(p)$ changes only at the edges where the factor $f_n(p) < 1$ and so the mean is more influenced than the median.
When we add outliers, then the quantile function $Q_X(p)$ is changed in the entire range. So the median might in some particular cases be more influenced than the mean.
Example: Say we have a mixture of two normal distributions with different variances and mixture proportions. Then the change of the quantile function is of a different type when we change the variance in comparison to when we change the proportions.
Below is an example of different quantile functions where we mixed two normal distributions.

*

*The black line is the quantile function for the mixture of $90\%$ a distribution with $\sigma = 1$ and $\phi = 10\%$ a distribution with $\sigma_{outlier} = 2$.

*On the left we changed the proportion of outliers $\phi \in \lbrace 20 \%, 30 \%, 40 \% \rbrace$.

*On the right we changed the variance of outliers with $ \sigma_{outlier} \in \lbrace 4, 8, 16 \rbrace$.


The quantile function of a mixture is a sum of two components in the horizontal direction. Whether we add more of one component or whether we change the component will have different effects on the sum.

Generalizations

*

*The conditions that the distribution is symmetric and that the distribution is centered at 0 can be lifted. It will make the integrals more complex. $$\begin{array}{rcrr}
Var[mean(X_n)] &=& \frac{1}{n}\int_0^1& 1 \cdot (Q_X(p)-Q_(p_{mean}))^2 \, dp \\
Var[median(X_n)] &=&  \frac{1}{n}\int_0^1& f_n(p) \cdot (Q_X(p) - Q_X(p_{median}))^2 \, dp
\end{array}$$ now these 2nd terms in the integrals are different. We have $(Q_X(p)-Q_(p_{mean}))^2$ and $(Q_X(p) - Q_X(p_{median}))^2$. But we still have that the factor in front of it is the constant $1$ versus the factor $f_n(p)$ which goes towards zero at the edges.


*The condition that we look at the variance is more difficult to relax. I have made a new question that looks for simple analogous cost functions. But we could imagine with some intuitive handwaving that we could eventually express the cost function as a sum of multiple expressions $$mean: E[S(X_n)] =  \sum_{i}g_i(n) \int_0^1 1 \cdot h_{i,n}(Q_X) \, dp  \\ median: E[S(X_n)] =  \sum_{i}g_i(n) \int_0^1 f_n(p) \cdot h_{i,n}(Q_X) \, dp $$ where we can not solve it with a single term but in each of the terms we still have the $f_n(p)$ factor, which goes towards zero at the edges.
