ML07: What is “Robust” ?
(1) Definition of “Robust”

Let’s take a close look at the definitions of “robust / robustness” from a variety of sources:

1. Robust statistics are statistics with good performance for data drawn from a wide range of probability distributions, especially for distributions that are not normal. [2] 2. A robust concept will operate without failure and produce positive results under a variety of conditions. For statistics, a test is robust if it still provides insight into a problem despite having its assumptions altered or violated. In economics, robustness is attributed to financial markets that continue to perform despite alterations in market conditions. In general, a system is robust if it can handle variability and remain effective. [3] 3. Robust statistics, therefore, are any statistics that yield good performance when data is drawn from a wide range of probability distributions that are largely unaffected by outliers or small departures from model assumptions in a given dataset. In other words, a robust statistic is resistant to errors in the results. [4]

Then, we turn to a classic statistics book Problem Solving: A Statistician’s Guide published in 1988:

4. A statistical procedure which is not much affected by minor departures is said to be robust and is fortunate that many procedures have this property. For example the t-test is robust to departure from normality. [5]

(2) Dealing with Errors and Outliers

Now that we went through all kinds of definitions with few variations, let’s see why we need robust statistics / robust model.

While handling with outliers, we have a couple of approaches at hand:

1. Treating outliers as errors, then removing them

A straightforward method often be adopt without full consideration. Here, we may remove the outliers, leaving the data points with missing values.

2. Using domain knowledge and finding outliers possible

It may be sensible to treat an outlier as a missing observation, but this may be improper if the distribution is heavy-tailed. Extreme observations which may, or may not, be errors are more difficult to handle. The tests deciding which outliers are ‘significant’, but they are less important than advice from people ‘in the field’ as to which suspect values are obviously silly or impossible and should be viewed with caution. [5]

3. Using robust methods
['Morton Kuo']
['Machine Learning', 'Statistics', 'Robust', 'AI', 'Robustness']