Decide on the number of adversarial samples to include during training A sample is considered adversarial if it drastically changes classifier's confidence $\in [0,1]$ when given as input. For example, if a spam/ham binary classifier considers some input $X$ to be 0.9 spam, then an adversarial input $X + \varepsilon$ (for some carefully crafted noise $\varepsilon$) may change the confidence to 0.01.
A common approach to make a model robust to such adversarial attacks is to include adversarial samples in the original dataset (essentially data augmentation). Here lies my question: what would be an informed / scientific way of deciding how many adversarial samples to include?
For instance, given a binary-labelled dataset, I can think of several factors that can decide the number of adversarial samples to be included:

*

*Vulnerable sample selection: generate a bunch of adversarial instances only for testing, then find such positive instances $X$ on which the model drastically changes its confidence, i.e. $f(X) - f(X+\varepsilon) \approx 1$. Finally, generate more adversarial examples for such vulnerable instances $X$.

*Try to balance the dataset by including $\approx \text{neg} - \text{pos}$ adversarial samples. This should make the training easier, but the real-world neg/pos distribution is discarded.

*Add as many adversarial samples as necessary so as to cover most of the noise; or in other words, make each adversarial method as likely as possible. For example, in the spam/ham problem, some methods may include: altering the punctuation, change upper/lower case, misspelled words etc.


There is no single answer.  It all depends on what your goals are.
Adversarial training uses exclusively adversarial examples.  TRADES uses 50% clean samples and 50% adversarial samples.  Mixed adversarial training uses a mixture of clean samples and adversarial samples; the proportion can be anything.
The proportion of clean vs adversarial samples is a hyperparameter that you can adjust; adjusting it changes the tradeoff between accuracy on clean samples, vs robustness (accuracy on adversarial samples).  Where on that tradeoff curve you want to live will depend on your application and how the model is used.
