<p>Yes, there would be a point. Assuming your rule set is accurate, then you can use data classified with it to train a model. This model can be expected to be more robust and properly categorise emails that your rule-set will not handle.</p>

<p>Why? Machine learning algorithms generally work on features, and identify relationships between those features that lead to a classification decision. A human rule author basically does the same, but they might not notice subtle relationships; a good ML algorithm, however, might pick those up.</p>

<p>So you could have a hybrid model, where you first use your rule-based classifier, and anything that does not get classified is then run through the ML classifier.</p>

CC BY-SA 4.0
2019-03-22T18:08:56.973Z
This. But you should also take into account that your rules and therefore your training data have incorrect classifications. For example, `no_reply` as a localpart is relatively common for automatically generated emails that are not spam at all, but requested by the recipient. My experience (I've built a deep learning spam classifier) is that with a very basic feature set reasonable results can be achieved (after training on manually classified samples).
Hans-Martin Mosner
CC BY-SA 4.0
2019-03-22T18:09:47.780Z
However, for production quality, you need several iterations of looking at the results (especially misclassifications) and adjusting feature extraction.
Hans-Martin Mosner
CC BY-SA 4.0
2019-03-22T09:02:50.930Z
2019-03-22T09:02:50.930Z
Oliver Mason
<p>Suppose that we have unlabeled data. That is, all we have are a collection of emails and want to determine whether any of them is spam or not. Let's say we have <span class="math-container">$1,000$</span> rules to determine whether a particular email is spam or not. For example, one rule could be that a sender's email address should not contain the text <code> no_reply </code>. If an email does contain this text, then it would be classified as spam.</p>
<p><em>What are the advantages/disadvantages of a rules-based approach for detecting spam vs. a non-rules-based approach/unsupervised methods for detecting spam?</em></p>
<p>Would there even be a point in constructing a non-rules based model given that we already have a rules-based model? Could we use the rules-based model to create some labeled training data and then apply supervised techniques?</p>

<comparison><unsupervised-learning><anomaly-detection><binary-classification><rule-based-systems>
Are there any advantages of using rules-based approaches versus models for detecting spam?
rulesguy
# Are there any advantages of using rules-based approaches versus models for detecting spam?

(Asked by: username_0 on 2019-03-21)

Suppose that we have unlabeled data. That is, all we have are a collection of emails and want to determine whether any of them is spam or not. Let's say we have $1,000$ rules to determine whether a particular email is spam or not. For example, one rule could be that a sender's email address should not contain the text  `no_reply` . If an email does contain this text, then it would be classified as spam.

*What are the advantages/disadvantages of a rules-based approach for detecting spam vs. a non-rules-based approach/unsupervised methods for detecting spam?*

Would there even be a point in constructing a non-rules based model given that we already have a rules-based model? Could we use the rules-based model to create some labeled training data and then apply supervised techniques?

**Tags:** comparison, unsupervised-learning, anomaly-detection, binary-classification, rule-based-systems


---

## 1 answers

### Answer by username_1 on 2019-03-22. Score: 0

Yes, there would be a point. Assuming your rule set is accurate, then you can use data classified with it to train a model. This model can be expected to be more robust and properly categorise emails that your rule-set will not handle.

Why? Machine learning algorithms generally work on features, and identify relationships between those features that lead to a classification decision. A human rule author basically does the same, but they might not notice subtle relationships; a good ML algorithm, however, might pick those up.

So you could have a hybrid model, where you first use your rule-based classifier, and anything that does not get classified is then run through the ML classifier.

> **Comments:**
> **username_2**: This. But you should also take into account that your rules and therefore your training data have incorrect classifications. For example, `no_reply` as a localpart is relatively common for automatically generated emails that are not spam at all, but requested by the recipient. My experience (I've built a deep learning spam classifier) is that with a very basic feature set reasonable results can be achieved (after training on manually classified samples).  
> **username_2**: However, for production quality, you need several iterations of looking at the results (especially misclassifications) and adjusting feature extraction.  

---

