Answers.Body: <blockquote>
    <p>Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a closed research field.</p>
  </blockquote>
  
  <p>Use of RNNs to detect spam grew out of the use of artificial networks to detect fraud in telecommunications and the financial industry as a result of the rise of attacks on long distance lines, ATMs, banks, and credit card systems in online and at data centers supporting physical points of sale.</p>
  
  <p>Although basic RNN design has given way to the newer LSTM and GRU approaches and its variants and extensions, artificial networks are now one of the primary fraud detection technologies. The dominance of this fraud detection strategy extends to SPAM detection, with its close ties to fraudulence. The spammers present the appearance of a relationship with their recipients that does not exist.</p>
  
  <p>The improvement on computing designs to recognize patterns in time series data and the application of those designs for fraud detection and countermeasures and the detection and routing or deletion of of unwanted incoming information will be a stable area of research and development for the foreseeable future.</p>
  
  <blockquote>
    <p>Question 2. What is the oldest published paper in this field?</p>
  </blockquote>
  
  <p>There is no oldest published paper. The first papers on RNN are given in this answer: <a href="https://ai.stackexchange.com/questions/8190/where-can-i-find-the-original-paper-that-introduced-rnns/9287#9287">Where can I find the original paper that introduced RNNs?</a>, but the move from pattern based detection to artificial networks to stateful artificial networks was gradual. The earliest deployments of these networks in server side or client side solutions occurred before any papers were published on the specific topic of RNN use in spam detection.</p>
  
  <blockquote>
    <p>Question 3. What are the pros and cons of using RNNs for email spam detection over other classification methods?</p>
  </blockquote>
  
  <p>Spam also has a strong temporal element.  What one considers undesirable spam in one year may be considered mission critical email a few years later, and vice versa. The performance in this space includes speed, accuracy, and reliability of classification, but also adaptation to changing user classification needs.</p>
  
  <p>It is because of these four performance characteristics in tandem that stateful networks derived from RNNs are commonly used for spam detection. The need for gated learning and forgetting at the cell level to support the variable adaptivity makes the LSTM and GRU variants common choices.</p>
  
  <p>Semantic document classification is riding on an emerging set of technologies, which are primarily artificial network designs that begin to broach the threshold of cognitive understanding of the text by storing linguistic structure in forms that allow analogy, comparison, and composition between them. Semantic algorithms that perform these operations on fuzzy associations in combination with recursive artificial networks may emerge as the dominant design as such designs are further developed.</p>
  
  <p><strong>References</strong></p>
  
  <p><a href="https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf" rel="nofollow noreferrer"><em>Detecting Spam Blogs: A Machine Learning Approach</em></a>, Pranam Kolari, 2006</p>
  
  <p><a href="https://arxiv.org/pdf/1807.02892.pdf" rel="nofollow noreferrer"><em>Automated labeling of bugs and tickets using attention-based mechanisms in recurrent neural networks</em></a>, Volodymyr Lyubinets et. al., 2018</p>
  
  <p><a href="http://weichengzhang.co/src/paper/Capstone_final_report.pdf" rel="nofollow noreferrer"><em>Spam Filter Through Deep Learning and Information Retrieval</em></a>, Weicheng Zhang, 2018</p>
  
  <p><a href="https://dl.acm.org/citation.cfm?id=569551" rel="nofollow noreferrer"><em>An Unsupervised Neural Network Approach to Profiling the Behavior of Mobile Phone Users for Use in Fraud Detection</em>, Peter Burge, John Shawe-Taylor, Journal of Parallel and Distributed Computing, Volume 61 Issue 7, July 2001, pp 915-925</a></p>
  
  <p><a href="http://www.logicnet.dk/reports/JunkDetection/JunkDetection.pdf" rel="nofollow noreferrer"><em>Intelligent junk mail detection using neural networks</em>, Michael Vinther, June 2002</a></p>
  
  <p><a href="http://222.252.30.203:8888/bitstream/123456789/12416/1/Intelligent%20systems%20and%20their%20applications.Vol.17.Iss.4.A.4.pdf" rel="nofollow noreferrer"><em>Mining for fraud, Margaret Weatherford</em>, IEEE Intelligent Systems, 2002</a></p>
  
  <p><a href="http://suraj.lums.edu.pk/~cs631s05/Papers/financial.pdf" rel="nofollow noreferrer"><em>Discovering golden nuggets: data mining in financial application</em>, D Zhang, L Zhou, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol. 34, No. 4, November 2004</a></p>
  
  <p><a href="https://arxiv.org/pdf/1009.6119.pdf" rel="nofollow noreferrer"><em>A Comprehensive Survey of Data Mining-based Fraud Detection Research</em>, C Phua, V Lee, K Smith, R Gayler - Arxiv preprint arXiv, 2007</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-14T00:35:54.287Z
Answers.LastActivityDate: 2019-01-01T21:23:10.047Z
Answers.LastEditDate: 2019-01-01T21:23:10.047Z
Answers.UserDisplayName: Douglas Daseeco
Answers.Body: <p>Initially, spam detection relied on simple rule based techniques to sort out spam. However following Paul Graham's famed article 'A Plan for Spam'  the Naive Bayes approach became very popular to the point that it became regarded as the baseline for dealing with spam. </p>
  
  <p>However following breakthroughs in deep learning, researchers have now turned their focus to neural networks to help them deal with the perenial problem of spam emails. Google recently reported that by introducing NN's to g-mail's spam filters. It took them from 99.5% to over 99.9% accuracy, suggesting that neural networks especially when used in conjunction with Bayesian classification may be effective for enhancing spam filters. You can refer to the link below to read about Google's success story <a href="https://www.wired.com/2015/07/google-says-ai-catches-99-9-percent-gmail-spam/" rel="nofollow noreferrer">https://www.wired.com/2015/07/google-says-ai-catches-99-9-percent-gmail-spam/</a></p>
  
  <p>Developing a spam filter using neural networks is basically a classification problem. You need to follow the steps below to develop such a system. (Nikhil B 2016)</p>
  
  <ul>
  <li>Collect a dataset of spam and legitimate email messages. Label these datasets. You can find email and spam datasets here <a href="http://csmining.org/index.php/spam-email-datasets-.html" rel="nofollow noreferrer">http://csmining.org/index.php/spam-email-datasets-.html</a></li>
  <li>Process these messages with feature extraction and vectorising techniques i.e. tf-idf vectorizer, word2vec, bag-of-words e.t.c. </li>
  <li>Once you have vectorised the dataset succesfully, apply a supervised learning NN algorithm i.e. radial basis network, multi-layer perceptron (MLP) or backpropagation.</li>
  <li>Train your labelled data-set on the neural network. Once training is complete you can use cross validation to calculate the precision of your trained model using the test dataset. </li>
  </ul>
  
  <p>Some of the advantages of using NN's for spam detection over other methods include.</p>
  
  <ul>
  <li>Neural networks have a higher accuracy of identifying spam as demonstrated by google. </li>
  <li>They have a lower false positive rate compared to other methods such as rule based techniques.</li>
  <li>Their main disadvantage is that they require specialised computing hardware to deploy.</li>
  </ul>
  
  <p>Some old influential papers published in the field include.  </p>
  
  <p>Machine Learning Techniques in Spam Filtering (2004) <a href="http://ats.cs.ut.ee/u/kt/hw/spam/spam.pdf" rel="nofollow noreferrer">http://ats.cs.ut.ee/u/kt/hw/spam/spam.pdf</a></p>
  
  <p>Detecting Spam Blogs: A Machine Learning Approach (2006) <a href="https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf" rel="nofollow noreferrer">https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf</a></p>
  
  <p>A review of machine learning approaches to Spam filtering (2009) <a href="https://www.sciencedirect.com/science/article/pii/S095741740900181X" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/pii/S095741740900181X</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-15T13:03:45.860Z
Answers.LastActivityDate: 2018-01-15T19:49:50.157Z
Answers.LastEditDate: 2018-01-15T19:49:50.157Z
Answers.UserDisplayName: Seth Simba
Body: <p>I am working on <a href="https://github.com/nfmcclure/tensorflow_cookbook/blob/master/09_Recurrent_Neural_Networks/02_Implementing_RNN_for_Spam_Prediction/02_implementing_rnn.py" rel="nofollow noreferrer">this code</a> for spam detection using recurrent neural networks. </p>
  
  <p>Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a closed research field. </p>
  
  <p>Question 2. What is the oldest published paper in this field? </p>
  
  <p>Quesiton 3. What are the pros and cons of using RNNs for email spam detection over other classification methods?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-04T10:26:13.247Z
Comments.Text: Please, next time, **ask only one question per post**.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-15T12:15:53.510Z
Comments.Text: I don't think any type of linguistic pattern detection is a statistics problem. It is a semantic pattern recognition problem that relies on probability and statistics as part, but only part, of the mathematical, scientific, linguistic, and psychological tool set used to approach and design a solution to the problem.
Comments.UserDisplayName: Douglas Daseeco
Tags: <classification><recurrent-neural-networks>
Title: Spam Detection using Recurrent Neural Networks
UserDisplayName: Hossein
ThreadText: # Spam Detection using Recurrent Neural Networks
  
  (Asked by: username_0 on 2017-06-10)
  
  I am working on [this code](https://github.com/nfmcclure/tensorflow_cookbook/blob/master/09_Recurrent_Neural_Networks/02_Implementing_RNN_for_Spam_Prediction/02_implementing_rnn.py) for spam detection using recurrent neural networks.
  
  Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a closed research field.
  
  Question 2. What is the oldest published paper in this field?
  
  Quesiton 3. What are the pros and cons of using RNNs for email spam detection over other classification methods?
  
  **Tags:** classification, recurrent-neural-networks
  > **Comments:**
  > **username_2**: I don't think any type of linguistic pattern detection is a statistics problem. It is a semantic pattern recognition problem that relies on probability and statistics as part, but only part, of the mathematical, scientific, linguistic, and psychological tool set used to approach and design a solution to the problem.  
  > **username_1**: Please, next time, **ask only one question per post**.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-10-14. Score: 0
  
  > Question 1. I am wondering whether this field (using RNNs for email spam detection) worths more researches or it is a closed research field.
  
  Use of RNNs to detect spam grew out of the use of artificial networks to detect fraud in telecommunications and the financial industry as a result of the rise of attacks on long distance lines, ATMs, banks, and credit card systems in online and at data centers supporting physical points of sale.
  
  Although basic RNN design has given way to the newer LSTM and GRU approaches and its variants and extensions, artificial networks are now one of the primary fraud detection technologies. The dominance of this fraud detection strategy extends to SPAM detection, with its close ties to fraudulence. The spammers present the appearance of a relationship with their recipients that does not exist.
  
  The improvement on computing designs to recognize patterns in time series data and the application of those designs for fraud detection and countermeasures and the detection and routing or deletion of of unwanted incoming information will be a stable area of research and development for the foreseeable future.
  
  > Question 2. What is the oldest published paper in this field?
  
  There is no oldest published paper. The first papers on RNN are given in this answer: [Where can I find the original paper that introduced RNNs?](https://ai.stackexchange.com/questions/8190/where-can-i-find-the-original-paper-that-introduced-rnns/9287#9287), but the move from pattern based detection to artificial networks to stateful artificial networks was gradual. The earliest deployments of these networks in server side or client side solutions occurred before any papers were published on the specific topic of RNN use in spam detection.
  
  > Question 3. What are the pros and cons of using RNNs for email spam detection over other classification methods?
  
  Spam also has a strong temporal element. What one considers undesirable spam in one year may be considered mission critical email a few years later, and vice versa. The performance in this space includes speed, accuracy, and reliability of classification, but also adaptation to changing user classification needs.
  
  It is because of these four performance characteristics in tandem that stateful networks derived from RNNs are commonly used for spam detection. The need for gated learning and forgetting at the cell level to support the variable adaptivity makes the LSTM and GRU variants common choices.
  
  Semantic document classification is riding on an emerging set of technologies, which are primarily artificial network designs that begin to broach the threshold of cognitive understanding of the text by storing linguistic structure in forms that allow analogy, comparison, and composition between them. Semantic algorithms that perform these operations on fuzzy associations in combination with recursive artificial networks may emerge as the dominant design as such designs are further developed.
  
  **References**
  
  [*Detecting Spam Blogs: A Machine Learning Approach*](https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf), Pranam Kolari, 2006
  
  [*Automated labeling of bugs and tickets using attention-based mechanisms in recurrent neural networks*](https://arxiv.org/pdf/1807.02892.pdf), Volodymyr Lyubinets et. al., 2018
  
  [*Spam Filter Through Deep Learning and Information Retrieval*](http://weichengzhang.co/src/paper/Capstone_final_report.pdf), Weicheng Zhang, 2018
  
  [*An Unsupervised Neural Network Approach to Profiling the Behavior of Mobile Phone Users for Use in Fraud Detection*, Peter Burge, John Shawe-Taylor, Journal of Parallel and Distributed Computing, Volume 61 Issue 7, July 2001, pp 915-925](https://dl.acm.org/citation.cfm?id=569551)
  
  [*Intelligent junk mail detection using neural networks*, Michael Vinther, June 2002](http://www.logicnet.dk/reports/JunkDetection/JunkDetection.pdf)
  
  [*Mining for fraud, Margaret Weatherford*, IEEE Intelligent Systems, 2002](http://222.252.30.203:8888/bitstream/123456789/12416/1/Intelligent%20systems%20and%20their%20applications.Vol.17.Iss.4.A.4.pdf)
  
  [*Discovering golden nuggets: data mining in financial application*, D Zhang, L Zhou, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol. 34, No. 4, November 2004](http://suraj.lums.edu.pk/~cs631s05/Papers/financial.pdf)
  
  [*A Comprehensive Survey of Data Mining-based Fraud Detection Research*, C Phua, V Lee, K Smith, R Gayler - Arxiv preprint arXiv, 2007](https://arxiv.org/pdf/1009.6119.pdf)
  
  
  ---
  
  ### Answer by username_3 on 2018-01-15. Score: 0
  
  Initially, spam detection relied on simple rule based techniques to sort out spam. However following Paul Graham's famed article 'A Plan for Spam' the Naive Bayes approach became very popular to the point that it became regarded as the baseline for dealing with spam.
  
  However following breakthroughs in deep learning, researchers have now turned their focus to neural networks to help them deal with the perenial problem of spam emails. Google recently reported that by introducing NN's to g-mail's spam filters. It took them from 99.5% to over 99.9% accuracy, suggesting that neural networks especially when used in conjunction with Bayesian classification may be effective for enhancing spam filters. You can refer to the link below to read about Google's success story <https://www.wired.com/2015/07/google-says-ai-catches-99-9-percent-gmail-spam/>
  
  Developing a spam filter using neural networks is basically a classification problem. You need to follow the steps below to develop such a system. (Nikhil B 2016)
  
  * Collect a dataset of spam and legitimate email messages. Label these datasets. You can find email and spam datasets here <http://csmining.org/index.php/spam-email-datasets-.html>
  * Process these messages with feature extraction and vectorising techniques i.e. tf-idf vectorizer, word2vec, bag-of-words e.t.c.
  * Once you have vectorised the dataset succesfully, apply a supervised learning NN algorithm i.e. radial basis network, multi-layer perceptron (MLP) or backpropagation.
  * Train your labelled data-set on the neural network. Once training is complete you can use cross validation to calculate the precision of your trained model using the test dataset.
  
  Some of the advantages of using NN's for spam detection over other methods include.
  
  * Neural networks have a higher accuracy of identifying spam as demonstrated by google.
  * They have a lower false positive rate compared to other methods such as rule based techniques.
  * Their main disadvantage is that they require specialised computing hardware to deploy.
  
  Some old influential papers published in the field include.
  
  Machine Learning Techniques in Spam Filtering (2004) <http://ats.cs.ut.ee/u/kt/hw/spam/spam.pdf>
  
  Detecting Spam Blogs: A Machine Learning Approach (2006) <https://www.aaai.org/Papers/AAAI/2006/AAAI06-212.pdf>
  
  A review of machine learning approaches to Spam filtering (2009) <https://www.sciencedirect.com/science/article/pii/S095741740900181X>
  
  
  ---
  
  