Answers.Body: <p>No, AI solves literally the problem of the <em>&quot;too many possible combinations&quot;</em></p>
  <p>Indeed, already nowadays most of the spam is managed by some AI system, combined with some rule based on the origin of emails and certifications of the domain, however, the main problem are adversarial attack</p>
  <p>Adversarial attacks are special string that fools the AI to believe that an email is save even if it's clear for any human that it's not. Recently a paper has shown that there is even generalization on these adversarial examples over multiple AIs, so a single string might break multiple systems at once</p>
  <p>(Gmail already uses AI to detect spams, and every time you mark an email as spam, Google uses that information to improve the AI system)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-19T00:25:20.317Z
Answers.Text: How are these AI-based systems different from rule-based systems?
Answers.UserDisplayName: Amazon Dies In Darkness
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-19T08:05:13.967Z
Answers.Text: If we play with examples, then consider an email list that is legitimate and has been subscribed to by the user, but that the user does not read anymore? Is it spam or not?
Answers.UserDisplayName: mavavilj
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-19T08:06:17.210Z
Answers.Text: Do you mean by "too many possible combinations" that it's about reducing volume and not precision? What if it does errors (i.e. loses important mail)?
Answers.UserDisplayName: mavavilj
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-20T10:46:39.983Z
Answers.Text: @RockPaperLz-MaskitorCasket one is made out of handcrafted rules and the other one has learnt the rules?..
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-20T10:47:11.500Z
Answers.Text: @mavavilj the definition of span depends on what the owner of the system wants
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-18T17:51:31.270Z
Answers.LastActivityDate: 2023-09-18T17:51:31.270Z
Answers.UserDisplayName: Alberto
Answers.Body: <p>While &quot;human-made labelling&quot; may not achieve perfect accuracy, it's worth considering alternative approaches. One such option involves employing AI-driven labeling techniques, often referred to as self-labelling. This method involves training the model on data that it labels autonomously.</p>
  <p>Furthermore, it's worth exploring the inclusion of supplementary information that models can leverage to enhance their predictions. This may encompass data points like dates, email senders, and other pertinent details. Such additional context can significantly contribute to the model's overall predictive capabilities.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-19T14:42:17.633Z
Answers.Text: This makes it sound tough still, because now it's also about understanding what a human might want to do. I just erased about 20k emails from my bloated inboxes (they were from a span of about 6 years), and I think that an algorithm would not have sped the work at all. It could've added work by requiring me to monitor what the algorithm does. If we'd use speed as a measure, then I think a human can maybe check 1-3 emails in a second so that the human is assured about the content.
Answers.UserDisplayName: mavavilj
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-19T13:58:00.277Z
Answers.LastActivityDate: 2023-09-19T13:58:00.277Z
Answers.UserDisplayName: Kanan Suleyman
Body: <p>How successful are the state-of-the-art (2023) email filters really?</p>
  <p>Some references claim that spam detection may reach high accuracy in test settings, but I've thought that email filtering should essentially be as large problem as filtering a whole language. Because one cannot guarantee anything about strings in emails. There will simply be too many possible combinations of strings.</p>
  <p>Spam filters might work, but I doubt they can ever reach 100% accurate labeling. And if they don't, they cannot useful, because even losing few important message can be intolerable.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-20T09:15:19.993Z
Comments.Text: Please try to ask an objectively answerable question (particularly in the title). The question 'how difficult is something' is not an objectively answerable question. Try to rephrase it such that an objective answer can be given that satisfies the question. Something along the lines (for example) of 'Which aspects of email filtering contribute most to its perceived difficulty?' or the like, depending on what part of the problem interests you most.
Comments.UserDisplayName: Robin van Hoorn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-21T05:51:40.487Z
Comments.Text: @RobinvanHoorn Why is robust email filtering about objective measures?
Comments.UserDisplayName: mavavilj
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-21T05:52:42.020Z
Comments.Text: "How hard" is certainly answerable in e.g. https://en.wikipedia.org/wiki/NP-hardness sense. So an answer could e.g. reference the amount of data needed to train convolutional networks or something.
Comments.UserDisplayName: mavavilj
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-21T10:05:32.223Z
Comments.Text: I like the question, however, it is the fact that the goal of this forum is about establishing factual knowledge, and not opinion-based information (check https://ai.stackexchange.com/tour). As such, we'd like to promote the formation of factually answerable questions. Regarding your second comment, yes NP-hardness is a perfectly good topic to talk about (whether something is NP or not). However, your question was not asking about whether email filtering is NP-hard or not, it was asking about the difficulty of the problem, which is perceived differently by people.
Comments.UserDisplayName: Robin van Hoorn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-22T06:51:39.400Z
Comments.Text: @RobinvanHoorn It would not be objective to give an algorithmic complexity answer, IF a human sorter can perform some functions better. Therefore the question is supposed to contain some amount of choice like ML algorithms usually do. As given in the answer by Alberto, it can be answered by suggesting what AI *can do*. The sidebar also had e.g. this question https://ai.stackexchange.com/questions/2645/how-do-we-know-that-now-artificial-intelligence-will-really-take-off?rq=1 which seems far more subjective. A possible answer could also reflect the amount of data in languages.
Comments.UserDisplayName: mavavilj
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-23T22:30:05.590Z
Comments.Text: It seems the big companies like Apple are not investing enough on this issue because I keep receiving spam, scam, etc. It's so annoying. Whatever tool it's using, it's quite useless.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-28T05:28:56.417Z
Comments.Text: What? Goodness in mathematical measure sense is certainly not opinion-based.
Comments.UserDisplayName: mavavilj
Tags: <machine-learning><large-language-models>
Title: How successful are the state-of-the-art (2023) email filters really?
UserDisplayName: mavavilj
ThreadText: # How successful are the state-of-the-art (2023) email filters really?
  
  (Asked by: username_0 on 2023-09-18)
  
  How successful are the state-of-the-art (2023) email filters really?
  
  Some references claim that spam detection may reach high accuracy in test settings, but I've thought that email filtering should essentially be as large problem as filtering a whole language. Because one cannot guarantee anything about strings in emails. There will simply be too many possible combinations of strings.
  
  Spam filters might work, but I doubt they can ever reach 100% accurate labeling. And if they don't, they cannot useful, because even losing few important message can be intolerable.
  
  **Tags:** machine-learning, large-language-models
  > **Comments:**
  > **username_1**: Please try to ask an objectively answerable question (particularly in the title). The question 'how difficult is something' is not an objectively answerable question. Try to rephrase it such that an objective answer can be given that satisfies the question. Something along the lines (for example) of 'Which aspects of email filtering contribute most to its perceived difficulty?' or the like, depending on what part of the problem interests you most.  
  > **username_0**: @RobinvanHoorn Why is robust email filtering about objective measures?  
  > **username_0**: "How hard" is certainly answerable in e.g. https://en.wikipedia.org/wiki/NP-hardness sense. So an answer could e.g. reference the amount of data needed to train convolutional networks or something.  
  > **username_1**: I like the question, however, it is the fact that the goal of this forum is about establishing factual knowledge, and not opinion-based information (check https://ai.stackexchange.com/tour). As such, we'd like to promote the formation of factually answerable questions. Regarding your second comment, yes NP-hardness is a perfectly good topic to talk about (whether something is NP or not). However, your question was not asking about whether email filtering is NP-hard or not, it was asking about the difficulty of the problem, which is perceived differently by people.  
  > **username_0**: @RobinvanHoorn It would not be objective to give an algorithmic complexity answer, IF a human sorter can perform some functions better. Therefore the question is supposed to contain some amount of choice like ML algorithms usually do. As given in the answer by Alberto, it can be answered by suggesting what AI *can do*. The sidebar also had e.g. this question https://ai.stackexchange.com/questions/2645/how-do-we-know-that-now-artificial-intelligence-will-really-take-off?rq=1 which seems far more subjective. A possible answer could also reflect the amount of data in languages.  
  > **username_2**: It seems the big companies like Apple are not investing enough on this issue because I keep receiving spam, scam, etc. It's so annoying. Whatever tool it's using, it's quite useless.  
  > **username_0**: What? Goodness in mathematical measure sense is certainly not opinion-based.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2023-09-18. Score: 1
  
  No, AI solves literally the problem of the *"too many possible combinations"*
  
  Indeed, already nowadays most of the spam is managed by some AI system, combined with some rule based on the origin of emails and certifications of the domain, however, the main problem are adversarial attack
  
  Adversarial attacks are special string that fools the AI to believe that an email is save even if it's clear for any human that it's not. Recently a paper has shown that there is even generalization on these adversarial examples over multiple AIs, so a single string might break multiple systems at once
  
  (Gmail already uses AI to detect spams, and every time you mark an email as spam, Google uses that information to improve the AI system)
  
  > **Comments:**
  > **username_4**: How are these AI-based systems different from rule-based systems?  
  > **username_0**: If we play with examples, then consider an email list that is legitimate and has been subscribed to by the user, but that the user does not read anymore? Is it spam or not?  
  > **username_0**: Do you mean by "too many possible combinations" that it's about reducing volume and not precision? What if it does errors (i.e. loses important mail)?  
  > **username_3**: @RockPaperLz-MaskitorCasket one is made out of handcrafted rules and the other one has learnt the rules?..  
  > **username_3**: @mavavilj the definition of span depends on what the owner of the system wants  
  
  ---
  
  ### Answer by username_5 on 2023-09-19. Score: 0
  
  While "human-made labelling" may not achieve perfect accuracy, it's worth considering alternative approaches. One such option involves employing AI-driven labeling techniques, often referred to as self-labelling. This method involves training the model on data that it labels autonomously.
  
  Furthermore, it's worth exploring the inclusion of supplementary information that models can leverage to enhance their predictions. This may encompass data points like dates, email senders, and other pertinent details. Such additional context can significantly contribute to the model's overall predictive capabilities.
  
  > **Comments:**
  > **username_0**: This makes it sound tough still, because now it's also about understanding what a human might want to do. I just erased about 20k emails from my bloated inboxes (they were from a span of about 6 years), and I think that an algorithm would not have sped the work at all. It could've added work by requiring me to monitor what the algorithm does. If we'd use speed as a measure, then I think a human can maybe check 1-3 emails in a second so that the human is assured about the content.  
  
  ---
  
  