Answers.Body: <h2>Part 1: <em>Identifying</em> and or <em>Scoring</em> Posts</h2>
  <p>There are various levels and types of analysis one might seek, depending on the output desired. At perhaps the simplest level, <strong>text classification</strong> could place the text into binary buckets of <em>pass</em> and <em>fail</em>. This level is similar to basic spam filtering. More complex classification could involve more buckets and or give a numerical rating.</p>
  <p>A perhaps closer term for the task in question is <strong><a href="https://en.wikipedia.org/wiki/Sentiment_analysis" rel="nofollow noreferrer">sentiment analysis</a></strong>. From Wikipedia:</p>
  <blockquote>
  <p><strong>Sentiment analysis</strong> (also known as <strong>opinion mining</strong> or <strong>emotion AI</strong>) is the use of <a href="https://en.wikipedia.org/wiki/Natural_language_processing" rel="nofollow noreferrer">natural language processing</a>, <a href="https://en.wikipedia.org/wiki/Text_analytics" rel="nofollow noreferrer">text analysis</a>, <a href="https://en.wikipedia.org/wiki/Computational_linguistics" rel="nofollow noreferrer">computational linguistics</a>, and <a href="https://en.wikipedia.org/wiki/Biometrics" rel="nofollow noreferrer">biometrics</a> to systematically identify, extract, quantify, and study affective states and subjective information.</p>
  </blockquote>
  <p><strong>Natural language processing (NLP)</strong>, itself, can be sub-divided into three broad approaches: <em>symbolic</em>, <em>statistical</em>, and <em>neural</em>. These days, and particularly in the context of AI, the last option is likely given the most attention.</p>
  <p>The main reasons for choosing neural networks over traditional hard-coded logic are <em>flexibility</em>, often <em>less coding</em>, and support for <em>elusive</em>, perhaps <em>heuristic</em> logic that may otherwise be difficult to systemise. Conventional statistical methods may resolve some complexity over manual logic, yet possibly at the expense of inferential depth. With <a href="https://en.wikipedia.org/wiki/Deep_learning" rel="nofollow noreferrer">deep neural networks</a>, obtaining deeper inference, for better classification and analysis, sometimes can be as simple as training on a network with more parameters, for more neurons.</p>
  <p>Neural-based <a href="https://en.wikipedia.org/wiki/Language_model" rel="nofollow noreferrer">language models</a> like <a href="https://en.wikipedia.org/wiki/GPT-3" rel="nofollow noreferrer">GPT-3</a> and <a href="https://www.eleuther.ai/projects/gpt-neox/" rel="nofollow noreferrer">GPT-NeoX</a> (third-party <a href="https://textsynth.com/playground.html" rel="nofollow noreferrer">playground</a> available) highlight the functional yet elusive reasoning sometimes expressed by trained neural networks. Their logic is often heuristic and hidden, a phenomenon termed <a href="https://en.wikipedia.org/wiki/Black_box" rel="nofollow noreferrer">black box</a>. For those new to language models, I recommend learning more (ie. by searching &quot;GPT-3&quot; on <a href="https://www.youtube.com/results?search_query=gpt-3" rel="nofollow noreferrer">YouTube</a>) and perhaps using a playground, like the one linked above.</p>
  <p>No doubt, the specific details of implementation can vary greatly. One could, for example, run the input through a grammar classifier (neural or otherwise), followed by feeding the resultant parts of speech (plus tense) into a sentiment analysis network. Doing so could hypothetically be cheaper to train, possibly, though not necessarily, at the expense of quality. As neural computing gets cheaper, the trend is simply using a bigger network with more training data, hopefully giving a better result with minimal coding.</p>
  <p>The amount of time required for development depends on available data -- especially tagged data -- and computing. A person with experience in sentiment analysis could possibly design or adapt a network in days. But preparing the training data could be a challenge, particularly for a large network. If you had access to, say, the set of flagged posts for a popular site, the data may already have the desired level of tagging and already be enough for training.</p>
  <p>Indeed, if insufficient data is available, a grammar pre-processor, as described earlier, may help. In general, less data means smaller networks and more coding. With the right skills, off-the-shelf pre-trained language models, like those mentioned earlier, may be able to support or provide text classification, perhaps even for this purpose.</p>
  <h2>Part 2: <em>Finding</em> Trends in Language Usage</h2>
  <p>Whether done manually or though AI, once the posts have been categorised, many options exist for identifying trends in grammar and word usage.</p>
  <p>At its simplest, basic word frequency could be analysed using conventional coding. For example, a sorted word-frequency mapping could be made and compared between each population of tagged posts, with the most frequent words first.</p>
  <p>If context-sensitive properties, such as grammatical <em>mood</em>, <em>tense</em>, or <em>part of speech</em> are of interest, then a grammar pre-processor is likely in order. Such a parser could be <em>neural</em> or <em>conventional</em>. The resulting output could be <em>hierarchical</em>, a simple <em>array</em>, or perhaps a <em>map</em>. Analysis of the results could be simple, like described above for word frequency; or the results could be trained through a neural network to establish more complex inference.</p>
  <p>Since language is generally one-dimensionally arranged, <a href="https://en.wikipedia.org/wiki/Sequential_pattern_mining" rel="nofollow noreferrer">sequential pattern mining</a> may be applied in search of unknown patterns. Presumably each population of tagged posts would be analysed separately, followed by taking the difference between the result sets. Those patterns of greatest discrepancy might be insightful. This step could be applied to either the raw <a href="https://en.wikipedia.org/wiki/Lexical_analysis#Token" rel="nofollow noreferrer">tokens</a> (ie. words), or the parsed grammar output.</p>
  <p>Other options include <a href="https://en.wikipedia.org/wiki/Cluster_analysis" rel="nofollow noreferrer">cluster analysis</a> and <a href="https://en.wikipedia.org/wiki/Self-organizing_map" rel="nofollow noreferrer">self-organizing map</a>, although special pre-processing, from a broad set of possibilities, may be recommended for these.</p>
  <p>Assuming the set of pattern types is open-ended, the set of programming paths is rather unbound.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-20T13:48:04.917Z
Answers.LastActivityDate: 2022-03-21T08:42:40.503Z
Answers.LastEditDate: 2022-03-21T08:42:40.503Z
Answers.UserDisplayName: Michael
Body: <h1>Opening thoughts</h1>
  <p>This does not only apply to SE comments, but the idea in general.</p>
  <p>This is <em>not</em> a Question for <a href="https://linguistics.stackexchange.com/">Linguistics.SE</a>; those Questions might come later, <em>after</em> AI analysis. Example Linguistics Quesions:</p>
  <ul>
  <li><strong>What grammar categories might AI use to research for an analysis of abusive vs helpful discussion?</strong> <em>(before the AI research, after this OP Question is answered)</em></li>
  <li><strong>What grammar patterns can we identify from AI researched that analyzed abusive vs helpful discussion?</strong> <em>(after the AI research)</em></li>
  </ul>
  <p>This is a Question about how AI might be useful in the real world, thus helping AI programmers decide where to effectively focus energies.</p>
  <h1>AI analyzes comments and discussion</h1>
  <p>Many web apps and sites (including Facebook, YouTube, and Stack Exchange) analyze posted content using what some people call AI algorithms.</p>
  <p>Presuming this is used also for comments on posts on sites such as these...</p>
  <p>AI may take many factors into consideration, viz buzz words (type 'COVID' on a post and watch the info-notice pop up), profanities, bigotous phrases, etc.</p>
  <p>I'm curious about the results if AI analyzed just the grammar of a history of comments that were deemed &quot;abusive&quot; juxtaposed against a history that was not deemed abusive.</p>
  <h1>Why ask?</h1>
  <p>Creative-analytical thinkers like Steven Levitt (viz <a href="https://www.freakonomics.com/" rel="nofollow noreferrer">Freakonomics</a>) and <a href="https://www.gladwellbooks.com/" rel="nofollow noreferrer">Malcolm Gladwell</a> like to discuss counter-intuitive findings from research. Levitt says that this is &quot;economics&quot; (nothing to do with money). Even the video game <a href="https://leagueoflegends.com" rel="nofollow noreferrer">League of Legends</a> has stats on how often certain gaming choices (items, champion, etc) win and lose. But, we need the data. I want to know if &quot;grammar&quot; is a good place to dig.</p>
  <p>I would be curious if there were any grammar patterns that might indicate abuse, as might be found by AI research from past comments. Pardon the grammar lingo, but for example:</p>
  <ul>
  <li>Complex subjects</li>
  <li>Imperatives</li>
  <li>Subjunctives</li>
  <li>Passives</li>
  <li>Verbal pauses (Bothering to type out &quot;Um...&quot; in &quot;Um... No.&quot;)</li>
  <li>Direct Objects vs Indirect Objects vs &quot;<a href="https://english.stackexchange.com/questions/441683">Raised Objects</a>&quot;</li>
  <li>Prepositions</li>
  </ul>
  <p>...Say research finds that comments containing &quot;at&quot; were 60% more likely to be flagged as abusive. That would be great content to ask on Linguistics to see if there were other patterns to analyze.</p>
  <p>I don't know what should be analyzed, nor do I know what all grammatical categories would go into such an analysis. That would be my next Question for Linguistics.</p>
  <h1>Scope of my question</h1>
  <p>I'm trying to ask for open-ended answers, not binary (yes/no) answers, while also narrowing scope. So, let me put it this way...</p>
  <h2>Can AI be used to analyse abusive vs non-abusive discussions through grammar patterns and categories?</h2>
  <p>If so, which models or algorithms can be used to achieve that? References are also appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-20T15:38:45.690Z
Comments.Text: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/134919/discussion-between-jesse-steele-and-nbro).
Comments.UserDisplayName: Jesse
Tags: <natural-language-processing><reference-request><model-request><natural-language-understanding>
Title: How might AI analyze abusive discussion using natural language grammar?
UserDisplayName: Jesse
ThreadText: # How might AI analyze abusive discussion using natural language grammar?
  
  (Asked by: username_0 on 2022-03-20)
  
  # Opening thoughts
  
  This does not only apply to SE comments, but the idea in general.
  
  This is *not* a Question for [Linguistics.SE](https://linguistics.stackexchange.com/); those Questions might come later, *after* AI analysis. Example Linguistics Quesions:
  
  * **What grammar categories might AI use to research for an analysis of abusive vs helpful discussion?** *(before the AI research, after this OP Question is answered)*
  * **What grammar patterns can we identify from AI researched that analyzed abusive vs helpful discussion?** *(after the AI research)*
  
  This is a Question about how AI might be useful in the real world, thus helping AI programmers decide where to effectively focus energies.
  
  # AI analyzes comments and discussion
  
  Many web apps and sites (including Facebook, YouTube, and Stack Exchange) analyze posted content using what some people call AI algorithms.
  
  Presuming this is used also for comments on posts on sites such as these...
  
  AI may take many factors into consideration, viz buzz words (type 'COVID' on a post and watch the info-notice pop up), profanities, bigotous phrases, etc.
  
  I'm curious about the results if AI analyzed just the grammar of a history of comments that were deemed "abusive" juxtaposed against a history that was not deemed abusive.
  
  # Why ask?
  
  Creative-analytical thinkers like Steven Levitt (viz [Freakonomics](https://www.freakonomics.com/)) and [Malcolm Gladwell](https://www.gladwellbooks.com/) like to discuss counter-intuitive findings from research. Levitt says that this is "economics" (nothing to do with money). Even the video game [League of Legends](https://leagueoflegends.com) has stats on how often certain gaming choices (items, champion, etc) win and lose. But, we need the data. I want to know if "grammar" is a good place to dig.
  
  I would be curious if there were any grammar patterns that might indicate abuse, as might be found by AI research from past comments. Pardon the grammar lingo, but for example:
  
  * Complex subjects
  * Imperatives
  * Subjunctives
  * Passives
  * Verbal pauses (Bothering to type out "Um..." in "Um... No.")
  * Direct Objects vs Indirect Objects vs "[Raised Objects](https://english.stackexchange.com/questions/441683)"
  * Prepositions
  
  ...Say research finds that comments containing "at" were 60% more likely to be flagged as abusive. That would be great content to ask on Linguistics to see if there were other patterns to analyze.
  
  I don't know what should be analyzed, nor do I know what all grammatical categories would go into such an analysis. That would be my next Question for Linguistics.
  
  # Scope of my question
  
  I'm trying to ask for open-ended answers, not binary (yes/no) answers, while also narrowing scope. So, let me put it this way...
  
  ## Can AI be used to analyse abusive vs non-abusive discussions through grammar patterns and categories?
  
  If so, which models or algorithms can be used to achieve that? References are also appreciated.
  
  **Tags:** natural-language-processing, reference-request, model-request, natural-language-understanding
  
  > **Comments:**
  > **username_0**: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/134919/discussion-between-username_0-steele-and-nbro).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-03-20. Score: 1 | Accepted answer
  
  ## Part 1: *Identifying* and or *Scoring* Posts
  
  There are various levels and types of analysis one might seek, depending on the output desired. At perhaps the simplest level, **text classification** could place the text into binary buckets of *pass* and *fail*. This level is similar to basic spam filtering. More complex classification could involve more buckets and or give a numerical rating.
  
  A perhaps closer term for the task in question is **[sentiment analysis](https://en.wikipedia.org/wiki/Sentiment_analysis)**. From Wikipedia:
  
  > **Sentiment analysis** (also known as **opinion mining** or **emotion AI**) is the use of [natural language processing](https://en.wikipedia.org/wiki/Natural_language_processing), [text analysis](https://en.wikipedia.org/wiki/Text_analytics), [computational linguistics](https://en.wikipedia.org/wiki/Computational_linguistics), and [biometrics](https://en.wikipedia.org/wiki/Biometrics) to systematically identify, extract, quantify, and study affective states and subjective information.
  
  **Natural language processing (NLP)**, itself, can be sub-divided into three broad approaches: *symbolic*, *statistical*, and *neural*. These days, and particularly in the context of AI, the last option is likely given the most attention.
  
  The main reasons for choosing neural networks over traditional hard-coded logic are *flexibility*, often *less coding*, and support for *elusive*, perhaps *heuristic* logic that may otherwise be difficult to systemise. Conventional statistical methods may resolve some complexity over manual logic, yet possibly at the expense of inferential depth. With [deep neural networks](https://en.wikipedia.org/wiki/Deep_learning), obtaining deeper inference, for better classification and analysis, sometimes can be as simple as training on a network with more parameters, for more neurons.
  
  Neural-based [language models](https://en.wikipedia.org/wiki/Language_model) like [GPT-3](https://en.wikipedia.org/wiki/GPT-3) and [GPT-NeoX](https://www.eleuther.ai/projects/gpt-neox/) (third-party [playground](https://textsynth.com/playground.html) available) highlight the functional yet elusive reasoning sometimes expressed by trained neural networks. Their logic is often heuristic and hidden, a phenomenon termed [black box](https://en.wikipedia.org/wiki/Black_box). For those new to language models, I recommend learning more (ie. by searching "GPT-3" on [YouTube](https://www.youtube.com/results?search_query=gpt-3)) and perhaps using a playground, like the one linked above.
  
  No doubt, the specific details of implementation can vary greatly. One could, for example, run the input through a grammar classifier (neural or otherwise), followed by feeding the resultant parts of speech (plus tense) into a sentiment analysis network. Doing so could hypothetically be cheaper to train, possibly, though not necessarily, at the expense of quality. As neural computing gets cheaper, the trend is simply using a bigger network with more training data, hopefully giving a better result with minimal coding.
  
  The amount of time required for development depends on available data -- especially tagged data -- and computing. A person with experience in sentiment analysis could possibly design or adapt a network in days. But preparing the training data could be a challenge, particularly for a large network. If you had access to, say, the set of flagged posts for a popular site, the data may already have the desired level of tagging and already be enough for training.
  
  Indeed, if insufficient data is available, a grammar pre-processor, as described earlier, may help. In general, less data means smaller networks and more coding. With the right skills, off-the-shelf pre-trained language models, like those mentioned earlier, may be able to support or provide text classification, perhaps even for this purpose.
  
  ## Part 2: *Finding* Trends in Language Usage
  
  Whether done manually or though AI, once the posts have been categorised, many options exist for identifying trends in grammar and word usage.
  
  At its simplest, basic word frequency could be analysed using conventional coding. For example, a sorted word-frequency mapping could be made and compared between each population of tagged posts, with the most frequent words first.
  
  If context-sensitive properties, such as grammatical *mood*, *tense*, or *part of speech* are of interest, then a grammar pre-processor is likely in order. Such a parser could be *neural* or *conventional*. The resulting output could be *hierarchical*, a simple *array*, or perhaps a *map*. Analysis of the results could be simple, like described above for word frequency; or the results could be trained through a neural network to establish more complex inference.
  
  Since language is generally one-dimensionally arranged, [sequential pattern mining](https://en.wikipedia.org/wiki/Sequential_pattern_mining) may be applied in search of unknown patterns. Presumably each population of tagged posts would be analysed separately, followed by taking the difference between the result sets. Those patterns of greatest discrepancy might be insightful. This step could be applied to either the raw [tokens](https://en.wikipedia.org/wiki/Lexical_analysis#Token) (ie. words), or the parsed grammar output.
  
  Other options include [cluster analysis](https://en.wikipedia.org/wiki/Cluster_analysis) and [self-organizing map](https://en.wikipedia.org/wiki/Self-organizing_map), although special pre-processing, from a broad set of possibilities, may be recommended for these.
  
  Assuming the set of pattern types is open-ended, the set of programming paths is rather unbound.
  
  
  ---
  
  