Body: <p>Detecting ChatGPT-generated content is a contentious topic at the moment.  E.g., I've been on Reddit r/ChatGPT, and there's a constant stream of users claiming they've been unfairly accused of plagiarism.</p>
  <p>One thing I'm curious about is character frequency, i.e., how frequently each English character (a, b, ..., z) occurs in ChatGPT-generated text vs. human-generated text.  I'm not sure if a statistically significant difference is present, and maybe there's research into this.  My guess would be that it's almost impossible to detect a difference.</p>
  <p><strong>Question</strong>: Does (English) ChatGPT-generated content have statistically significantly different character frequency than human-generated content?</p>
  <p>(For this question, I want to ask about English only.)</p>
  <p>I asked <a href="https://vitalentum.net/free-chat-gpt" rel="nofollow noreferrer">ChatGPT</a> this question and it said:</p>
  <blockquote>
  <p>... Overall, it is possible that ChatGPT-generated content has statistically significant differences in character frequency compared to human-generated content, but this would depend on the specific training data and settings used to train the model. Further research would be needed to determine the extent of these differences.</p>
  </blockquote>
  <p>So maybe there's some subtle difference.  I didn't find a related post by <a href="https://cn.bing.com/search?q=Does+(English)+ChatGPT-generated+content+have+statistically+significantly+different+character+frequency+than+human-generated+content%3F" rel="nofollow noreferrer">Bing-searching the question</a>.</p>
  <hr />
  <p>I stumbled upon this paper which points out how scientists use more capital letters than ChatGPT:</p>
  <blockquote>
  <p>Scientists also use more proper nouns and/or acronyms, both of which
  are captured in the frequency of capital letters, and scientists use more numbers.<br />
  <sub>Desaire et al., <a href="https://doi.org/10.1016/j.xcrp.2023.101426" rel="nofollow noreferrer">Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools</a>, Cell Reports Physical Science 4, 101426, 2023.</sub></p>
  </blockquote>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-27T13:49:48.130Z
Comments.Text: See this related post on Linguistics: https://linguistics.stackexchange.com/questions/46703/linguistic-analysis-of-chatgpts-default-style-of-writing
Comments.UserDisplayName: Julius Hamilton
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-27T13:52:58.040Z
Comments.Text: And fingerprinting https://www.beren.io/2023-02-26-Fingerprinting-LLMs-with-unconditioned-distribution/
Comments.UserDisplayName: Julius Hamilton
Tags: <chatgpt><statistics>
Title: Does (English) ChatGPT-generated content have statistically significantly different character frequency than human-generated content?
UserDisplayName: Rebecca J. Stones
ThreadText: # Does (English) ChatGPT-generated content have statistically significantly different character frequency than human-generated content?
  
  (Asked by: username_0 on 2023-06-27)
  
  Detecting ChatGPT-generated content is a contentious topic at the moment. E.g., I've been on Reddit r/ChatGPT, and there's a constant stream of users claiming they've been unfairly accused of plagiarism.
  
  One thing I'm curious about is character frequency, i.e., how frequently each English character (a, b, ..., z) occurs in ChatGPT-generated text vs. human-generated text. I'm not sure if a statistically significant difference is present, and maybe there's research into this. My guess would be that it's almost impossible to detect a difference.
  
  **Question**: Does (English) ChatGPT-generated content have statistically significantly different character frequency than human-generated content?
  
  (For this question, I want to ask about English only.)
  
  I asked [ChatGPT](https://vitalentum.net/free-chat-gpt) this question and it said:
  
  > ... Overall, it is possible that ChatGPT-generated content has statistically significant differences in character frequency compared to human-generated content, but this would depend on the specific training data and settings used to train the model. Further research would be needed to determine the extent of these differences.
  
  So maybe there's some subtle difference. I didn't find a related post by [Bing-searching the question](https://cn.bing.com/search?q=Does+(English)+ChatGPT-generated+content+have+statistically+significantly+different+character+frequency+than+human-generated+content%3F).
  
  ---
  
  I stumbled upon this paper which points out how scientists use more capital letters than ChatGPT:
  
  > Scientists also use more proper nouns and/or acronyms, both of which
  > are captured in the frequency of capital letters, and scientists use more numbers.  
  > Desaire et al., [Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools](https://doi.org/10.1016/j.xcrp.2023.101426), Cell Reports Physical Science 4, 101426, 2023.
  
  **Tags:** chatgpt, statistics
  
  > **Comments:**
  > **username_1**: See this related post on Linguistics: https://linguistics.stackexchange.com/questions/46703/linguistic-analysis-of-chatgpts-default-style-of-writing  
  > **username_1**: And fingerprinting https://www.beren.io/2023-02-26-Fingerprinting-LLMs-with-unconditioned-distribution/  
  
  ---
  
  ## 0 answers
  
  