Answers.Body: <blockquote>
  <p>Why would an AI researcher publish their breakthrough models solely on arxiv?</p>
  </blockquote>
  <p>Unwilling to spend on going through <a href="https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment/" rel="nofollow noreferrer">peer review process</a> and other constraints such as <a href="https://academia.stackexchange.com/q/57071/452">page limits</a>.</p>
  <blockquote>
  <p>Even though experts may review articles on arxiv, what sort of gatekeeping is in place to ensure the results and techniques explained in arxiv articles are valid, and not due to errors, misunderstandings, or perhaps even outright making things up?</p>
  </blockquote>
  <p>None. FYI: <a href="https://academia.stackexchange.com/q/32367/452">Why doesn't arXiv have a comment section?</a>. It is even ok to <a href="https://academia.stackexchange.com/q/86346/452">upload joke papers to arXiv</a>.</p>
  <blockquote>
  <p>As an example, when I look at <a href="https://openai.com/publications/" rel="nofollow noreferrer">OpenAI's publications page</a>, it's a bunch of arxiv links, and no indication that any were accepted at any journal.</p>
  </blockquote>
  <p>OpenAI sometimes publishes at conferences e.g. <a href="https://openreview.net/forum?id=TG8KACxEON" rel="nofollow noreferrer">Training language models to follow instructions with human feedback</a> was published at NeurIPS 2022. But their <a href="https://arxiv.org/pdf/2203.02155.pdf" rel="nofollow noreferrer">arXiv version</a> is much longer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-23T09:12:22.940Z
Answers.Text: @yters yes but good investment or acquisition teams typically consult AI experts before deciding.
Answers.UserDisplayName: Franck Dernoncourt
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T18:59:44.900Z
Answers.Text: Is there a danger of publications giving the appearance of rigorous review, e.g. if a company is trying to raise venture capital and wants to show potential investors they have a solid product?
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T18:35:09.097Z
Answers.LastActivityDate: 2023-02-22T22:33:54.423Z
Answers.LastEditDate: 2023-02-22T22:33:54.423Z
Answers.UserDisplayName: Franck Dernoncourt
Body: <p>My question is in general why would an AI researcher choose to publish solely on arxiv, and what are the downsides and upsides?</p>
  <p>I understand on the one hand arxiv allows for rapid release of results.  On the other hand, publishing on arxiv does not undergo the same scrutiny as a journal publication.  The latter seems problematic if publishing on arxiv gives the appearance of having undergone rigorous peer review, since acceptance to arxiv is very lax in comparison to a journal.  Additionally, if there are problems with research published on arxiv, there is not the same level of reliability the problems will be addressed as in a mainstream journal.</p>
  <p>As an example, when I look at <a href="https://openai.com/publications/" rel="nofollow noreferrer">OpenAI's publications page</a>, it's a bunch of arxiv links, and no indication that any were accepted at any journal.</p>
  <p>Their recent work, especially with ChatGPT, is groundbreaking and would make a huge impact in the research community.  So it  doesn't make sense to me why the ChatGPT results, and the like, haven't been submitted to an academic journal.</p>
  <p>Additionally, in a situation like this, where a great breakthrough is being claimed, there is not the rigor of fact checking that comes with a journal publication.</p>
  <p><a href="https://ai.stackexchange.com/questions/30036/why-many-deep-learning-research-papers-continue-to-be-in-arxiv?rq=1">This</a> is a similar question, asking why most deep learning papers are published on arxiv instead of in journals.</p>
  <p>Even though experts may review articles on arxiv, what sort of gatekeeping is in place to ensure the results and techniques explained in arxiv articles are valid, and not due to errors, misunderstandings, or perhaps even outright making things up?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T08:59:37.417Z
Comments.Text: Are you trying to claim that papers submitted to arxiv are *not* scrutinized by experts? I suspect far more experts have looked at the arxiv article than would have been exposed to it via peer review for a print journal prior to publication. When an artcle *is* printed in such a journal, you typically don't have access to the review comments as a normal reader. So could you clarify - are you looking for some technical gating on the published article - that it has been verified by experts other than those that work for OpenAI? Or do you want to know what OpenAI's publish policy is and why?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T09:41:37.627Z
Comments.Text: Do you know that OpenAI didn't submit their work to any journals? The submission process takes several months sometimes years, so maybe they did submit but the articles are still in the submission process.
Comments.UserDisplayName: quarague
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T10:45:47.307Z
Comments.Text: Often, an initial version of a paper is shared on Arxiv and later submitted to a journal or conference. The submission to the journal may never happen, but I doubt it doesn't happen for OpenAI (in most cases) - that indeed would be surprising. This is not really an AI question, but it's about a company. If you reformulate your question to be more general, i.e. "why do researcher (first) share their work on Arxiv rather than in journal/conferences?" that may be considered a question that is on-topic here. Right now, you're asking a question about a company, so this is not really an AI topic.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T12:52:57.897Z
Comments.Text: @nbro I made the question more general.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T15:32:48.967Z
Comments.Text: @nbro: With a company as embedded in the AI space as OpenAI, I think we can allow direct questions about them. The difference between the general question and company-specific question is quite large here. The general question is easily answerable, but slightly opinion-based. The company-specific question might not be answerable, but if it is it will be because OpenAI have shared their publishing policies and it would directly address the OP's original issue (put simply, the OP does not think OpenAI are honest about the capabilities of their model, and wants to see some independent review)
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T17:28:36.197Z
Comments.Text: A simple answer is they want to be "the first" to establish that idea. Considering plagiarism aside, I believe the chance that two ideas are identical are very high these days, considering that there are a lot of people working in this field. Well, I already experienced the same thing.
Comments.UserDisplayName: Minh-Long Luu
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T17:41:28.703Z
Comments.Text: I can understand being first, but why no follow up submission for academic peer review through the journals?  Or, if they are doing so, why not publish their publication record?  There is the possibility they are not being entirely forthcoming in their arxiv articles, and there's no check in place to catch such an issue.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T17:55:36.067Z
Comments.Text: @NeilSlater The original question was "Why does OpenAI choose to not submit any of its results to journals, where it can be scrutinized by experts?", and this really is not a question about AI but OpenAI. I am open to discussion on meta. If people believe that a question about an AI company's internal policy should be on-topic here, then one may consider them on-topic, although I am against considering these questions on-topic, as this is not really about AI but the company. Moreover, this may lead to discussions about the company, which would get more promotion (as if it didn't have enough).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T17:56:33.803Z
Comments.Text: You write "Even though experts review articles on arxiv", but I would write "Even **if** experts review articles on arxiv" because it's not necessarily the case that a paper on Arxiv will be reviewed. Maybe it will only be read or maybe not read at all (Note: reading is not the same thing as reviewing). It seems that your original question has changed, so I would update the title.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-13T17:04:23.077Z
Comments.Text: ArXiv has [moderation](https://arxiv.org/help/moderation) and [endorsement](https://arxiv.org/help/endorsement) of their authors. ArXiv is the opposite of [viXra](https://academia.stackexchange.com/a/105595/103165).
Comments.UserDisplayName: Rob
Tags: <research><academia><chatgpt>
Title: Why would an AI researcher publish their breakthrough models solely on arxiv?
UserDisplayName: yters
ThreadText: # Why would an AI researcher publish their breakthrough models solely on arxiv?
  
  (Asked by: username_0 on 2023-01-09)
  
  My question is in general why would an AI researcher choose to publish solely on arxiv, and what are the downsides and upsides?
  
  I understand on the one hand arxiv allows for rapid release of results. On the other hand, publishing on arxiv does not undergo the same scrutiny as a journal publication. The latter seems problematic if publishing on arxiv gives the appearance of having undergone rigorous peer review, since acceptance to arxiv is very lax in comparison to a journal. Additionally, if there are problems with research published on arxiv, there is not the same level of reliability the problems will be addressed as in a mainstream journal.
  
  As an example, when I look at [OpenAI's publications page](https://openai.com/publications/), it's a bunch of arxiv links, and no indication that any were accepted at any journal.
  
  Their recent work, especially with ChatGPT, is groundbreaking and would make a huge impact in the research community. So it doesn't make sense to me why the ChatGPT results, and the like, haven't been submitted to an academic journal.
  
  Additionally, in a situation like this, where a great breakthrough is being claimed, there is not the rigor of fact checking that comes with a journal publication.
  
  [This](https://ai.stackexchange.com/questions/30036/why-many-deep-learning-research-papers-continue-to-be-in-arxiv?rq=1) is a similar question, asking why most deep learning papers are published on arxiv instead of in journals.
  
  Even though experts may review articles on arxiv, what sort of gatekeeping is in place to ensure the results and techniques explained in arxiv articles are valid, and not due to errors, misunderstandings, or perhaps even outright making things up?
  
  **Tags:** research, academia, chatgpt
  
  > **Comments:**
  > **username_1**: Are you trying to claim that papers submitted to arxiv are *not* scrutinized by experts? I suspect far more experts have looked at the arxiv article than would have been exposed to it via peer review for a print journal prior to publication. When an artcle *is* printed in such a journal, you typically don't have access to the review comments as a normal reader. So could you clarify - are you looking for some technical gating on the published article - that it has been verified by experts other than those that work for OpenAI? Or do you want to know what OpenAI's publish policy is and why?  
  > **username_2**: Do you know that OpenAI didn't submit their work to any journals? The submission process takes several months sometimes years, so maybe they did submit but the articles are still in the submission process.  
  > **username_3**: Often, an initial version of a paper is shared on Arxiv and later submitted to a journal or conference. The submission to the journal may never happen, but I doubt it doesn't happen for OpenAI (in most cases) - that indeed would be surprising. This is not really an AI question, but it's about a company. If you reformulate your question to be more general, i.e. "why do researcher (first) share their work on Arxiv rather than in journal/conferences?" that may be considered a question that is on-topic here. Right now, you're asking a question about a company, so this is not really an AI topic.  
  > **username_0**: @username_3 I made the question more general.  
  > **username_1**: @username_3: With a company as embedded in the AI space as OpenAI, I think we can allow direct questions about them. The difference between the general question and company-specific question is quite large here. The general question is easily answerable, but slightly opinion-based. The company-specific question might not be answerable, but if it is it will be because OpenAI have shared their publishing policies and it would directly address the OP's original issue (put simply, the OP does not think OpenAI are honest about the capabilities of their model, and wants to see some independent review)  
  > **username_4**: A simple answer is they want to be "the first" to establish that idea. Considering plagiarism aside, I believe the chance that two ideas are identical are very high these days, considering that there are a lot of people working in this field. Well, I already experienced the same thing.  
  > **username_0**: I can understand being first, but why no follow up submission for academic peer review through the journals?  Or, if they are doing so, why not publish their publication record?  There is the possibility they are not being entirely forthcoming in their arxiv articles, and there's no check in place to catch such an issue.  
  > **username_3**: @username_1 The original question was "Why does OpenAI choose to not submit any of its results to journals, where it can be scrutinized by experts?", and this really is not a question about AI but OpenAI. I am open to discussion on meta. If people believe that a question about an AI company's internal policy should be on-topic here, then one may consider them on-topic, although I am against considering these questions on-topic, as this is not really about AI but the company. Moreover, this may lead to discussions about the company, which would get more promotion (as if it didn't have enough).  
  > **username_3**: You write "Even though experts review articles on arxiv", but I would write "Even **if** experts review articles on arxiv" because it's not necessarily the case that a paper on Arxiv will be reviewed. Maybe it will only be read or maybe not read at all (Note: reading is not the same thing as reviewing). It seems that your original question has changed, so I would update the title.  
  > **username_5**: ArXiv has [moderation](https://arxiv.org/help/moderation) and [endorsement](https://arxiv.org/help/endorsement) of their authors. ArXiv is the opposite of [viXra](https://academia.stackexchange.com/a/105595/103165).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_6 on 2023-01-11. Score: 3
  
  > Why would an AI researcher publish their breakthrough models solely on arxiv?
  
  Unwilling to spend on going through [peer review process](https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment/) and other constraints such as [page limits](https://academia.stackexchange.com/q/57071/452).
  
  > Even though experts may review articles on arxiv, what sort of gatekeeping is in place to ensure the results and techniques explained in arxiv articles are valid, and not due to errors, misunderstandings, or perhaps even outright making things up?
  
  None. FYI: [Why doesn't arXiv have a comment section?](https://academia.stackexchange.com/q/32367/452). It is even ok to [upload joke papers to arXiv](https://academia.stackexchange.com/q/86346/452).
  
  > As an example, when I look at [OpenAI's publications page](https://openai.com/publications/), it's a bunch of arxiv links, and no indication that any were accepted at any journal.
  
  OpenAI sometimes publishes at conferences e.g. [Training language models to follow instructions with human feedback](https://openreview.net/forum?id=TG8KACxEON) was published at NeurIPS 2022. But their [arXiv version](https://arxiv.org/pdf/2203.02155.pdf) is much longer.
  
  > **Comments:**
  > **username_0**: Is there a danger of publications giving the appearance of rigorous review, e.g. if a company is trying to raise venture capital and wants to show potential investors they have a solid product?  
  > **username_6**: @username_0 yes but good investment or acquisition teams typically consult AI experts before deciding.  
  
  ---
  
  