Answers.Body: <p>Actually it depends, generative models are specific kind of machine learning models. Generative often means a model that models the probability distribution of data <span class="math-container">$p(x)$</span>, you cannot do translation using this kind of model.</p>
  <p>But also you can build conditional generative models, where the probability of data points <span class="math-container">$x$</span> conditioned to some input <span class="math-container">$y$</span> is modeled <span class="math-container">$p(x \, | \, y)$</span>, and in this case it is possible to perform a translation task using a generative model, where <span class="math-container">$x$</span> is the translated text, and <span class="math-container">$y$</span> is the original text.</p>
  <p>So it all depends on the model you use for the task, if you use a (conditional ) generative model to perform translation, assuming that the model is trained for that task (like ChatGPT), then yes, translation can be done using generative models.</p>
  <p>The concept of translation in AI/ML is more general, to just produce an output given an input, for example, <a href="https://arxiv.org/abs/1703.10593" rel="nofollow noreferrer">CycleGAN</a> is a generative model that translates between two sets of unpaired images, from horse to zebra and viceversa. And GANs are generative models.</p>
  <p>About the specific question on DeepL and Google Translate, it depends on what underlying model they use, if at some point they use generative models to perform translation, then they should also be flagged as generative AI.</p>
  <p>In general the concept of Generative AI has... deformed over time... (due to hype). Generative AI has existed long before LLMs and Difussion models like Dall-E and Stable Difussion, just look at Generative Adversarial Networks (GANs),</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T12:11:36.420Z
Answers.Text: We're not asking if you _can_. We're asking if these translation tools _are_ generative. Your answer is basically "we don't know"... Considering the accuracy of the translations, and the tendency for generative AI to generate inconsistent output, I'm fairly sure generative AI doesn't play a role in them.
Answers.UserDisplayName: Cerbrus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T16:05:42.807Z
Answers.Text: @Cerbrus The OP is asking more generally, if language translation can fall into the category of generative AI, and the answer is definitely Yes.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T16:46:19.827Z
Answers.Text: @Dr.Snoopy: you understood me right. Still I agree with Cerbrus on one point: your answer did not directly answer my question about common usage of the term Generative AI. However, your former comment does, thanks for clarifying.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T19:14:15.363Z
Answers.Text: @DocBrown Good, I integrated that comment into my answer now.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T11:26:11.717Z
Answers.LastActivityDate: 2024-01-25T19:14:01.277Z
Answers.LastEditDate: 2024-01-25T19:14:01.277Z
Answers.UserDisplayName: Dr. Snoopy
Answers.Body: <p>The distinction between &quot;generative&quot; and &quot;non-generative&quot; AI isn't an especially useful one. A language model (to first approximation) takes a sentence as input and tells you how probable it is that a native speaker would phrase it that way. Any remotely decent translation system is going to contain a language model, either as a separate component or integrated into the translation model proper.  This helps ensure that the text it outputs is grammatically correct and scans well. Without a language model, you get a word-for-word translation that won't sound natural at all.</p>
  <p>But if you have a probability distribution over the next word given the previous words, not only can you answer how probable an existing sentence is, but you can take a series of words and sample the next word in proportion to its probability under that model. You then put that word in the context and generate a new one, and so on. Any model that can evaluate the probability of a sentence can be trivially modified to generate a sentence. Similarly, you can take any generative model and output the distribution over the next word instead of just selecting a single one; the product of these successive distributions will be the joint distribution over sentences of that length.  The models are exactly the same in either case; only the user interface differs.</p>
  <p>(Of course, when setting <em>policies</em>, those user interface details might actually matter, since we might want to encourage users to fix their grammar but discourage them from posting answers when they have no idea if they're correct. But those details are entirely uninteresting mathematically.)</p>
  <p>Historically, the use of language models in things like machine translation and voice recognition are the serious uses, and using them to generate text is a neat side task that can be used to sanity check the sort of sentences it thinks are representative of the model. (Text generators can also be fun toys, of course, which is how they're being used in things like ChatGPT.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T21:19:45.907Z
Answers.Text: Maybe the distinction is not clear cut, but obviously there are AI systems which clearly do not count as GenAI, and others which clearly count as such for the majority of people. In the middle, there is a grey area. Do you think systems like Google Translate belong more clearly into the "non-GenAI" catgegory, the "GenAI" category or somewhere into that grey area?
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T22:34:13.343Z
Answers.Text: @DocBrown The problem is that the classification "Generative AI" is not based on strict mathematical or engineering properties. Likewise, banning use of "generative AI" cannot be relaibly based on a model containing some mathematical or enginering element. Translation systems are, to me, generative systems. They sample from a probability distribution of outputs, and to create a translated sequence, usually reinsert what they just sampled as "written so far", just like LLMs performing text completion. The models are very similar. What is different are the task-based constraints on ground truth.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T22:49:34.127Z
Answers.Text: @NeilSlater: yes, thanks. This confirms to me that Generative AI is a vague term which is commonly used for a lot more systems than the current AI ban on Stack Overflow seems to be intended for. Hence I think it needs to be clarified whenever it is used in a certain context. I miss exactly that clarification in SO's current AI policy text.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T07:22:29.353Z
Answers.Text: @DocBrown I think focusing on a strict definition for the ban is looking at the wrong thing. What the authors of the ban want to prevent is use of low-effort machine assisted answers that have a high chance of looking authoratitve, but that contain mistakes. The ban is a rough heuristic towards that goal. Looking for loopholes or theoretical overreach, either as a poster or as a mod, is missing the point.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T07:40:53.613Z
Answers.Text: @NeilSlater: yes, absolutely, but I am not looking for a strict definition for the ban, I am looking for a better heuristics which is not so single-edged as the current phrasing is. My post here, however, is an attempt to question my POV that a better wording is needed, since the term GenAI has IMHO a broader meaning than some of the MSO protagonists are pretending.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T16:12:40.090Z
Answers.Text: @DocBrown It's not that the meaning is broad so much as that the meaning is based on "how it's used" rather than "what it is". Given that, it would probably make sense to phrase the policies as something like "Don't use language models to create semantically significant parts of your answer (as opposed to modifying an answer you wrote, without changing the semantics (e.g., via translation or grammar improvement)." Say what you don't want the users doing instead of trying to exhaustively list all the tools they might use to do it.
Answers.UserDisplayName: Ray
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T17:34:57.027Z
Answers.Text: @Ray:  That sounds really good - that is something I would greatly support. I understand that the policy lists examples of well-known tools which are banned, to make it unmistakable clear clear whats forbidden. An additional paragraph about responsible use of AI could state exactly what you wrote - other AI tools might be used for improvements of an answer as long they don't add or change the semantics of any input a user of such a tool provides.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T20:41:39.503Z
Answers.LastActivityDate: 2024-01-25T20:51:56.977Z
Answers.LastEditDate: 2024-01-25T20:51:56.977Z
Answers.UserDisplayName: Ray
Answers.Body: <p>The original question has been updated so my answer is being updated to reflect this.</p>
  <p>The question being asked can be boiled down to just &quot;What is it that makes something a generative AI?&quot; I am quite familiar in my field of computer vision with Generative Adversarial Networks, but I believe after looking at Large Language Models that they operate on a similar principle. I will go over both of these.</p>
  <p><a href="https://www.leewayhertz.com/generative-adversarial-networks/" rel="nofollow noreferrer">Generative Adversarial Networks</a></p>
  <p>First I will go over the GAN which I know well, though I have not implemented one in almost 8 years. In a GAN, you have 2 machine learning models, one is generative, and one is not. There is a non-generative model called the discriminator who's job is to determine if an image belongs in the dataset or not. Then you have the generative model who is considered an attacker or adversary, who is there to try to fool the discriminator, by producing an image that the discriminator thinks belongs to the dataset, but it does not.</p>
  <p>During training we cycle between training the discriminator on the dataset until it cannot be fooled. Then we train the adversary until it can fool the discriminator. Then we add the adversary's images to the dataset and go back to the start again. We continue this process until the discriminator can no longer be trained to tell the difference any more, and the adversary wins. Now the GAN is complete and we discard the discriminator.</p>
  <p>The GAN can be altered to have an input parameter where you can select a class, and it will attempt to create an output of that class to fool the discriminator with that class. This is an optional feature, but something many GANs try to implement.</p>
  <p><a href="https://medium.com/@surabhimali/llms-and-gans-the-ai-technologies-that-will-create-our-new-reality-part-4-aiseries-2d3b2c757b0b" rel="nofollow noreferrer">LLM vs GAN</a></p>
  <p>LLMs on the other hand looks to only have one model. If you dig into it though, you will find out there really is 2!.</p>
  <p><a href="https://medium.com/low-code-for-advanced-data-science/creating-a-local-llm-vector-store-from-pdfs-with-knime-and-gpt4all-311bf61dd20e" rel="nofollow noreferrer">Creating LLM Vector Stores</a></p>
  <p>The Vector store is a model too! It is a generative model that provides an output to the final model that interprets the output. So you have a model that is trained on a large language database, and a vector store that is trained to output segments of words similar to the input it is given. Both the input you give and the input the vector store outputs is given to the regular model, and it takes that and provides an output to you.</p>
  <p>So mechanically, what ties these two types of generative AI together? How can we create a mechanical definition? You have a normal network, either a discriminator, or interpreter, and then you apply to it a generative network, either a adversary or tokenizer(?) respectively during training. When you are done, either you keep them together in the latter case, or you discard the discriminator and publish the adversary in the former case, and you have a generative AI.</p>
  <p>For older translation models, they likely did not have the processing power, or technology available to perform these kinds of tasks. This is a relatively new field. They would not have been able to be generative AI. They at most would be comprised of one model during both training and publishing.</p>
  <p>Old answer continues below:</p>
  <pre><code>&gt; [arXiv:2307.15208][4] states &quot;Generative AI refers to a set of
  artificial intelligence techniques and models designed to learn the underlying
  patterns and structure of a dataset and generate new data points that plausibly
  could be part of the original dataset.&quot;
  
  Considering this, what we see is that a generative AI differs from a traditional AI in that it creates new information where none was before. A traditional AI might detect which paintings belong to Raphael, while a generative AI might try to create a new painting in Raphael's style.
  
  I think your misconception here is that all neural network AIs produce information of some kind, but they do not produce new information. The information was already there in the input in some form in a traditional AI. 
  
   - The location of the dog was in the picture already, the neural network just pointed to it and said, &quot;Here it is.&quot; 
   - The tune matched a Metallica song in a database already, the neural network just correctly identified it.
   - The information was already encoded in a foreign language, the neural network just decoded the information and then reencoded it into a new language of your choice.
  
  Now if the model had instead read the foreign language, decoded the information, and used it as a prompt to write a novel in the language of your choice, the contents of which included new information never before seen in that input, now we are talking about generative AI. This is in fact what they do.
  
  A lot of the media talks about how chat GPT can hallucinate it's answers when you ask it straight forward questions, but the truth is, it is generative AI. It is giving you answers that could have conceivably been in the answer set. They just aren't. This is what some experts mean by it hallucinating up new answers. It's pulling them out of the void like a generative AI is supposed to do. It just isn't what the average person expects this type of program to do.
  
  In short, what the difference between a traditional AI and a generative AI is that a traditional AI learns a dataset, and tries to tell you something about the input with respect to the dataset. The generative AI takes the input and tries to produce something similar that it thinks would be from the same grouping in what it believes the dataset should be.
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:01:18.923Z
Answers.Text: Strange. It did not appear to be behind a paywall for me, but you are correct in that it is a medical paper, but it is also the correct definition as well. I don't see what the issue is with the definition. https://arxiv.org/pdf/2307.15208.pdf
Answers.UserDisplayName: Chthonic One
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:07:12.030Z
Answers.Text: The definition in that paper (about medical imaging) looks like it was written by authors who picked a phrasing which suits their needs in context of that paper. This is nothing I would consider to be a valid statement about the general, common use of the term GenAI in the field of AI, more *one possible use* of the term in the field. In that context, that is ok and surely correct, but I would not draw any conclusions about the terminolgy for translator systems from it.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:16:08.913Z
Answers.Text: ... note also I never said I believed all artificial neural network systems count as GenAI, that would have been clearly nonsense. I was talking specifically about translators, where you feed one text in as input, and get a - translated - text as output. My understanding of this process is that is is not uncommon to call this "generation" (of a text in a diffferent language).
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:18:57.437Z
Answers.Text: As I noted in my answer, this is a misconception on your part. Generation in this case has more than one meaning. Yes, it generates text. No it does not generate new information that did not exist in the original message, at least not intentionally. Of course mistranslations do happen, and when they do, new information can be accidentally injected, but that is considered a bug, and not a feature.
Answers.UserDisplayName: Chthonic One
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:23:59.910Z
Answers.Text: So you deny that the common usage of the term GenAI in the field can include translator systems (as long as they are based on what you call "traditional AI")? Note I mean the common usage of the term over the last decade and maybe earlier, not the "popular usage of the last year".
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:26:46.673Z
Answers.Text: I deny the fact that the definition used by me or by others here is wrong simply because it is not used in a more general case. The people who create this definition are the ones pioneering the systems, and they do so by writing papers on what they do in specific cases. You can learn what they mean by when they define their terminology in their papers. When all of the papers agree on a definition, you know what it means in the general case. There isn't going to be a paper that defines it for you in the general case. Other than that, the definition is set arbitrarily by those that do this work.
Answers.UserDisplayName: Chthonic One
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:28:25.140Z
Answers.Text: As such, the definition says that a generative AI will produce a result out of nothing that will match what it believes should be in the dataset. A translation AI does not produce a result out of nothing. It produces it from it's input. By definition, a translation AI is not generative.
Answers.UserDisplayName: Chthonic One
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T23:29:18.340Z
Answers.Text: I did not say the definition in that paper is wrong, quite the opposite. I only said it is most probably not telling us what the common usage in the field is, because it is just one paper with a specific context.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T23:34:23.837Z
Answers.Text: Ok, think that new answer is a lot better that your first since it offers a much deeper insight in what counts as generative AI in the field, thanks for that. Still it leaves me puzzled what this precisely means for my title question.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-28T06:46:27.370Z
Answers.Text: I'm glad I could help.  It took a while to figure out where our disconnects were. Sometimes when you learn a field you forget how to talk to people who haven't learned it yet, you know?
Answers.UserDisplayName: Chthonic One
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T22:14:44.940Z
Answers.LastActivityDate: 2024-01-26T18:28:16.063Z
Answers.LastEditDate: 2024-01-26T18:28:16.063Z
Answers.UserDisplayName: Chthonic One
Answers.Body: <p>No, they are not &quot;generative&quot;.</p>
  <p>As clearly <a href="https://genai.stackexchange.com/questions/313/is-google-translate-the-web-application-a-generative-artificial-intelligence">explained by Wicket</a> on the GenAI Beta site, sourced from <a href="https://genai.meta.stackexchange.com/questions/163/what-is-generative-ai-genai-according-to-this-site/169#169">SirBenet's answer</a> on the same site, Google Translate isn't <em>&quot;generative&quot;</em>:</p>
  <blockquote>
  <p>output could be considered too tightly defined by the input text</p>
  </blockquote>
  <p>These translation tools don't <em>generate</em> new text. They <em>translate</em>.<br />
  Sure, they use AI to infer meaning to result in better translations, but you can't tell them to <em>&quot;Translate Macbeth to Swahili&quot;</em>...</p>
  <p>You'll never get <em>more</em> out of the tools than you put into them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T11:25:07.893Z
Answers.Text: Sirbenet's answer was refering to [What is generative AI (GenAI) according to this site?](https://genai.meta.stackexchange.com/q/163), where "this site" means "Gen AI meta". I am asking about the **common use of the term in the field of AI**.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T12:27:37.343Z
Answers.Text: In this question here, I am asking about the general use of the term in the field of AI, not just at SE or SO.
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T12:28:15.773Z
Answers.Text: You're free to ignore my answer.
Answers.UserDisplayName: Cerbrus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T11:01:11.050Z
Answers.LastActivityDate: 2024-01-25T11:01:11.050Z
Answers.UserDisplayName: Cerbrus
Answers.Body: <p>Generative AI, as defined by IBM research, refers to deep-learning models capable of creating new content, be it text, images, or other media, based on their training data. This definition indeed encompasses models like GPT-3 or GPT-4, which can generate text in various styles and formats, including translations.</p>
  <p>However, when it comes to full-text translators like Google Translate, DeepL, or Bing Translate, there's a nuanced difference. These systems are typically based on neural machine translation (NMT) models, a specific application of deep learning tailored for the task of translating text from one language to another. While these NMT systems are indeed 'generative' in the sense that they produce new text in a target language, their primary function is not to create original content but to convert existing content from one language to another as accurately as possible.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T13:33:13.680Z
Answers.Text: I would also say that typical models that people call "Generative AI" like ChatGPT and Dall-E, are not meant to create original content either, so that would not be a good criteria to define generative models.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T13:34:57.650Z
Answers.Text: Wait... What _are_ they meant to do then, if not "create original content"?
Answers.UserDisplayName: Cerbrus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T13:36:02.257Z
Answers.Text: So is my understanding correct that in the field of AI, "Generative AI" is a term which can include more than just LLMs? Or would you say that this interpretation is quite unusual and too far-fetched?
Answers.UserDisplayName: Doc Brown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T16:03:34.387Z
Answers.Text: Depending on the model, they are trained to predict next word given some context (LLMs), or to imitate a training set of natural images (Stable Difussion and Dall-E) , creating new content is a strong claim that is generally unproven. For example see the NYT lawsuit against OpenAI.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T16:03:48.387Z
Answers.Text: @DocBrown Correct, Generative AI existed long before LLMs.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T16:15:40.333Z
Answers.Text: _"see the NYT lawsuit against OpenAI."_ is not very helpful. If you ask a LLM like ChatGPT to generate a story about imaginary characters you made up living on the moon, it will. You can't tell me that that isn't "new content"...
Answers.UserDisplayName: Cerbrus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T19:18:12.623Z
Answers.Text: @Cerbrus It will plagiarize an existing story with your characters, this has been shown over and over, the NYT lawsuit just shows that it can produce word by word content from the NYT. It applies to almost everything that is generated, another example about jokes: https://arstechnica.com/information-technology/2023/06/researchers-discover-that-chatgpt-prefers-repeating-25-jokes-over-and-over/
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T20:13:22.420Z
Answers.Text: @Dr.Snoopy "It will" is not the same as "it can".  The lawsuit claims to have shown you *can* get it to reproduce specific content.  To show that this is the only thing it does, you have to show *all* content it produces is plagiarism, which specific examples fail to prove.
Answers.UserDisplayName: Yakk
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T20:23:32.147Z
Answers.Text: @Yakk No, it is a machine learning model, it only works on its training set boundaries, to claim novelty in generating content, you need specific evidence that has not been produced at all, while there is plenty of research showing the contrary, there is even a recent paper proving that Transformers do not generalize outside of their training set: https://arxiv.org/abs/2311.00871
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T21:30:25.287Z
Answers.Text: @Dr.Snoopy If I remember correctly, ChatGPT had also closely replicated a news article written after its cutoff, so it couldn't have been plagiarizing
Answers.UserDisplayName: Esther
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T07:08:11.637Z
Answers.Text: @Esther may be the news article itself was plagiarized from something that existed before chatgpt's cutoff
Answers.UserDisplayName: user17915
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T14:55:08.613Z
Answers.Text: @Dr.Snoopy I mean, a pseudo-RNG produces novel output in terms of pixel static.  Such a step is part of art generation by techniques like stable diffusion.  It might not be *interesting* novel content, but one of the algorithms inputs are samples from a larger space of novel "noise" than the total number of images ever uploaded to the internet.  Unless there is convergence in the output image from the random source, it *cannot* only be duplicating images, there aren't enough holes to put the pigeons in.  If you want *interestingly* novel content, no guarantee, but it is novel.
Answers.UserDisplayName: Yakk
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-28T21:27:10.027Z
Answers.Text: @Yakk Sorry but your explanation did not make any sense to me, Diffusion models just estimate probability distributions, there is no magic there.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-25T13:26:58.477Z
Answers.LastActivityDate: 2024-01-25T13:26:58.477Z
Answers.UserDisplayName: FledDev
Body: <p>My question relates to full-text translators that are not specifically based on LLMs. My current understanding is that the term Generative AI goes beyond LLMs and that the full-text translators (especially those which are based on artificial neural networks) also fall into this category.</p>
  <p><a href="https://en.wikipedia.org/wiki/Generative_artificial_intelligence" rel="noreferrer">In the Wikipedia article about Generative AI</a>, I could only find the statement that generative AI systems such as ChatGPT are also used for translations. That is quite obvious, but this does not answer the question of whether other full text translators are also commonly referred to as Generative AI.</p>
  <p><a href="https://research.ibm.com/blog/what-is-generative-AI" rel="noreferrer">IBM research defines Generative AI as</a></p>
  <blockquote>
  <p>Generative AI refers to deep-learning models that can generate high-quality text, images, and other content based on the data they were trained on.</p>
  </blockquote>
  <p>AFAIK DeepL, Google Translate, Bing Translate but also a few not so well-known systems falls clearly into that definition.</p>
  <p>My question was triggered by <a href="https://meta.stackoverflow.com/a/427857">an answer on Meta Stack Overflow</a> that contained the following sentence:</p>
  <blockquote>
  <p>Besides, the banner clearly states: &quot;Answers generated by artificial intelligence tools&quot;. Translations aren't generated answers. They're translations.</p>
  </blockquote>
  <p>with the implicit conclusion that it is clear to everyone that pure translators do not count as generative AI, hence it does not need any further explanation. Other participants in that thread seem to agree to that point of view. However, I think their use of terminology is not the typical use in the field of AI, and I would like to hear what the experts say.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-25T10:44:57.243Z
Comments.Text: [I found a duplicate on GenAI SE that answers this.](https://genai.stackexchange.com/questions/313/is-google-translate-the-web-application-a-generative-artificial-intelligence)
Comments.UserDisplayName: Cerbrus
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-25T17:21:56.857Z
Comments.Text: I found this interesting link about the [history of Generative AI](https://toloka.ai/blog/history-of-generative-ai/). Translators apart from LLMs are not explicitly mentioned. Still it gives a good outline about the >70 years old history of the field, the  breakthroughs since 2010, and the wide range of tools for which the term GenAI is used.
Comments.UserDisplayName: Doc Brown
Tags: <terminology><machine-translation><generative-ai>
Title: Do full-text translators such as DeepL or Google Translate fall under the term "Generative AI"?
UserDisplayName: Doc Brown
ThreadText: # Do full-text translators such as DeepL or Google Translate fall under the term "Generative AI"?
  
  (Asked by: username_0 on 2024-01-25)
  
  My question relates to full-text translators that are not specifically based on LLMs. My current understanding is that the term Generative AI goes beyond LLMs and that the full-text translators (especially those which are based on artificial neural networks) also fall into this category.
  
  [In the Wikipedia article about Generative AI](https://en.wikipedia.org/wiki/Generative_artificial_intelligence), I could only find the statement that generative AI systems such as ChatGPT are also used for translations. That is quite obvious, but this does not answer the question of whether other full text translators are also commonly referred to as Generative AI.
  
  [IBM research defines Generative AI as](https://research.ibm.com/blog/what-is-generative-AI)
  
  > Generative AI refers to deep-learning models that can generate high-quality text, images, and other content based on the data they were trained on.
  
  AFAIK DeepL, Google Translate, Bing Translate but also a few not so well-known systems falls clearly into that definition.
  
  My question was triggered by [an answer on Meta Stack Overflow](https://meta.stackoverflow.com/a/427857) that contained the following sentence:
  
  > Besides, the banner clearly states: "Answers generated by artificial intelligence tools". Translations aren't generated answers. They're translations.
  
  with the implicit conclusion that it is clear to everyone that pure translators do not count as generative AI, hence it does not need any further explanation. Other participants in that thread seem to agree to that point of view. However, I think their use of terminology is not the typical use in the field of AI, and I would like to hear what the experts say.
  
  **Tags:** terminology, machine-translation, generative-ai
  > **Comments:**
  > **username_1**: [I found a duplicate on GenAI SE that answers this.](https://genai.stackexchange.com/questions/313/is-google-translate-the-web-application-a-generative-artificial-intelligence)  
  > **username_0**: I found this interesting link about the [history of Generative AI](https://toloka.ai/blog/history-of-generative-ai/). Translators apart from LLMs are not explicitly mentioned. Still it gives a good outline about the >70 years old history of the field, the  breakthroughs since 2010, and the wide range of tools for which the term GenAI is used.  
  
  ---
  
  ## 5 answers
  
  ### Answer by username_6 on 2024-01-25. Score: 8 | Accepted answer
  
  Generative AI, as defined by IBM research, refers to deep-learning models capable of creating new content, be it text, images, or other media, based on their training data. This definition indeed encompasses models like GPT-3 or GPT-4, which can generate text in various styles and formats, including translations.
  
  However, when it comes to full-text translators like Google Translate, DeepL, or Bing Translate, there's a nuanced difference. These systems are typically based on neural machine translation (NMT) models, a specific application of deep learning tailored for the task of translating text from one language to another. While these NMT systems are indeed 'generative' in the sense that they produce new text in a target language, their primary function is not to create original content but to convert existing content from one language to another as accurately as possible.
  
  > **Comments:**
  > **username_2**: I would also say that typical models that people call "Generative AI" like ChatGPT and Dall-E, are not meant to create original content either, so that would not be a good criteria to define generative models.  
  > **username_1**: Wait... What _are_ they meant to do then, if not "create original content"?  
  > **username_0**: So is my understanding correct that in the field of AI, "Generative AI" is a term which can include more than just LLMs? Or would you say that this interpretation is quite unusual and too far-fetched?  
  > **username_2**: Depending on the model, they are trained to predict next word given some context (LLMs), or to imitate a training set of natural images (Stable Difussion and Dall-E) , creating new content is a strong claim that is generally unproven. For example see the NYT lawsuit against OpenAI.  
  > **username_2**: @DocBrown Correct, Generative AI existed long before LLMs.  
  > **username_1**: _"see the NYT lawsuit against OpenAI."_ is not very helpful. If you ask a LLM like ChatGPT to generate a story about imaginary characters you made up living on the moon, it will. You can't tell me that that isn't "new content"...  
  > **username_2**: @Cerbrus It will plagiarize an existing story with your characters, this has been shown over and over, the NYT lawsuit just shows that it can produce word by word content from the NYT. It applies to almost everything that is generated, another example about jokes: https://arstechnica.com/information-technology/2023/06/researchers-discover-that-chatgpt-prefers-repeating-25-jokes-over-and-over/  
  > **username_7**: @Dr.Snoopy "It will" is not the same as "it can".  The lawsuit claims to have shown you *can* get it to reproduce specific content.  To show that this is the only thing it does, you have to show *all* content it produces is plagiarism, which specific examples fail to prove.  
  > **username_2**: @Yakk No, it is a machine learning model, it only works on its training set boundaries, to claim novelty in generating content, you need specific evidence that has not been produced at all, while there is plenty of research showing the contrary, there is even a recent paper proving that Transformers do not generalize outside of their training set: https://arxiv.org/abs/2311.00871  
  > **username_8**: @Dr.Snoopy If I remember correctly, ChatGPT had also closely replicated a news article written after its cutoff, so it couldn't have been plagiarizing  
  > **username_9**: @Esther may be the news article itself was plagiarized from something that existed before chatgpt's cutoff  
  > **username_7**: @Dr.Snoopy I mean, a pseudo-RNG produces novel output in terms of pixel static.  Such a step is part of art generation by techniques like stable diffusion.  It might not be *interesting* novel content, but one of the algorithms inputs are samples from a larger space of novel "noise" than the total number of images ever uploaded to the internet.  Unless there is convergence in the output image from the random source, it *cannot* only be duplicating images, there aren't enough holes to put the pigeons in.  If you want *interestingly* novel content, no guarantee, but it is novel.  
  > **username_2**: @Yakk Sorry but your explanation did not make any sense to me, Diffusion models just estimate probability distributions, there is no magic there.  
  
  ---
  
  ### Answer by username_3 on 2024-01-25. Score: 7
  
  The distinction between "generative" and "non-generative" AI isn't an especially useful one. A language model (to first approximation) takes a sentence as input and tells you how probable it is that a native speaker would phrase it that way. Any remotely decent translation system is going to contain a language model, either as a separate component or integrated into the translation model proper. This helps ensure that the text it outputs is grammatically correct and scans well. Without a language model, you get a word-for-word translation that won't sound natural at all.
  
  But if you have a probability distribution over the next word given the previous words, not only can you answer how probable an existing sentence is, but you can take a series of words and sample the next word in proportion to its probability under that model. You then put that word in the context and generate a new one, and so on. Any model that can evaluate the probability of a sentence can be trivially modified to generate a sentence. Similarly, you can take any generative model and output the distribution over the next word instead of just selecting a single one; the product of these successive distributions will be the joint distribution over sentences of that length. The models are exactly the same in either case; only the user interface differs.
  
  (Of course, when setting *policies*, those user interface details might actually matter, since we might want to encourage users to fix their grammar but discourage them from posting answers when they have no idea if they're correct. But those details are entirely uninteresting mathematically.)
  
  Historically, the use of language models in things like machine translation and voice recognition are the serious uses, and using them to generate text is a neat side task that can be used to sanity check the sort of sentences it thinks are representative of the model. (Text generators can also be fun toys, of course, which is how they're being used in things like ChatGPT.)
  
  > **Comments:**
  > **username_0**: Maybe the distinction is not clear cut, but obviously there are AI systems which clearly do not count as GenAI, and others which clearly count as such for the majority of people. In the middle, there is a grey area. Do you think systems like Google Translate belong more clearly into the "non-GenAI" catgegory, the "GenAI" category or somewhere into that grey area?  
  > **username_4**: @DocBrown The problem is that the classification "Generative AI" is not based on strict mathematical or engineering properties. Likewise, banning use of "generative AI" cannot be relaibly based on a model containing some mathematical or enginering element. Translation systems are, to me, generative systems. They sample from a probability distribution of outputs, and to create a translated sequence, usually reinsert what they just sampled as "written so far", just like LLMs performing text completion. The models are very similar. What is different are the task-based constraints on ground truth.  
  > **username_0**: @NeilSlater: yes, thanks. This confirms to me that Generative AI is a vague term which is commonly used for a lot more systems than the current AI ban on Stack Overflow seems to be intended for. Hence I think it needs to be clarified whenever it is used in a certain context. I miss exactly that clarification in SO's current AI policy text.  
  > **username_4**: @DocBrown I think focusing on a strict definition for the ban is looking at the wrong thing. What the authors of the ban want to prevent is use of low-effort machine assisted answers that have a high chance of looking authoratitve, but that contain mistakes. The ban is a rough heuristic towards that goal. Looking for loopholes or theoretical overreach, either as a poster or as a mod, is missing the point.  
  > **username_0**: @NeilSlater: yes, absolutely, but I am not looking for a strict definition for the ban, I am looking for a better heuristics which is not so single-edged as the current phrasing is. My post here, however, is an attempt to question my POV that a better wording is needed, since the term GenAI has IMHO a broader meaning than some of the MSO protagonists are pretending.  
  > **username_3**: @DocBrown It's not that the meaning is broad so much as that the meaning is based on "how it's used" rather than "what it is". Given that, it would probably make sense to phrase the policies as something like "Don't use language models to create semantically significant parts of your answer (as opposed to modifying an answer you wrote, without changing the semantics (e.g., via translation or grammar improvement)." Say what you don't want the users doing instead of trying to exhaustively list all the tools they might use to do it.  
  > **username_0**: @Ray:  That sounds really good - that is something I would greatly support. I understand that the policy lists examples of well-known tools which are banned, to make it unmistakable clear clear whats forbidden. An additional paragraph about responsible use of AI could state exactly what you wrote - other AI tools might be used for improvements of an answer as long they don't add or change the semantics of any input a user of such a tool provides.  
  
  ---
  
  ### Answer by username_2 on 2024-01-25. Score: 5
  
  Actually it depends, generative models are specific kind of machine learning models. Generative often means a model that models the probability distribution of data $p(x)$, you cannot do translation using this kind of model.
  
  But also you can build conditional generative models, where the probability of data points $x$ conditioned to some input $y$ is modeled $p(x \, | \, y)$, and in this case it is possible to perform a translation task using a generative model, where $x$ is the translated text, and $y$ is the original text.
  
  So it all depends on the model you use for the task, if you use a (conditional ) generative model to perform translation, assuming that the model is trained for that task (like ChatGPT), then yes, translation can be done using generative models.
  
  The concept of translation in AI/ML is more general, to just produce an output given an input, for example, [CycleGAN](https://arxiv.org/abs/1703.10593) is a generative model that translates between two sets of unpaired images, from horse to zebra and viceversa. And GANs are generative models.
  
  About the specific question on DeepL and Google Translate, it depends on what underlying model they use, if at some point they use generative models to perform translation, then they should also be flagged as generative AI.
  
  In general the concept of Generative AI has... deformed over time... (due to hype). Generative AI has existed long before LLMs and Difussion models like Dall-E and Stable Difussion, just look at Generative Adversarial Networks (GANs),
  
  > **Comments:**
  > **username_1**: We're not asking if you _can_. We're asking if these translation tools _are_ generative. Your answer is basically "we don't know"... Considering the accuracy of the translations, and the tendency for generative AI to generate inconsistent output, I'm fairly sure generative AI doesn't play a role in them.  
  > **username_2**: @Cerbrus The OP is asking more generally, if language translation can fall into the category of generative AI, and the answer is definitely Yes.  
  > **username_0**: @Dr.Snoopy: you understood me right. Still I agree with Cerbrus on one point: your answer did not directly answer my question about common usage of the term Generative AI. However, your former comment does, thanks for clarifying.  
  > **username_2**: @DocBrown Good, I integrated that comment into my answer now.  
  
  ---
  
  ### Answer by username_5 on 2024-01-25. Score: 2
  
  The original question has been updated so my answer is being updated to reflect this.
  The question being asked can be boiled down to just "What is it that makes something a generative AI?" I am quite familiar in my field of computer vision with Generative Adversarial Networks, but I believe after looking at Large Language Models that they operate on a similar principle. I will go over both of these.
  Generative Adversarial Networks
  First I will go over the GAN which I know well, though I have not implemented one in almost 8 years. In a GAN, you have 2 machine learning models, one is generative, and one is not. There is a non-generative model called the discriminator who's job is to determine if an image belongs in the dataset or not. Then you have the generative model who is considered an attacker or adversary, who is there to try to fool the discriminator, by producing an image that the discriminator thinks belongs to the dataset, but it does not.
  During training we cycle between training the discriminator on the dataset until it cannot be fooled. Then we train the adversary until it can fool the discriminator. Then we add the adversary's images to the dataset and go back to the start again. We continue this process until the discriminator can no longer be trained to tell the difference any more, and the adversary wins. Now the GAN is complete and we discard the discriminator.
  The GAN can be altered to have an input parameter where you can select a class, and it will attempt to create an output of that class to fool the discriminator with that class. This is an optional feature, but something many GANs try to implement.
  LLM vs GAN
  LLMs on the other hand looks to only have one model. If you dig into it though, you will find out there really is 2!.
  Creating LLM Vector Stores
  The Vector store is a model too! It is a generative model that provides an output to the final model that interprets the output. So you have a model that is trained on a large language database, and a vector store that is trained to output segments of words similar to the input it is given. Both the input you give and the input the vector store outputs is given to the regular model, and it takes that and provides an output to you.
  So mechanically, what ties these two types of generative AI together? How can we create a mechanical definition? You have a normal network, either a discriminator, or interpreter, and then you apply to it a generative network, either a adversary or tokenizer(?) respectively during training. When you are done, either you keep them together in the latter case, or you discard the discriminator and publish the adversary in the former case, and you have a generative AI.
  For older translation models, they likely did not have the processing power, or technology available to perform these kinds of tasks. This is a relatively new field. They would not have been able to be generative AI. They at most would be comprised of one model during both training and publishing.
  Old answer continues below:
  > [arXiv:2307.15208][4] states "Generative AI refers to a set of
  artificial intelligence techniques and models designed to learn the underlying
  patterns and structure of a dataset and generate new data points that plausibly
  could be part of the original dataset."
  
  Considering this, what we see is that a generative AI differs from a traditional AI in that it creates new information where none was before. A traditional AI might detect which paintings belong to Raphael, while a generative AI might try to create a new painting in Raphael's style.
  
  I think your misconception here is that all neural network AIs produce information of some kind, but they do not produce new information. The information was already there in the input in some form in a traditional AI. 
  
   - The location of the dog was in the picture already, the neural network just pointed to it and said, "Here it is." 
   - The tune matched a Metallica song in a database already, the neural network just correctly identified it.
   - The information was already encoded in a foreign language, the neural network just decoded the information and then reencoded it into a new language of your choice.
  
  Now if the model had instead read the foreign language, decoded the information, and used it as a prompt to write a novel in the language of your choice, the contents of which included new information never before seen in that input, now we are talking about generative AI. This is in fact what they do.
  
  A lot of the media talks about how chat GPT can hallucinate it's answers when you ask it straight forward questions, but the truth is, it is generative AI. It is giving you answers that could have conceivably been in the answer set. They just aren't. This is what some experts mean by it hallucinating up new answers. It's pulling them out of the void like a generative AI is supposed to do. It just isn't what the average person expects this type of program to do.
  
  In short, what the difference between a traditional AI and a generative AI is that a traditional AI learns a dataset, and tries to tell you something about the input with respect to the dataset. The generative AI takes the input and tries to produce something similar that it thinks would be from the same grouping in what it believes the dataset should be.
  
  > **Comments:**
  > **username_5**: Strange. It did not appear to be behind a paywall for me, but you are correct in that it is a medical paper, but it is also the correct definition as well. I don't see what the issue is with the definition. https://arxiv.org/pdf/2307.15208.pdf  
  > **username_0**: The definition in that paper (about medical imaging) looks like it was written by authors who picked a phrasing which suits their needs in context of that paper. This is nothing I would consider to be a valid statement about the general, common use of the term GenAI in the field of AI, more *one possible use* of the term in the field. In that context, that is ok and surely correct, but I would not draw any conclusions about the terminolgy for translator systems from it.  
  > **username_0**: ... note also I never said I believed all artificial neural network systems count as GenAI, that would have been clearly nonsense. I was talking specifically about translators, where you feed one text in as input, and get a - translated - text as output. My understanding of this process is that is is not uncommon to call this "generation" (of a text in a diffferent language).  
  > **username_5**: As I noted in my answer, this is a misconception on your part. Generation in this case has more than one meaning. Yes, it generates text. No it does not generate new information that did not exist in the original message, at least not intentionally. Of course mistranslations do happen, and when they do, new information can be accidentally injected, but that is considered a bug, and not a feature.  
  > **username_0**: So you deny that the common usage of the term GenAI in the field can include translator systems (as long as they are based on what you call "traditional AI")? Note I mean the common usage of the term over the last decade and maybe earlier, not the "popular usage of the last year".  
  > **username_5**: I deny the fact that the definition used by me or by others here is wrong simply because it is not used in a more general case. The people who create this definition are the ones pioneering the systems, and they do so by writing papers on what they do in specific cases. You can learn what they mean by when they define their terminology in their papers. When all of the papers agree on a definition, you know what it means in the general case. There isn't going to be a paper that defines it for you in the general case. Other than that, the definition is set arbitrarily by those that do this work.  
  > **username_5**: As such, the definition says that a generative AI will produce a result out of nothing that will match what it believes should be in the dataset. A translation AI does not produce a result out of nothing. It produces it from it's input. By definition, a translation AI is not generative.  
  > **username_0**: I did not say the definition in that paper is wrong, quite the opposite. I only said it is most probably not telling us what the common usage in the field is, because it is just one paper with a specific context.  
  > **username_0**: Ok, think that new answer is a lot better that your first since it offers a much deeper insight in what counts as generative AI in the field, thanks for that. Still it leaves me puzzled what this precisely means for my title question.  
  > **username_5**: I'm glad I could help.  It took a while to figure out where our disconnects were. Sometimes when you learn a field you forget how to talk to people who haven't learned it yet, you know?  
  
  ---
  
  ### Answer by username_1 on 2024-01-25. Score: -1
  
  No, they are not "generative".
  
  As clearly [explained by Wicket](https://genai.stackexchange.com/questions/313/is-google-translate-the-web-application-a-generative-artificial-intelligence) on the GenAI Beta site, sourced from [SirBenet's answer](https://genai.meta.stackexchange.com/questions/163/what-is-generative-ai-genai-according-to-this-site/169#169) on the same site, Google Translate isn't *"generative"*:
  
  > output could be considered too tightly defined by the input text
  
  These translation tools don't *generate* new text. They *translate*.  
  Sure, they use AI to infer meaning to result in better translations, but you can't tell them to *"Translate Macbeth to Swahili"*...
  
  You'll never get *more* out of the tools than you put into them.
  
  > **Comments:**
  > **username_0**: Sirbenet's answer was refering to [What is generative AI (GenAI) according to this site?](https://genai.meta.stackexchange.com/q/163), where "this site" means "Gen AI meta". I am asking about the **common use of the term in the field of AI**.  
  > **username_0**: In this question here, I am asking about the general use of the term in the field of AI, not just at SE or SO.  
  > **username_1**: You're free to ignore my answer.  
  
  ---
  
  