Answers.Body: <p>Philosophically speaking,</p>
  <p>I believe a perspective shift is necessary. When you say, 'A solution exists', the next statement should be, 'Well, what is the problem?'. While you are correct in pointing out instances where the model is unique and not so much so, their correct problem statement is not being talked about.</p>
  <p>I am afraid, that the concept of a &quot;Well-posed problem&quot; is theoretical, and while a problem might look absolutely well-posed in theory, things might work out differently in the real world.</p>
  <p>Let's talk about the theory of an LLM (will intentionally leave out the technical bits). Ideally, given an exhaustive dataset of all the world's data and a NN with practically infinite training parameters; a trained language model should be able to be attentive enough to completely understand the context before providing any answers. Such a model will work something like this-</p>
  <p>User: &quot;Fill in the blank, 'I would love to have some ____'&quot;</p>
  <p>LLM: &quot;What do you love&quot;</p>
  <p>User: &quot;Among many things, chocolate, ice-creams, dance, booze, etc.&quot;</p>
  <p>LLM: &quot;Okay, and how are you feeling today?&quot;</p>
  <p>User: &quot;Particularly hot, and alone.&quot;</p>
  <p>LLM: (to itself: maybe some sugar to lift up the mood, and some cold ice cream to help with the hot day)</p>
  <p>LLM: Let's go have some Chocolate Ice-cream.</p>
  <p>User: Wow! I would never have thought about it myself! Thanks, Overlord.</p>
  <p>Strictly theoretically speaking, the Freudian concept of pan-determinism says, given the complete history of a human being, you can pinpoint the exact decision they will make given a situation. So, in this ideal world, where the model thought this information is enough to figure out the answer would be absolutely right. As there will exist only one fill to the blank.</p>
  <p>Now, let's travel to the real world, shall we?</p>
  <p>Here, everything is limited. From low memory, and limited silicon chip supply, to limited attention spans (of LLMs of course). Because here, our LLM doesn't have (yet) the capability to reason on itself, unless explicitly advised to do so, it will answer based on defined heuristics like Greedy Search (see, now that you think of it, because someone included greedy search in the architecture, suddenly it feels like it was a well-defined problem after all? Because given the limitations of the language models and the architecture set bu the creature, there is only one solution, isn't there? Unless of course, you decide you want to increase the randomness in selection). But, back to the point, transformers or any other kind of NN are essentially numerical models which theoretically speaking have a well-defined convergence to a point but practically never converge; we always judge the trained parameters to be good enough and stop the training.</p>
  <p>Given that we can probably never really achieve the ideal doesn't mean that it doesn't exist. With infinite resources, we can build the ideal LLM that can do things that make it a Well-posed problem.</p>
  <p>And in due time we might have LLMs good enough to predict human decisions which makes you feel like they only had one solution which they duly produced, <a href="https://www.youtube.com/watch?v=NOCsdhzo6Jg&amp;ab_channel=3Blue1Brown" rel="nofollow noreferrer">but as is with illusions, they are never real.</a></p>
  <p>UPDATE: One more thing that I just recalled is that while it is a well-posed problem, engineers and developers go to great lengths to disturb its &quot;pose&quot; just because again, practicality supersedes idealism. What I mean by that is illustrated best using an example, that of the Bi-directional Encoder Representations from Transformers or the BERT architecture.</p>
  <p>The trainers went to great lengths to intentionally introduce errors in training to avoid overfitting and achieve higher efficiency, which on the downside had the side-effect of not predicting a specific token with high confidence. What I am referring to is their use of masking [Mask] only 80% of the time during training, and intentionally using [word] instead of a mask, or a [wrong word] instead of the mask equally over the remaining 20% of the times.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-02T06:57:33.420Z
Answers.LastActivityDate: 2023-07-02T17:39:41.163Z
Answers.LastEditDate: 2023-07-02T17:39:41.163Z
Answers.UserDisplayName: Chinmay
Body: <p>Hadamard defines (<a href="https://en.wikipedia.org/wiki/Well-posed_problem" rel="nofollow noreferrer">Well-posed problem (Wikipedia)</a>) a well-posed problem as one for which:</p>
  <ol>
  <li>a solution exists,</li>
  <li>the solution is unique,</li>
  <li>the solution depends continuously on the data (e.g. it is stable)</li>
  </ol>
  <p>Now for an autoregressive language model, the pretaining objective consists of predicting the next token given all previous tokens. But as I see it, while the objective helps the model pick nuances of the language and learn &quot;language representation&quot;, there is no &quot;correct&quot; answer.
  Given the example, say, &quot;I would love to have some ____&quot;, &quot;chocolate&quot; is just as &quot;correct&quot; as &quot;coffee&quot;.
  Thus, the solution is not &quot;unique&quot; as required by the definition.
  But again, we could think of the problem as taking as input all the previous tokens and returning a probability distribution over the whole vocabulary. In this way of looking at the problem, the solution, i.e., the probability distribution, is unique.
  So, is the problem of language modelling well-posed?
  Is either of the two approaches the right reason? If not, what is the correct reason?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-09T20:05:37.190Z
Comments.Text: I'd rephrase the problem as a whole to 'can multi-modal problems be well-posed' and you have yourself a super interesting philosophical (and much more broadly applicable) question. One category mutli-modal problem is problems for which there are multiple correct answers to a single input. Coloring in a grayscale image for example. (snooker balls in an image can be of many different colors).
Comments.UserDisplayName: Robin van Hoorn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-04T08:01:24.163Z
Comments.Text: One of the most interesting theoretical questions ever asked here and still gots no attention. This shows how generate this site and the world are. Unfortunately, the answer below is not really what I am expecting and not good. This question is not philosophical. This is only philosophical in that it makes us think that maybe we're doing something wrong in AI and that we even need to understand what we're doing. That's why AI doesn't really advance and people just brute-force with the hope and dreams of getting rich or famous because of a breakthrough.
Comments.UserDisplayName: nbro
Tags: <natural-language-processing><math><sequence-modeling><language-model>
Title: Is the problem of Language Modelling a Well-Posed Learning Problem?
UserDisplayName: aren't eistert
ThreadText: # Is the problem of Language Modelling a Well-Posed Learning Problem?
  
  (Asked by: username_0 on 2023-05-09)
  
  Hadamard defines ([Well-posed problem (Wikipedia)](https://en.wikipedia.org/wiki/Well-posed_problem)) a well-posed problem as one for which:
  
  1. a solution exists,
  2. the solution is unique,
  3. the solution depends continuously on the data (e.g. it is stable)
  
  Now for an autoregressive language model, the pretaining objective consists of predicting the next token given all previous tokens. But as I see it, while the objective helps the model pick nuances of the language and learn "language representation", there is no "correct" answer.
  Given the example, say, "I would love to have some \_\_\_\_", "chocolate" is just as "correct" as "coffee".
  Thus, the solution is not "unique" as required by the definition.
  But again, we could think of the problem as taking as input all the previous tokens and returning a probability distribution over the whole vocabulary. In this way of looking at the problem, the solution, i.e., the probability distribution, is unique.
  So, is the problem of language modelling well-posed?
  Is either of the two approaches the right reason? If not, what is the correct reason?
  
  **Tags:** natural-language-processing, math, sequence-modeling, language-model
  
  > **Comments:**
  > **username_1**: I'd rephrase the problem as a whole to 'can multi-modal problems be well-posed' and you have yourself a super interesting philosophical (and much more broadly applicable) question. One category mutli-modal problem is problems for which there are multiple correct answers to a single input. Coloring in a grayscale image for example. (snooker balls in an image can be of many different colors).  
  > **username_2**: One of the most interesting theoretical questions ever asked here and still gots no attention. This shows how generate this site and the world are. Unfortunately, the answer below is not really what I am expecting and not good. This question is not philosophical. This is only philosophical in that it makes us think that maybe we're doing something wrong in AI and that we even need to understand what we're doing. That's why AI doesn't really advance and people just brute-force with the hope and dreams of getting rich or famous because of a breakthrough.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2023-07-02. Score: 0
  
  Philosophically speaking,
  
  I believe a perspective shift is necessary. When you say, 'A solution exists', the next statement should be, 'Well, what is the problem?'. While you are correct in pointing out instances where the model is unique and not so much so, their correct problem statement is not being talked about.
  
  I am afraid, that the concept of a "Well-posed problem" is theoretical, and while a problem might look absolutely well-posed in theory, things might work out differently in the real world.
  
  Let's talk about the theory of an LLM (will intentionally leave out the technical bits). Ideally, given an exhaustive dataset of all the world's data and a NN with practically infinite training parameters; a trained language model should be able to be attentive enough to completely understand the context before providing any answers. Such a model will work something like this-
  
  User: "Fill in the blank, 'I would love to have some \_\_\_\_'"
  
  LLM: "What do you love"
  
  User: "Among many things, chocolate, ice-creams, dance, booze, etc."
  
  LLM: "Okay, and how are you feeling today?"
  
  User: "Particularly hot, and alone."
  
  LLM: (to itself: maybe some sugar to lift up the mood, and some cold ice cream to help with the hot day)
  
  LLM: Let's go have some Chocolate Ice-cream.
  
  User: Wow! I would never have thought about it myself! Thanks, Overlord.
  
  Strictly theoretically speaking, the Freudian concept of pan-determinism says, given the complete history of a human being, you can pinpoint the exact decision they will make given a situation. So, in this ideal world, where the model thought this information is enough to figure out the answer would be absolutely right. As there will exist only one fill to the blank.
  
  Now, let's travel to the real world, shall we?
  
  Here, everything is limited. From low memory, and limited silicon chip supply, to limited attention spans (of LLMs of course). Because here, our LLM doesn't have (yet) the capability to reason on itself, unless explicitly advised to do so, it will answer based on defined heuristics like Greedy Search (see, now that you think of it, because someone included greedy search in the architecture, suddenly it feels like it was a well-defined problem after all? Because given the limitations of the language models and the architecture set bu the creature, there is only one solution, isn't there? Unless of course, you decide you want to increase the randomness in selection). But, back to the point, transformers or any other kind of NN are essentially numerical models which theoretically speaking have a well-defined convergence to a point but practically never converge; we always judge the trained parameters to be good enough and stop the training.
  
  Given that we can probably never really achieve the ideal doesn't mean that it doesn't exist. With infinite resources, we can build the ideal LLM that can do things that make it a Well-posed problem.
  
  And in due time we might have LLMs good enough to predict human decisions which makes you feel like they only had one solution which they duly produced, [but as is with illusions, they are never real.](https://www.youtube.com/watch?v=NOCsdhzo6Jg&ab_channel=3Blue1Brown)
  
  UPDATE: One more thing that I just recalled is that while it is a well-posed problem, engineers and developers go to great lengths to disturb its "pose" just because again, practicality supersedes idealism. What I mean by that is illustrated best using an example, that of the Bi-directional Encoder Representations from Transformers or the BERT architecture.
  
  The trainers went to great lengths to intentionally introduce errors in training to avoid overfitting and achieve higher efficiency, which on the downside had the side-effect of not predicting a specific token with high confidence. What I am referring to is their use of masking [Mask] only 80% of the time during training, and intentionally using [word] instead of a mask, or a [wrong word] instead of the mask equally over the remaining 20% of the times.
  
  
  ---
  
  