Answers.Body: <p>You cannot really invert the loss, because that's undefined most likely.</p>
  <p>Take linear regression with OLS, then we know that the loss function is quadratic wrt the parameters (assuming to have 1 covariate only):</p>
  <p><a href="https://i.sstatic.net/9ilqt.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9ilqt.png" alt="enter image description here" /></a></p>
  <p>Now, if you try to maximize that function, you would quickly see that they don't have any maximum, so you would just get ginormous predictions, and no pattern would be caught/learnt</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T16:15:21.620Z
Answers.Text: For this type of model, more variation around the regression line could be achieved amplifying the error variance
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T20:54:00.730Z
Answers.Text: @JaumeOliverLafont what do you mean? "_we are training to maximize error instead of minimizing it._" it's pretty well mathematically defined, and no matter what you do, you cannot get a better solution worsening the error (note that _"Allowing for some transitions other than those observed"_ does not fall in the category of this definition the OP is asking)
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T21:03:37.030Z
Answers.Text: I mean the error e in y=wx + b + e. You estimate w and b as usual -minimizing- but then generate y for given x with larger noise, so it has larger deviation around the expected value.
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T21:04:13.160Z
Answers.Text: The transitions make sense in the context of HMM then OP mentions.
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T21:07:03.787Z
Answers.Text: @JaumeOliverLafont I might agree with you, that might seem a exploration-exploitation strategy, however HMM are trained to maximize $p(y_i|y_{j<i}, x)$, and to do so we assume that $x,y$ and the conditional are some fixed distribution... if you increase the noise, you might end up in a low density region of your space, that your model has never seen, so doesn't know how to behave
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T15:33:56.393Z
Answers.LastActivityDate: 2023-11-20T15:33:56.393Z
Answers.UserDisplayName: Alberto
Answers.Body: <p>In HMM, a simple mechanism to reduce overfitting and therefore generating variety in the system output is tweaking the transition matrix A and/or the symbol emission probability B. Allowing for some transitions other than those observed in the training data or other symbols in states where they were not actually seen before can introduce a richer behavior.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-20T16:00:31.647Z
Answers.LastActivityDate: 2023-11-20T16:00:31.647Z
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.Body: <p>Going off of @Alberto's answer:</p>
  <p>I agree that you would just get ginormous predictions, but I am not convinced no pattern would be learnt: in your example, we would learn to avoid predictions near zero: for a more complex error landscape, this might be more interesting. For example, we would not just be getting ginormous predictions, but the most ginormous predictions possible: we would not ascend the landscape randomly, we would ascend via the gradient, at the steepest possible path. With many variables, the steepest possible path may be interesting or unique.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-27T16:53:18.170Z
Answers.LastActivityDate: 2023-11-27T16:53:18.170Z
Answers.UserDisplayName: BigMistake
Body: <p>Imagine we have some sort of &quot;next token predictor,&quot; either with transformer architecture, LSTM, or just a HMM (though the terminology I use here will be less aligned to HMMs, I believe the question is generalizable to all generative NLP).</p>
  <p>We reverse the cost function. That is, we are training to maximize error instead of minimizing it. In the case where error is neither maximized nor minimized, the behavior will be fairly boring. However, a model which is maximizing error may still need to learn patterns of syntax and which words usually follow one another in order to avoid them. I would expect that in some abstract way, it may behave creatively, because it is trying to produce output which is <em>not</em> in the training data, and is furthest away from it. In fact, it ideally should understand the user's query in order to avoid using words that follow it.</p>
  <p>This makes me think the output may be non-boring, although probably not practically useful.</p>
  
Tags: <natural-language-processing><training><datasets><chatgpt><natural-language-generation>
Title: Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?
UserDisplayName: BigMistake
ThreadText: # Would maximizing (instead of minimizing) error of an LLM/HMM lead to complex behavior?
  
  (Asked by: username_0 on 2023-11-20)
  
  Imagine we have some sort of "next token predictor," either with transformer architecture, LSTM, or just a HMM (though the terminology I use here will be less aligned to HMMs, I believe the question is generalizable to all generative NLP).
  
  We reverse the cost function. That is, we are training to maximize error instead of minimizing it. In the case where error is neither maximized nor minimized, the behavior will be fairly boring. However, a model which is maximizing error may still need to learn patterns of syntax and which words usually follow one another in order to avoid them. I would expect that in some abstract way, it may behave creatively, because it is trying to produce output which is *not* in the training data, and is furthest away from it. In fact, it ideally should understand the user's query in order to avoid using words that follow it.
  
  This makes me think the output may be non-boring, although probably not practically useful.
  
  **Tags:** natural-language-processing, training, datasets, chatgpt, natural-language-generation
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2023-11-20. Score: 2
  
  You cannot really invert the loss, because that's undefined most likely.
  
  Take linear regression with OLS, then we know that the loss function is quadratic wrt the parameters (assuming to have 1 covariate only):
  
  [![enter image description here](https://i.sstatic.net/9ilqt.png)](https://i.sstatic.net/9ilqt.png)
  
  Now, if you try to maximize that function, you would quickly see that they don't have any maximum, so you would just get ginormous predictions, and no pattern would be caught/learnt
  
  > **Comments:**
  > **username_2**: For this type of model, more variation around the regression line could be achieved amplifying the error variance  
  > **username_1**: @username_2 what do you mean? "_we are training to maximize error instead of minimizing it._" it's pretty well mathematically defined, and no matter what you do, you cannot get a better solution worsening the error (note that _"Allowing for some transitions other than those observed"_ does not fall in the category of this definition the OP is asking)  
  > **username_2**: I mean the error e in y=wx + b + e. You estimate w and b as usual -minimizing- but then generate y for given x with larger noise, so it has larger deviation around the expected value.  
  > **username_2**: The transitions make sense in the context of HMM then OP mentions.  
  > **username_1**: @username_2 I might agree with you, that might seem a exploration-exploitation strategy, however HMM are trained to maximize $p(y_i|y_{j<i}, x)$, and to do so we assume that $x,y$ and the conditional are some fixed distribution... if you increase the noise, you might end up in a low density region of your space, that your model has never seen, so doesn't know how to behave  
  
  ---
  
  ### Answer by username_2 on 2023-11-20. Score: 1
  
  In HMM, a simple mechanism to reduce overfitting and therefore generating variety in the system output is tweaking the transition matrix A and/or the symbol emission probability B. Allowing for some transitions other than those observed in the training data or other symbols in states where they were not actually seen before can introduce a richer behavior.
  
  
  ---
  
  ### Answer by username_0 on 2023-11-27. Score: 0
  
  Going off of @username_1's answer:
  
  I agree that you would just get ginormous predictions, but I am not convinced no pattern would be learnt: in your example, we would learn to avoid predictions near zero: for a more complex error landscape, this might be more interesting. For example, we would not just be getting ginormous predictions, but the most ginormous predictions possible: we would not ascend the landscape randomly, we would ascend via the gradient, at the steepest possible path. With many variables, the steepest possible path may be interesting or unique.
  
  
  ---
  
  