Answers.Body: <p>The concept of &quot;music similarity&quot; isn't very clear cut -- I'm not sure that I <em>personally</em> would popular songs outliers. Artists take inspiration from existing music, so musical styles, production techniques etc. from popular songs should spread more than unpopular songs. I think this problem will come down to the data that you're analyzing/training on and the way you choose to encode/generate the songs.</p>
  <p>GANs in particular have a well-documented problem of <a href="https://developers.google.com/machine-learning/gan/problems" rel="nofollow noreferrer">mode-collapse</a>, where the generator over-optimizes for a single discriminator, leading to a lack of diversity in generations. Considering the diversity of different genres, styles, instruments, etc. in music, having a model capable of handling a diverse dataset is particularly important, <em>especially</em> if popular songs do turn out to be outliers.</p>
  <p>For example, consider the MNIST dataset. The generator might notice that 1s are really easy to generate, so will generate realistic, hard-to-detect 1s. After a couple iterations, the discriminator will start classifying all 1s as fake. But now, the generator can simply move onto the next easiest digit to generate, as the discriminator is just checking for 1s. <a href="https://cedar.buffalo.edu/%7Esrihari/CSE676/22.3-GAN%20Mode%20Collapse.pdf" rel="nofollow noreferrer">In other words, while real-world data has many modes, because of the nature of the &quot;game&quot; being played, GANs will collapse to only a few.</a> See the linked websites for some possible solutions to this.</p>
  <p>You may also want to consider using a different modeling method. Models like variants of RNNs and Transformers are often used (see: <a href="https://arxiv.org/pdf/2210.13944.pdf" rel="nofollow noreferrer">Table 3</a>) as they were designed for sequential data. Given its success with image-generation, diffusion models have also been popular recently. <a href="https://openreview.net/pdf?id=H1xQVn09FX" rel="nofollow noreferrer">However, GANs still have their advantages, like being faster due to their non-autoregressive nature</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T08:51:04.147Z
Answers.LastActivityDate: 2023-07-22T08:51:04.147Z
Answers.UserDisplayName: Alexander Wan
Answers.Body: <p>I'm not sure what you plan to do, but I think that you are misunderstanding both what an outlier is and how GANs work.</p>
  <p>An observation being an outlier is a property that's relative to the expected underlying distribution of the data. When an observation emerges that is very unlikely to have been drawn from a random variable with the same distribution as the rest, that observation is labeled an outlier. Imagine you came up with a numerical ranking for the quality of songs, and you were studying the entire field of music. Here, the sample space is the set of all possible songs, and the measurable space your variable is mapping this space into is a subset of the image of your ranking function.</p>
  <p>During your study, you observe that this variable is distributed in a certain way: Most songs have a mid ranking, only a few songs have a very high ranking, and only a few songs have a very low ranking (let's assume this is the case, even though it's probably not). An observation that is extremely low, so low that you wouldn't expect to see a song with such a low ranking in a million years, could be considered an outlier. Likewise, a song with an exceptionally high ranking could also be an outlier. This could indicate that something happened that broke your ranking measurement (for example, a cat sat on the keyboard while your ranking algorithm was rating a song).</p>
  <p>However, let's now imagine that we are limiting our study only to good songs, so that our sample space is no longer that of all possible songs. Maybe even consider only the set of exceptionally good songs. Could a song with an exceptionally high ranking be an outlier here? No, having a high ranking in this subset is the norm, so unless it was so incredibly high that it again destroyed even the distribution of exceptionally good songs, it wouldn't be considered an outlier. An average song, however, which wouldn't have been an outlier in the first case, would be an outlier in this second case.</p>
  <p>So if the only thing you show to your algorithm is good songs, how are good songs going to be outliers?</p>
  <p>That being said, this shouldn't matter to you because the discriminator of a GAN isn't concerned with distinguishing the quality of a song but with distinguishing original and artificial samples. The only samples the discriminator sees are the samples the generator is trained with and the generator's creations. An outlier here could be a song that, even though it is natural, has features that make the discriminator inappropriately think it is artificial (which I doubt a lot would mean the song is 'too good').</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T11:29:13.683Z
Answers.LastActivityDate: 2023-07-23T12:16:38.123Z
Answers.LastEditDate: 2023-07-23T12:16:38.123Z
Answers.UserDisplayName: Mario Asis
Body: <p>I was thinking about making an adversarial network to generate popular music, so one AI which generates and then two others which detect whether the song is AI generated and the expected view count of the song. The AI would thus learn to generate songs which are both realistic and popular.</p>
  <p>However I came to a dilemma: Would popular cover songs be flagged as being AI-generated (not realistic)?</p>
  <p>It seems like the most popular songs (Old town road, Wrecking ball, Gangnam style, crazy frog...) are all significantly different from normal music, and so would be flagged as AI generated. This would make it very difficult to generate music which satisfies both realism and popularity, since the most popular songs would have low realism scores...</p>
  <p>Does anyone know methods to handle this sort of issue?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-22T05:24:59.583Z
Comments.Text: I think answer to your question lies in the way how ChatGPT prompt is handled to begin with. I am not too familiar with LLM but I am sure it cannot generate same lyrics each time I ask the same question. Or new lyrics at all - just answer some existing ones best it can.
Comments.UserDisplayName: harism
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-22T05:31:10.017Z
Comments.Text: You are talking about generating the lyrics or the composition?
Comments.UserDisplayName: Chinmay
Tags: <generative-adversarial-networks>
Title: Are popular songs considered outliers to AI
UserDisplayName: Alex Breeze
ThreadText: # Are popular songs considered outliers to AI
  
  (Asked by: username_0 on 2023-07-22)
  
  I was thinking about making an adversarial network to generate popular music, so one AI which generates and then two others which detect whether the song is AI generated and the expected view count of the song. The AI would thus learn to generate songs which are both realistic and popular.
  
  However I came to a dilemma: Would popular cover songs be flagged as being AI-generated (not realistic)?
  
  It seems like the most popular songs (Old town road, Wrecking ball, Gangnam style, crazy frog...) are all significantly different from normal music, and so would be flagged as AI generated. This would make it very difficult to generate music which satisfies both realism and popularity, since the most popular songs would have low realism scores...
  
  Does anyone know methods to handle this sort of issue?
  
  **Tags:** generative-adversarial-networks
  
  > **Comments:**
  > **username_1**: I think answer to your question lies in the way how ChatGPT prompt is handled to begin with. I am not too familiar with LLM but I am sure it cannot generate same lyrics each time I ask the same question. Or new lyrics at all - just answer some existing ones best it can.  
  > **username_2**: You are talking about generating the lyrics or the composition?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2023-07-22. Score: 2
  
  The concept of "music similarity" isn't very clear cut -- I'm not sure that I *personally* would popular songs outliers. Artists take inspiration from existing music, so musical styles, production techniques etc. from popular songs should spread more than unpopular songs. I think this problem will come down to the data that you're analyzing/training on and the way you choose to encode/generate the songs.
  
  GANs in particular have a well-documented problem of [mode-collapse](https://developers.google.com/machine-learning/gan/problems), where the generator over-optimizes for a single discriminator, leading to a lack of diversity in generations. Considering the diversity of different genres, styles, instruments, etc. in music, having a model capable of handling a diverse dataset is particularly important, *especially* if popular songs do turn out to be outliers.
  
  For example, consider the MNIST dataset. The generator might notice that 1s are really easy to generate, so will generate realistic, hard-to-detect 1s. After a couple iterations, the discriminator will start classifying all 1s as fake. But now, the generator can simply move onto the next easiest digit to generate, as the discriminator is just checking for 1s. [In other words, while real-world data has many modes, because of the nature of the "game" being played, GANs will collapse to only a few.](https://cedar.buffalo.edu/%7Esrihari/CSE676/22.3-GAN%20Mode%20Collapse.pdf) See the linked websites for some possible solutions to this.
  
  You may also want to consider using a different modeling method. Models like variants of RNNs and Transformers are often used (see: [Table 3](https://arxiv.org/pdf/2210.13944.pdf)) as they were designed for sequential data. Given its success with image-generation, diffusion models have also been popular recently. [However, GANs still have their advantages, like being faster due to their non-autoregressive nature](https://openreview.net/pdf?id=H1xQVn09FX).
  
  
  ---
  
  ### Answer by username_4 on 2023-07-22. Score: 1
  
  I'm not sure what you plan to do, but I think that you are misunderstanding both what an outlier is and how GANs work.
  
  An observation being an outlier is a property that's relative to the expected underlying distribution of the data. When an observation emerges that is very unlikely to have been drawn from a random variable with the same distribution as the rest, that observation is labeled an outlier. Imagine you came up with a numerical ranking for the quality of songs, and you were studying the entire field of music. Here, the sample space is the set of all possible songs, and the measurable space your variable is mapping this space into is a subset of the image of your ranking function.
  
  During your study, you observe that this variable is distributed in a certain way: Most songs have a mid ranking, only a few songs have a very high ranking, and only a few songs have a very low ranking (let's assume this is the case, even though it's probably not). An observation that is extremely low, so low that you wouldn't expect to see a song with such a low ranking in a million years, could be considered an outlier. Likewise, a song with an exceptionally high ranking could also be an outlier. This could indicate that something happened that broke your ranking measurement (for example, a cat sat on the keyboard while your ranking algorithm was rating a song).
  
  However, let's now imagine that we are limiting our study only to good songs, so that our sample space is no longer that of all possible songs. Maybe even consider only the set of exceptionally good songs. Could a song with an exceptionally high ranking be an outlier here? No, having a high ranking in this subset is the norm, so unless it was so incredibly high that it again destroyed even the distribution of exceptionally good songs, it wouldn't be considered an outlier. An average song, however, which wouldn't have been an outlier in the first case, would be an outlier in this second case.
  
  So if the only thing you show to your algorithm is good songs, how are good songs going to be outliers?
  
  That being said, this shouldn't matter to you because the discriminator of a GAN isn't concerned with distinguishing the quality of a song but with distinguishing original and artificial samples. The only samples the discriminator sees are the samples the generator is trained with and the generator's creations. An outlier here could be a song that, even though it is natural, has features that make the discriminator inappropriately think it is artificial (which I doubt a lot would mean the song is 'too good').
  
  
  ---
  
  