Answers.Body: <p>Generative AI isn't a great starting point if you haven't worked with AI before.  I would recommend starting with something simpler and working your way up to that.  There is good stuff online though for learning it, and if you want to dive into the deep end that is where I'd start.  Build a few generative AI models to create text or images and then start thinking about how to apply them to your musical application.  Here's a good place to get started that isn't hidden behind a pay wall:</p>
  <p><a href="https://lore.com/generative-ai" rel="nofollow noreferrer">https://lore.com/generative-ai</a></p>
  <p>If you want to mess around with AI applications in general, but still want something useful for inspiration while your work your way up to your generative AI masterpiece, a fun project might just be to build something that will play things you or other people have already composed that could provide inspiration related to what you are working on now.  You know how Spotify gives you song suggestions based on what you have listened to before?  Imagine something like that, but for composers.  You play a little something you have been working on, and then the program finds stuff in its catalog with a similar feel to that an plays them back.  This could provide inspiration in the moment by showing you a bunch of stuff sort of like what you just played to give you ideas where to spin off next.  Even better though, this will give you data and tools to use later to create your generative AI, as well as giving you a deeper understanding of what you are doing when you get to that point.</p>
  <p>Spotify's song recommendations are driven by a nearest neighbors algorithm called Annoy.  Another I particularly like is Facebook's Faiss.  Use some signal processing methods to extract feature vectors from various compositions, index those feature vectors using some nearest neighbors library like those mentioned above, and then you will have something that will spit out songs sort of like the one it just heard when you play it something new.  It's not creating a new composition, but it's a good start toward learning what AI models are doing when they try to generate something &quot;like&quot; something else.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-08T02:42:07.423Z
Answers.Text: Thanks for the response, but this does not help me (at least not yet!). The key thing to make my concept work is the ability for the AI to take file input in MusicXML format. Let me dig in a bit deeper. There are various software packages for composers these days - Finale, Sibelius, Dorico, etc. These packages are not compatible with one another as they each have their own internal storage mechanism, but you can exchange compositions using MusicXML which is the lingua franca for composition software.
Answers.UserDisplayName: E Heilner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-08T02:42:07.423Z
Answers.Text: So - as a composer you enter the music using standard musical notation. E.g., here is Mary Had a Little Lamb: https://i.sstatic.net/z7sOp.png And then you export the composition using MusicXML. If you're curious, here's the full [MusicXML file](https://www.dropbox.com/scl/fi/666ugitk24mwpvw7nop7w/Mary-Had-a-Little-Lamb-Full-score-01-Flow-1.musicxml?rlkey=mrt7nsgw3xrepmen99wvk8sng&dl=0).
Answers.UserDisplayName: E Heilner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-08T14:42:44.523Z
Answers.Text: I mostly work with images and am not super well versed on NLP but this feels like an NLP-ish problem.  You have a sort of "language" you want to extract information from to generate predictive "text".  It's kind of a complicated one though, with multiple properties that need to be encoded for each token.  LSTMs are definitely something you should look into, as well as NLP libraries like Gensim, CoreNLP, or NLTK.  I'm not familiar enough with this area to really give a step by step breakdown but I think you can basically treat each measure like a word or sentence and play around from there.
Answers.UserDisplayName: Jeremiah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-08T14:55:09.217Z
Answers.Text: Getting it into XML format is the easy part.  That's just normal programming.  Take the values your model returns and format them as XML.  You aren't really trying to train a thing to generate an XML document.  When you actually get to the generative AI phase what you really want to generate is a time series of notes, with properties like pitch and duration, and then you can format them however you'd like.
Answers.UserDisplayName: Jeremiah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-11T19:52:45.423Z
Answers.Text: Jeremiah - Thanks again for taking the time to respond. So far I am not seeing anything that meets my needs. Just to give a few more details for anyone else reading, music is far more than a time series of notes - there are literally hundreds of properties that have to be accounted for: time signature, key signature, staff, accents, embellishments, playing techniques, slurs, phrasing, formatting, etc, etc. For this to be practical both the input & output have to be in MusicXML format to work with any of the standard composition software packages. [MusicXML](https://www.musicxml.com/)
Answers.UserDisplayName: E Heilner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-12T15:25:05.337Z
Answers.Text: I'm not sure you understand the difference between data and format.  My point is that getting data out of and into XML are pre and post processing tasks, and that feeding raw XML into a machine learning model of any kind will not have the results you are looking for.  Being this hung up on data format kind of makes the point I was making above about not really being ready for a project like this.  You should make some toys and work your way up to something that generates compositions.
Answers.UserDisplayName: Jeremiah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-12T15:29:13.813Z
Answers.Text: If you train an AI to generate MusicXML documents, it will generate MusicXML documents, but unless it's optimizing on meaningfully extracted data, those documents are unlikely to be anything that sounds good.  Neither my answer nor my comment indicates that there aren't numerous and complex facets to a musical score.  My point is that there *are*, and that you need to encode all that complexity into tokens and vectors if you want a neural net to be able to optimize on them.  This isn't an XML generation problem.  It's a time series generation problem, a very complicated one.
Answers.UserDisplayName: Jeremiah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-12T16:29:05.330Z
Answers.Text: The model isn't generating an XML document, but a song.  A model trained on raw XML will learn to generate text that looks like XML, not music that sounds like music.  The data containing the patterns you want to optimize on are what matters.  XML is just the format that data is in.  In this case, you have measures, and those measures contain some attributes as well as a bunch of notes with their own attributes.  An AI is going to need to tokenize and compare all those measures and notes and their attributes. NLP libraries have a lot of tools for doing stuff like this.
Answers.UserDisplayName: Jeremiah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-12T16:51:01.243Z
Answers.Text: All those different software packages that use MusicXML aren't working with the data in that format.  They are extracting the data, storing it as various primitive data types, and working with that.  The whole point of machine readable markup languages like XML is to allow the computer to do this.  When you are using software, those formats really matter.  When you are making software, they matter less.  Converting that dropbox file from XML to YAML to JSON back to XML is child's play.  A programmer just sees a structure with a bunch of data in it.
Answers.UserDisplayName: Jeremiah
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-13T16:08:57.427Z
Answers.Text: Jeremiah - Thanks yet again for taking the time to help this newbie.  I get what you're saying - but I just don't have the expertise to take this on.  I'll have to be patient - hopefully someone with more time/skill will realize that there's a market for this.
Answers.UserDisplayName: E Heilner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-06T17:34:24.413Z
Answers.LastActivityDate: 2023-12-06T17:34:24.413Z
Answers.UserDisplayName: Jeremiah
Body: <p>First time submitter here.  I'm a composer of (for want of a better term) contemporary classical music - and I would like to use AI to improve my productivity.  The specific issue I'm dealing with is that I have plenty of raw ideas (inspirations) but converting an inspiration into a finished composition is (to a certain extent) more mechanical and tedious.</p>
  <p>Here's what I'm thinking: I would first train the AI in my musical style using my existing compositions (including version histories).  The training input would be somewhere between 2-5K files in <a href="https://www.musicxml.com/" rel="nofollow noreferrer">MusicXML</a> format - my composition software (Dorico) does this.  The AI would learn my style of composing.  I would then feed a new idea or a partly finished composition (again in MusicXML format) to the AI and ask where it should go next based my style.  The output should be in MusicXML format.  I would take this and feed it back into my composition software and then go from there - I could pick &amp; choose what I like - and then feed that back into the AI for further ideas.  I.e. - I'm envisioning some sort of iterative process.</p>
  <p>Any thoughts / suggestions?  Thanks!</p>
  
Tags: <machine-learning><training-datasets>
Title: XML Input to assist in composing muusic
UserDisplayName: E Heilner
ThreadText: # XML Input to assist in composing muusic
  
  (Asked by: username_0 on 2023-12-05)
  
  First time submitter here. I'm a composer of (for want of a better term) contemporary classical music - and I would like to use AI to improve my productivity. The specific issue I'm dealing with is that I have plenty of raw ideas (inspirations) but converting an inspiration into a finished composition is (to a certain extent) more mechanical and tedious.
  
  Here's what I'm thinking: I would first train the AI in my musical style using my existing compositions (including version histories). The training input would be somewhere between 2-5K files in [MusicXML](https://www.musicxml.com/) format - my composition software (Dorico) does this. The AI would learn my style of composing. I would then feed a new idea or a partly finished composition (again in MusicXML format) to the AI and ask where it should go next based my style. The output should be in MusicXML format. I would take this and feed it back into my composition software and then go from there - I could pick & choose what I like - and then feed that back into the AI for further ideas. I.e. - I'm envisioning some sort of iterative process.
  
  Any thoughts / suggestions? Thanks!
  
  **Tags:** machine-learning, training-datasets
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-12-06. Score: 2
  
  Generative AI isn't a great starting point if you haven't worked with AI before. I would recommend starting with something simpler and working your way up to that. There is good stuff online though for learning it, and if you want to dive into the deep end that is where I'd start. Build a few generative AI models to create text or images and then start thinking about how to apply them to your musical application. Here's a good place to get started that isn't hidden behind a pay wall:
  
  <https://lore.com/generative-ai>
  
  If you want to mess around with AI applications in general, but still want something useful for inspiration while your work your way up to your generative AI masterpiece, a fun project might just be to build something that will play things you or other people have already composed that could provide inspiration related to what you are working on now. You know how Spotify gives you song suggestions based on what you have listened to before? Imagine something like that, but for composers. You play a little something you have been working on, and then the program finds stuff in its catalog with a similar feel to that an plays them back. This could provide inspiration in the moment by showing you a bunch of stuff sort of like what you just played to give you ideas where to spin off next. Even better though, this will give you data and tools to use later to create your generative AI, as well as giving you a deeper understanding of what you are doing when you get to that point.
  
  Spotify's song recommendations are driven by a nearest neighbors algorithm called Annoy. Another I particularly like is Facebook's Faiss. Use some signal processing methods to extract feature vectors from various compositions, index those feature vectors using some nearest neighbors library like those mentioned above, and then you will have something that will spit out songs sort of like the one it just heard when you play it something new. It's not creating a new composition, but it's a good start toward learning what AI models are doing when they try to generate something "like" something else.
  
  > **Comments:**
  > **username_0**: Thanks for the response, but this does not help me (at least not yet!). The key thing to make my concept work is the ability for the AI to take file input in MusicXML format. Let me dig in a bit deeper. There are various software packages for composers these days - Finale, Sibelius, Dorico, etc. These packages are not compatible with one another as they each have their own internal storage mechanism, but you can exchange compositions using MusicXML which is the lingua franca for composition software.  
  > **username_0**: So - as a composer you enter the music using standard musical notation. E.g., here is Mary Had a Little Lamb: https://i.sstatic.net/z7sOp.png And then you export the composition using MusicXML. If you're curious, here's the full [MusicXML file](https://www.dropbox.com/scl/fi/666ugitk24mwpvw7nop7w/Mary-Had-a-Little-Lamb-Full-score-01-Flow-1.musicxml?rlkey=mrt7nsgw3xrepmen99wvk8sng&dl=0).  
  > **username_1**: I mostly work with images and am not super well versed on NLP but this feels like an NLP-ish problem.  You have a sort of "language" you want to extract information from to generate predictive "text".  It's kind of a complicated one though, with multiple properties that need to be encoded for each token.  LSTMs are definitely something you should look into, as well as NLP libraries like Gensim, CoreNLP, or NLTK.  I'm not familiar enough with this area to really give a step by step breakdown but I think you can basically treat each measure like a word or sentence and play around from there.  
  > **username_1**: Getting it into XML format is the easy part.  That's just normal programming.  Take the values your model returns and format them as XML.  You aren't really trying to train a thing to generate an XML document.  When you actually get to the generative AI phase what you really want to generate is a time series of notes, with properties like pitch and duration, and then you can format them however you'd like.  
  > **username_0**: Jeremiah - Thanks again for taking the time to respond. So far I am not seeing anything that meets my needs. Just to give a few more details for anyone else reading, music is far more than a time series of notes - there are literally hundreds of properties that have to be accounted for: time signature, key signature, staff, accents, embellishments, playing techniques, slurs, phrasing, formatting, etc, etc. For this to be practical both the input & output have to be in MusicXML format to work with any of the standard composition software packages. [MusicXML](https://www.musicxml.com/)  
  > **username_1**: I'm not sure you understand the difference between data and format.  My point is that getting data out of and into XML are pre and post processing tasks, and that feeding raw XML into a machine learning model of any kind will not have the results you are looking for.  Being this hung up on data format kind of makes the point I was making above about not really being ready for a project like this.  You should make some toys and work your way up to something that generates compositions.  
  > **username_1**: If you train an AI to generate MusicXML documents, it will generate MusicXML documents, but unless it's optimizing on meaningfully extracted data, those documents are unlikely to be anything that sounds good.  Neither my answer nor my comment indicates that there aren't numerous and complex facets to a musical score.  My point is that there *are*, and that you need to encode all that complexity into tokens and vectors if you want a neural net to be able to optimize on them.  This isn't an XML generation problem.  It's a time series generation problem, a very complicated one.  
  > **username_1**: The model isn't generating an XML document, but a song.  A model trained on raw XML will learn to generate text that looks like XML, not music that sounds like music.  The data containing the patterns you want to optimize on are what matters.  XML is just the format that data is in.  In this case, you have measures, and those measures contain some attributes as well as a bunch of notes with their own attributes.  An AI is going to need to tokenize and compare all those measures and notes and their attributes. NLP libraries have a lot of tools for doing stuff like this.  
  > **username_1**: All those different software packages that use MusicXML aren't working with the data in that format.  They are extracting the data, storing it as various primitive data types, and working with that.  The whole point of machine readable markup languages like XML is to allow the computer to do this.  When you are using software, those formats really matter.  When you are making software, they matter less.  Converting that dropbox file from XML to YAML to JSON back to XML is child's play.  A programmer just sees a structure with a bunch of data in it.  
  > **username_0**: Jeremiah - Thanks yet again for taking the time to help this newbie.  I get what you're saying - but I just don't have the expertise to take this on.  I'll have to be patient - hopefully someone with more time/skill will realize that there's a market for this.  
  
  ---
  
  