readme: ---
  license: mit
  language: en
  widget:
  - text: "This is a traditional Irish dance music."
  inference:
    parameters:
      top_p: 0.9
      max_length: 1024
      do_sample: True
  ---
  # Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task
  
  ## Model description
  
  This language-music model takes [BART-base](https://huggingface.co/facebook/bart-base) fine-tunes on 282,870 English text-music pairs, where all scores are represented in ABC notation. It was introduced in the paper [Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task](https://arxiv.org/abs/2211.11216) by Wu et al. and released in [this repository](https://github.com/sander-wood/text-to-music). 
  
  It is capable of generating complete and semantically consistent sheet music directly from descriptions in natural language based on text. To the best of our knowledge, this is the first model that achieves text-conditional symbolic music generation which is trained on real text-music pairs, and the music is generated entirely by the model and without any hand-crafted rules.
  
  ## Intended uses & limitations
  
  You can use this model for text-conditional music generation. All scores generated by this model can be written on one stave (for vocal solo or instrumental solo) in standard classical notation, and are in a variety of styles, e.g., blues, classical, folk, jazz, pop, and world music. We recommend using the script in [this repository](https://github.com/sander-wood/text-to-music) for inference. The generated tunes are in ABC notation, and can be converted to sheet music or audio using [this website](https://ldzhangyx.github.io/abc/), or [this software](https://sourceforge.net/projects/easyabc/).
  
  Its creativity is limited, can not perform well on tasks requiring a high degree of creativity (e.g., melody style transfer), and it is input-sensitive. For more information, please check [our paper](https://arxiv.org/abs/2211.11216).
  
  ### How to use
  
  Here is how to use this model in PyTorch:
  
  ```python
  import torch
  from samplings import top_p_sampling, temperature_sampling
  from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
  
  tokenizer = AutoTokenizer.from_pretrained('sander-wood/text-to-music')
  model = AutoModelForSeq2SeqLM.from_pretrained('sander-wood/text-to-music')
  model = model
  
  max_length = 1024
  top_p = 0.9
  temperature = 1.0
  
  text = "This is a traditional Irish dance music."
  input_ids = tokenizer(text, 
                        return_tensors='pt', 
                        truncation=True, 
                        max_length=max_length)['input_ids']
  
  decoder_start_token_id = model.config.decoder_start_token_id
  eos_token_id = model.config.eos_token_id
  
  decoder_input_ids = torch.tensor([[decoder_start_token_id]])
  
  for t_idx in range(max_length):
      outputs = model(input_ids=input_ids, 
      decoder_input_ids=decoder_input_ids)
      probs = outputs.logits[0][-1]
      probs = torch.nn.Softmax(dim=-1)(probs).detach().numpy()
      sampled_id = temperature_sampling(probs=top_p_sampling(probs, 
                                                             top_p=top_p, 
                                                             return_probs=True),
                                        temperature=temperature)
      decoder_input_ids = torch.cat((decoder_input_ids, torch.tensor([[sampled_id]])), 1)
      if sampled_id!=eos_token_id:
          continue
      else:
          tune = "X:1\n"
          tune += tokenizer.decode(decoder_input_ids[0], skip_special_tokens=True)
          print(tune)
          break
  ```
  
  ### Generation Examples
  Here are some examples generated by this model without cherry-picking.
  ```
  ######################## INPUT TEXT ########################
  
  This is a traditional Irish dance music.
  Note Length-1/8
  Meter-6/8
  Key-D
  
  ####################### OUTPUT TUNES #######################
  
  X:1
  L:1/8
  M:6/8
  K:D
   A | BEE BEE | Bdf edB | BAF FEF | DFA BAF | BEE BEE | Bdf edB | BAF DAF | FED E2 :: A |
   Bef gfe | faf edB | BAF FEF | DFA BAF | Bef gfe | faf edB | BAF DAF | FED E2 :|
  
  X:2
  L:1/8
  M:6/8
  K:D
   A |: DED F2 A | d2 f ecA | G2 B F2 A | E2 F GFE | DED F2 A | d2 f ecA | Bgf edc |1 d3 d2 A :|2
   d3 d2 a || a2 f d2 e | f2 g agf | g2 e c2 d | e2 f gfe | fed gfe | agf bag | fed cde | d3 d2 a |
   agf fed | Adf agf | gfe ecA | Ace gfe | fed gfe | agf bag | fed cde | d3 d2 ||
  
  X:3
  L:1/8
  M:6/8
  K:D
   BEE BEE | Bdf edB | BAF FEF | DFA dBA | BEE BEE | Bdf edB | BAF FEF |1 DED DFA :|2 DED D2 e |:
   faf edB | BAF DFA | BAF FEF | DFA dBA | faf edB | BAF DFA | BdB AFA |1 DED D2 e :|2 DED DFA ||
  ```
  
  ```
  ######################## INPUT TEXT ########################
  
  This is a jazz-swing lead sheet with chord and vocal.
  
  ####################### OUTPUT TUNES #######################
  
  X:1
  L:1/8
  M:4/4
  K:F
  "F" CFG |"F" A6 z G |"Fm7" A3 G"Bb7" A3 G |"F" A6 z G |"F7" A4"Eb7" G4 |"F" F6 z F |
  "Dm" A3 G"Dm/C" A3 G |"Bb" A2"Gm" B2"C7" G3 G |"F" F8- |"Dm7""G7" F6 z2 |"C" C4 C3 C |
  "C7" C2 B,2"F" C4 |"F" C4 C3 C |"Dm" D2 C2"Dm/C" D4 |"Bb" D4 D3 D |"Bb" D2 C2"