Minor Updates for main project consistency

update colab

requirements update

reverse HF gradio version

update gradio req

README.md

@@ -4,7 +4,7 @@ emoji: 🎼
 colorFrom: white
 colorTo: red
 sdk: gradio
-sdk_version: 3.34.0
+sdk_version: 3.33.1
 app_file: app.py
 pinned: false
 license: creativeml-openrail-m
@@ -60,26 +60,13 @@ pip install -e .  # or if you cloned the repo locally
 
 ## Usage
 We offer a number of way to interact with MusicGen:
-1. You can play with MusicGen by running the jupyter notebook at [`demo.ipynb`](./demo.ipynb) locally, or use the provided [colab notebook](https://colab.research.google.com/drive/1fxGqfg96RBUvGxZ1XXN07s3DthrKUl4-?usp=sharing).
-2. You can use the gradio demo locally by running `python app.py`.
-3. A demo is also available on the [`facebook/MusicGen`  HuggingFace Space](https://huggingface.co/spaces/facebook/MusicGen) (huge thanks to all the HF team for their support).
-4. Finally, you can run the [Gradio demo with a Colab GPU](https://colab.research.google.com/drive/1-Xe9NCdIs2sCUbiSmwHXozK6AAhMm7_i?usp=sharing),
-as adapted from [@camenduru Colab](https://github.com/camenduru/MusicGen-colab).
-
-### More info about Top-k, Top-p, Temperature and Classifier Free Guidance from ChatGPT
-
-
-Top-k: Top-k is a parameter used in text generation models, including music generation models. It determines the number of most likely next tokens to consider at each step of the generation process. The model ranks all possible tokens based on their predicted probabilities, and then selects the top-k tokens from the ranked list. The model then samples from this reduced set of tokens to determine the next token in the generated sequence. A smaller value of k results in a more focused and deterministic output, while a larger value of k allows for more diversity in the generated music.
-
-Top-p (or nucleus sampling): Top-p, also known as nucleus sampling or probabilistic sampling, is another method used for token selection during text generation. Instead of specifying a fixed number like top-k, top-p considers the cumulative probability distribution of the ranked tokens. It selects the smallest possible set of tokens whose cumulative probability exceeds a certain threshold (usually denoted as p). The model then samples from this set to choose the next token. This approach ensures that the generated output maintains a balance between diversity and coherence, as it allows for a varying number of tokens to be considered based on their probabilities.
-
-Temperature: Temperature is a parameter that controls the randomness of the generated output. It is applied during the sampling process, where a higher temperature value results in more random and diverse outputs, while a lower temperature value leads to more deterministic and focused outputs. In the context of music generation, a higher temperature can introduce more variability and creativity into the generated music, but it may also lead to less coherent or structured compositions. On the other hand, a lower temperature can produce more repetitive and predictable music.
-
-Classifier-Free Guidance: Classifier-Free Guidance refers to a technique used in some music generation models where a separate classifier network is trained to provide guidance or control over the generated music. This classifier is trained on labeled data to recognize specific musical characteristics or styles. During the generation process, the output of the generator model is evaluated by the classifier, and the generator is encouraged to produce music that aligns with the desired characteristics or style. This approach allows for more fine-grained control over the generated music, enabling users to specify certain attributes they want the model to capture.
-
-These parameters, such as top-k, top-p, temperature, and classifier-free guidance, provide different ways to influence the output of a music generation model and strike a balance between creativity, diversity, coherence, and control. The specific values for these parameters can be tuned based on the desired outcome and user preferences.
-
-
+1. A demo is also available on the [`facebook/MusicGen`  HuggingFace Space](https://huggingface.co/spaces/facebook/MusicGen) (huge thanks to all the HF team for their support).
+2. You can run the Gradio demo in Colab: [colab notebook](https://colab.research.google.com/drive/1-Xe9NCdIs2sCUbiSmwHXozK6AAhMm7_i?usp=sharing).
+3. You can use the gradio demo locally by running `python app.py`.
+4. You can play with MusicGen by running the jupyter notebook at [`demo.ipynb`](./demo.ipynb) locally (if you have a GPU).
+5. Checkout [@camenduru Colab page](https://github.com/camenduru/MusicGen-colab) which is regularly
+  updated with contributions from @camenduru and the community.
+6. Finally, MusicGen is available in 🤗 Transformers from v4.31.0 onwards, see section [🤗 Transformers Usage](#-transformers-usage) below.
 
 ## API
 
@@ -120,6 +107,56 @@ for idx, one_wav in enumerate(wav):
     # Will save under {idx}.wav, with loudness normalization at -14 db LUFS.
     audio_write(f'{idx}', one_wav.cpu(), model.sample_rate, strategy="loudness", loudness_compressor=True)
 ```
+## 🤗 Transformers Usage
+
+MusicGen is available in the 🤗 Transformers library from version 4.31.0 onwards, requiring minimal dependencies 
+and additional packages. Steps to get started:
+
+1. First install the 🤗 [Transformers library](https://github.com/huggingface/transformers) from main:
+
+```
+pip install git+https://github.com/huggingface/transformers.git
+```
+
+2. Run the following Python code to generate text-conditional audio samples:
+
+```py
+from transformers import AutoProcessor, MusicgenForConditionalGeneration
+
+
+processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
+model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
+
+inputs = processor(
+    text=["80s pop track with bassy drums and synth", "90s rock song with loud guitars and heavy drums"],
+    padding=True,
+    return_tensors="pt",
+)
+
+audio_values = model.generate(**inputs, max_new_tokens=256)
+```
+
+3. Listen to the audio samples either in an ipynb notebook:
+
+```py
+from IPython.display import Audio
+
+sampling_rate = model.config.audio_encoder.sampling_rate
+Audio(audio_values[0].numpy(), rate=sampling_rate)
+```
+
+Or save them as a `.wav` file using a third-party library, e.g. `scipy`:
+
+```py
+import scipy
+
+sampling_rate = model.config.audio_encoder.sampling_rate
+scipy.io.wavfile.write("musicgen_out.wav", rate=sampling_rate, data=audio_values[0, 0].numpy())
+```
+
+For more details on using the MusicGen model for inference using the 🤗 Transformers library, refer to the 
+[MusicGen docs](https://huggingface.co/docs/transformers/main/en/model_doc/musicgen) or the hands-on 
+[Google Colab](https://colab.research.google.com/github/sanchit-gandhi/notebooks/blob/main/MusicGen.ipynb).
 
 
 ## Model Card
@@ -137,6 +174,9 @@ Yes. We will soon release the training code for MusicGen and EnCodec.
 
 @FurkanGozukara made a complete tutorial for [Audiocraft/MusicGen on Windows](https://youtu.be/v-YpvPkhdO4)
 
+#### I need help for running the demo on Colab
+
+Check [@camenduru tutorial on Youtube](https://www.youtube.com/watch?v=EGfxuTy9Eeo).
 
 ## Citation
 ```
@@ -151,3 +191,5 @@ Yes. We will soon release the training code for MusicGen and EnCodec.
 ## License
 * The code in this repository is released under the MIT license as found in the [LICENSE file](LICENSE).
 * The weights in this repository are released under the CC-BY-NC 4.0 license as found in the [LICENSE_weights file](LICENSE_weights).
+[arxiv]: https://arxiv.org/abs/2306.05284
+[musicgen_samples]: https://ai.honu.io/papers/musicgen/

app.py

@@ -11,11 +11,13 @@ import argparse
 import torch
 import gradio as gr
 import os
+from pathlib import Path
 import time
+import typing as tp
 import warnings
 from audiocraft.models import MusicGen
 from audiocraft.data.audio import audio_write
-from audiocraft.data.audio_utils import apply_fade, apply_tafade
+from audiocraft.data.audio_utils import apply_fade, apply_tafade, apply_splice_effect
 from audiocraft.utils.extend import generate_music_segments, add_settings_to_image, INTERRUPTING
 import numpy as np
 import random
@@ -38,6 +40,28 @@ def interrupt():
     global INTERRUPTING
     INTERRUPTING = True
 
+class FileCleaner:
+    def __init__(self, file_lifetime: float = 3600):
+        self.file_lifetime = file_lifetime
+        self.files = []
+
+    def add(self, path: tp.Union[str, Path]):
+        self._cleanup()
+        self.files.append((time.time(), Path(path)))
+
+    def _cleanup(self):
+        now = time.time()
+        for time_added, path in list(self.files):
+            if now - time_added > self.file_lifetime:
+                if path.exists():
+                    path.unlink()
+                self.files.pop(0)
+            else:
+                break
+
+
+#file_cleaner = FileCleaner()
+
 def toggle_audio_src(choice):
     if choice == "mic":
         return gr.update(source="microphone", value=None, label="Microphone")
@@ -235,8 +259,9 @@ def predict(model, text, melody_filepath, duration, dimension, topk, topp, tempe
                     overlapping_output_fadein = output_segments[i][:, :, :overlap_samples]
                     #overlapping_output_fadein = apply_fade(overlapping_output_fadein,sample_rate=MODEL.sample_rate,duration=overlap,out=False,start=False, curve_start=0.0, current_device=MODEL.device)
                     overlapping_output_fadein = apply_tafade(overlapping_output_fadein,sample_rate=MODEL.sample_rate,duration=overlap,out=False,start=False, shape="linear")
-
+                    
                     overlapping_output = torch.cat([overlapping_output_fadeout[:, :, :-(overlap_samples // 2)], overlapping_output_fadein],dim=2)
+                    ###overlapping_output, overlap_sample_rate = apply_splice_effect(overlapping_output_fadeout, MODEL.sample_rate, overlapping_output_fadein, MODEL.sample_rate, overlap)
                     print(f" overlap size Fade:{overlapping_output.size()}\n output: {output.size()}\n segment: {output_segments[i].size()}")
                     ##overlapping_output = torch.cat([output[:, :, -overlap_samples:], output_segments[i][:, :, :overlap_samples]], dim=1) #stack tracks
                     ##print(f" overlap size stack:{overlapping_output.size()}\n output: {output.size()}\n segment: {output_segments[i].size()}")

audiocraft/data/audio_utils.py

@@ -262,4 +262,32 @@ def apply_fade(audio: torch.Tensor, sample_rate, duration=3.0, out=True, start=T
 
     wav = normalize_loudness(audio_faded,sample_rate, loudness_headroom_db=18, loudness_compressor=True)
     _clip_wav(wav, log_clipping=False, stem_name=stem_name)
-    return wav
+    return wav
+
+def apply_splice_effect(waveform1, sample_rate1, waveform2, sample_rate2, overlap):
+    # Convert sample rates to integers
+    sample_rate1 = int(sample_rate1)
+    sample_rate2 = int(sample_rate2)
+
+    # Convert tensors to mono-channel if needed
+    if waveform1.ndim > 2:
+        waveform1 = waveform1.mean(dim=1)
+    if waveform2.ndim > 2:
+        waveform2 = waveform2.mean(dim=1)
+
+    ## Convert tensors to numpy arrays
+    #waveform1_np = waveform1.numpy()
+    #waveform2_np = waveform2.numpy()
+
+    # Apply splice effect using torchaudio.sox_effects.apply_effects_tensor
+    effects = [
+        ["splice", f"-q {waveform1},{overlap}"],
+    ]
+    output_waveform, output_sample_rate = torchaudio.sox_effects.apply_effects_tensor(
+        torch.cat([waveform1.unsqueeze(0), waveform2.unsqueeze(0)], dim=2),
+        sample_rate1,
+        effects
+    )
+
+    return output_waveform.squeeze(0), output_sample_rate
+

pre-requirements.txt

@@ -0,0 +1,2 @@
+pip>=23.2
+gradio_client==0.2.7

requirements.txt

@@ -11,6 +11,7 @@ sentencepiece
 spacy==3.5.2
 torch>=2.0.0
 torchaudio>=2.0.0
+soundfile
 huggingface_hub
 tqdm
 transformers