MERaLiON
/

MERaLiON-AudioLLM-Whisper-SEA-LION

@@ -406,17 +406,17 @@ Refer to the [AudioBench paper](https://arxiv.org/abs/2406.16020) for more detai
 </table>
 </div>
-## Uses
-Here we provide a code snippet illustrating the process of loading both the processor and model, alongside detailed instructions on executing the MERaLiON-AudioLLM model for content generation.
 > [!WARNING]
 > **Out of Scope use**: This model is not intended for use in tool calling, math, and coding tasks.
-### CPU Inference
-MERaLiON-AudioLLM also supports batch inference.
 ```python
 import librosa
@@ -459,7 +459,7 @@ generated_ids = outputs[:, inputs['input_ids'].size(1):]
 response = processor.batch_decode(generated_ids, skip_special_tokens=True)
 ```
-### GPU Inference
 ```python
 import torch
@@ -513,10 +513,6 @@ generated_ids = outputs[:, inputs['input_ids'].size(1):]
 response = processor.batch_decode(generated_ids, skip_special_tokens=True)
 ```
-### vLLM Inference
-We support hosting the model using vLLM framework. Refer to the guide [here](vllm_plugin_meralion/README.md).
 ## Disclaimer
 The current MERaLiON-AudioLLM has not been specifically aligned for safety and may generate content that is inappropriate, offensive, or harmful. Developers and users are responsible for performing their own safety fine-tuning and implementing necessary security measures. The authors shall not be held liable for any claims, damages, or other liabilities arising from the use of the released models, weights, or code.

 </table>
 </div>
+## Inference
 > [!WARNING]
 > **Out of Scope use**: This model is not intended for use in tool calling, math, and coding tasks.
+### vLLM Inference
+We support hosting the model using vLLM framework. Refer to the guide [here](vllm_plugin_meralion/README.md).
+### Huggingface CPU Inference
 ```python
 import librosa
 response = processor.batch_decode(generated_ids, skip_special_tokens=True)
 ```
+### Huggingface GPU Inference
 ```python
 import torch
 response = processor.batch_decode(generated_ids, skip_special_tokens=True)
 ```
 ## Disclaimer
 The current MERaLiON-AudioLLM has not been specifically aligned for safety and may generate content that is inappropriate, offensive, or harmful. Developers and users are responsible for performing their own safety fine-tuning and implementing necessary security measures. The authors shall not be held liable for any claims, damages, or other liabilities arising from the use of the released models, weights, or code.

vllm_plugin_meralion/README.md CHANGED Viewed

@@ -13,7 +13,7 @@ pip install transformers==4.46.3
 ```
 As the [vLLM documentation](https://docs.vllm.ai/en/stable/models/adding_model.html#out-of-tree-model-integration) recommends,
-we provide a way to register our model via [vLLM plugins](https://docs.vllm.ai/en/stable/design/plugin_system.html#plugin-system).
 ```bash

 ```
 As the [vLLM documentation](https://docs.vllm.ai/en/stable/models/adding_model.html#out-of-tree-model-integration) recommends,
+we provide a way to register our model via [vLLM plugins](https://docs.vllm.ai/en/stable/design/plugin_system.html#plugin-system). Simply install by running the following command.
 ```bash