Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,414 @@
+---
+license: apache-2.0
+language:
+- en
+- hi
+library_name: transformers
+tags:
+- text-to-speech
+- tts
+- hindi
+- english
+- llama
+- audio
+- speech
+- india
+datasets:
+- proprietary
+pipeline_tag: text-to-speech
+co2_eq_emissions:
+  emissions: 0
+  source: "Not specified"
+  training_type: "unknown"
+  geographical_location: "unknown"
+---
+
+# <span style="color: #7FFF7F;">Veena GGUF Models</span>
+
+
+## <span style="color: #7F7FFF;">Model Generation Details</span>
+
+This model was generated using [llama.cpp](https://github.com/ggerganov/llama.cpp) at commit [`8846aace`](https://github.com/ggerganov/llama.cpp/commit/8846aace4934ad29651ea61b8c7e3f6b0556e3d2).
+
+
+
+
+
+---
+
+## <span style="color: #7FFF7F;">Quantization Beyond the IMatrix</span>
+
+I've been experimenting with a new quantization approach that selectively elevates the precision of key layers beyond what the default IMatrix configuration provides.
+
+In my testing, standard IMatrix quantization underperforms at lower bit depths, especially with Mixture of Experts (MoE) models. To address this, I'm using the `--tensor-type` option in `llama.cpp` to manually "bump" important layers to higher precision. You can see the implementation here:  
+👉 [Layer bumping with llama.cpp](https://github.com/Mungert69/GGUFModelBuilder/blob/main/model-converter/tensor_list_builder.py)
+
+While this does increase model file size, it significantly improves precision for a given quantization level.
+
+### **I'd love your feedback—have you tried this? How does it perform for you?**
+
+
+
+
+---
+
+<a href="https://readyforquantum.com/huggingface_gguf_selection_guide.html" style="color: #7FFF7F;">
+  Click here to get info on choosing the right GGUF model format
+</a>
+
+---
+
+
+
+<!--Begin Original Model Card-->
+
+
+# Veena - Text to Speech for Indian Languages
+
+Veena is a state-of-the-art neural text-to-speech (TTS) model specifically designed for Indian languages, developed by Maya Research. Built on a Llama architecture backbone, Veena generates natural, expressive speech in Hindi and English with remarkable quality and ultra-low latency.
+
+## Model Overview
+
+**Veena** is a 3B parameter autoregressive transformer model based on the Llama architecture. It is designed to synthesize high-quality speech from text in Hindi and English, including code-mixed scenarios. The model outputs audio at a 24kHz sampling rate using the SNAC neural codec.
+
+* **Model type:** Autoregressive Transformer
+* **Base Architecture:** Llama (3B parameters)
+* **Languages:** Hindi, English
+* **Audio Codec:** SNAC @ 24kHz
+* **License:** Apache 2.0
+* **Developed by:** Maya Research
+* **Model URL:** [https://huggingface.co/maya-research/veena](https://huggingface.co/maya-research/veena)
+
+## Key Features
+
+* **4 Distinct Voices:** `kavya`, `agastya`, `maitri`, and `vinaya` - each with unique vocal characteristics.
+* **Multilingual Support:** Native Hindi and English capabilities with code-mixed support.
+* **Ultra-Fast Inference:** Sub-80ms latency on H100-80GB GPUs.
+* **High-Quality Audio:** 24kHz output with the SNAC neural codec.
+* **Production-Ready:** Optimized for real-world deployment with 4-bit quantization support.
+
+## How to Get Started with the Model
+
+### Installation
+
+To use Veena, you need to install the `transformers`, `torch`, `torchaudio`, `snac`, and `bitsandbytes` libraries.
+
+```bash
+pip install transformers torch torchaudio
+pip install snac bitsandbytes  # For audio decoding and quantization
+```
+
+### Basic Usage
+
+The following Python code demonstrates how to generate speech from text using Veena with 4-bit quantization for efficient inference.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from snac import SNAC
+import soundfile as sf
+
+# Model configuration for 4-bit inference
+quantization_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_use_double_quant=True,
+)
+
+# Load model and tokenizer
+model = AutoModelForCausalLM.from_pretrained(
+    "maya-research/veena-tts",
+    quantization_config=quantization_config,
+    device_map="auto",
+    trust_remote_code=True,
+)
+tokenizer = AutoTokenizer.from_pretrained("maya-research/veena-tts", trust_remote_code=True)
+
+# Initialize SNAC decoder
+snac_model = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").eval().cuda()
+
+# Control token IDs (fixed for Veena)
+START_OF_SPEECH_TOKEN = 128257
+END_OF_SPEECH_TOKEN = 128258
+START_OF_HUMAN_TOKEN = 128259
+END_OF_HUMAN_TOKEN = 128260
+START_OF_AI_TOKEN = 128261
+END_OF_AI_TOKEN = 128262
+AUDIO_CODE_BASE_OFFSET = 128266
+
+# Available speakers
+speakers = ["kavya", "agastya", "maitri", "vinaya"]
+
+def generate_speech(text, speaker="kavya", temperature=0.4, top_p=0.9):
+    """Generate speech from text using specified speaker voice"""
+
+    # Prepare input with speaker token
+    prompt = f"<spk_{speaker}> {text}"
+    prompt_tokens = tokenizer.encode(prompt, add_special_tokens=False)
+
+    # Construct full sequence: [HUMAN] <spk_speaker> text [/HUMAN] [AI] [SPEECH]
+    input_tokens = [
+        START_OF_HUMAN_TOKEN,
+        *prompt_tokens,
+        END_OF_HUMAN_TOKEN,
+        START_OF_AI_TOKEN,
+        START_OF_SPEECH_TOKEN
+    ]
+
+    input_ids = torch.tensor([input_tokens], device=model.device)
+
+    # Calculate max tokens based on text length
+    max_tokens = min(int(len(text) * 1.3) * 7 + 21, 700)
+
+    # Generate audio tokens
+    with torch.no_grad():
+        output = model.generate(
+            input_ids,
+            max_new_tokens=max_tokens,
+            do_sample=True,
+            temperature=temperature,
+            top_p=top_p,
+            repetition_penalty=1.05,
+            pad_token_id=tokenizer.pad_token_id,
+            eos_token_id=[END_OF_SPEECH_TOKEN, END_OF_AI_TOKEN]
+        )
+
+    # Extract SNAC tokens
+    generated_ids = output[0][len(input_tokens):].tolist()
+    snac_tokens = [
+        token_id for token_id in generated_ids
+        if AUDIO_CODE_BASE_OFFSET <= token_id < (AUDIO_CODE_BASE_OFFSET + 7 * 4096)
+    ]
+
+    if not snac_tokens:
+        raise ValueError("No audio tokens generated")
+
+    # Decode audio
+    audio = decode_snac_tokens(snac_tokens, snac_model)
+    return audio
+
+def decode_snac_tokens(snac_tokens, snac_model):
+    """De-interleave and decode SNAC tokens to audio"""
+    if not snac_tokens or len(snac_tokens) % 7 != 0:
+        return None
+
+    # De-interleave tokens into 3 hierarchical levels
+    codes_lvl = [[] for _ in range(3)]
+    llm_codebook_offsets = [AUDIO_CODE_BASE_OFFSET + i * 4096 for i in range(7)]
+
+    for i in range(0, len(snac_tokens), 7):
+        # Level 0: Coarse (1 token)
+        codes_lvl[0].append(snac_tokens[i] - llm_codebook_offsets[0])
+        # Level 1: Medium (2 tokens)
+        codes_lvl[1].append(snac_tokens[i+1] - llm_codebook_offsets[1])
+        codes_lvl[1].append(snac_tokens[i+4] - llm_codebook_offsets[4])
+        # Level 2: Fine (4 tokens)
+        codes_lvl[2].append(snac_tokens[i+2] - llm_codebook_offsets[2])
+        codes_lvl[2].append(snac_tokens[i+3] - llm_codebook_offsets[3])
+        codes_lvl[2].append(snac_tokens[i+5] - llm_codebook_offsets[5])
+        codes_lvl[2].append(snac_tokens[i+6] - llm_codebook_offsets[6])
+
+    # Convert to tensors for SNAC decoder
+    hierarchical_codes = []
+    for lvl_codes in codes_lvl:
+        tensor = torch.tensor(lvl_codes, dtype=torch.int32, device=snac_model.device).unsqueeze(0)
+        if torch.any((tensor < 0) | (tensor > 4095)):
+            raise ValueError("Invalid SNAC token values")
+        hierarchical_codes.append(tensor)
+
+    # Decode with SNAC
+    with torch.no_grad():
+        audio_hat = snac_model.decode(hierarchical_codes)
+
+    return audio_hat.squeeze().clamp(-1, 1).cpu().numpy()
+
+# --- Example Usage ---
+
+# Hindi
+text_hindi = "आज मैंने एक नई तकनीक के बारे में सीखा जो कृत्रिम बुद्धिमत्ता का उपयोग करके मानव जैसी आवाज़ उत्पन्न कर सकती है।"
+audio = generate_speech(text_hindi, speaker="kavya")
+sf.write("output_hindi_kavya.wav", audio, 24000)
+
+# English
+text_english = "Today I learned about a new technology that uses artificial intelligence to generate human-like voices."
+audio = generate_speech(text_english, speaker="agastya")
+sf.write("output_english_agastya.wav", audio, 24000)
+
+# Code-mixed
+text_mixed = "मैं तो पूरा presentation prepare कर चुका हूं! कल रात को ही मैंने पूरा code base चेक किया।"
+audio = generate_speech(text_mixed, speaker="maitri")
+sf.write("output_mixed_maitri.wav", audio, 24000)
+```
+
+## Uses
+
+Veena is ideal for a wide range of applications requiring high-quality, low-latency speech synthesis for Indian languages, including:
+
+  * **Accessibility:** Screen readers and voice-enabled assistance for visually impaired users.
+  * **Customer Service:** IVR systems, voice bots, and automated announcements.
+  * **Content Creation:** Dubbing for videos, e-learning materials, and audiobooks.
+  * **Automotive:** In-car navigation and infotainment systems.
+  * **Edge Devices:** Voice-enabled smart devices and IoT applications.
+
+## Technical Specifications
+
+### Architecture
+
+Veena leverages a 3B parameter transformer-based architecture with several key innovations:
+
+  * **Base Architecture:** Llama-style autoregressive transformer (3B parameters)
+  * **Audio Codec:** SNAC (24kHz) for high-quality audio token generation
+  * **Speaker Conditioning:** Special speaker tokens (`<spk_kavya>`, `<spk_agastya>`, `<spk_maitri>`, `<spk_vinaya>`)
+  * **Parameter-Efficient Training:** LoRA adaptation with differentiated ranks for attention and FFN modules.
+  * **Context Length:** 2048 tokens
+
+### Training
+
+#### Training Infrastructure
+
+  * **Hardware:** 8× NVIDIA H100 80GB GPUs
+  * **Distributed Training:** DDP with optimized communication
+  * **Precision:** BF16 mixed precision training with gradient checkpointing
+  * **Memory Optimization:** 4-bit quantization with NF4 + double quantization
+
+#### Training Configuration
+
+  * **LoRA Configuration:**
+      * `lora_rank_attention`: 192
+      * `lora_rank_ffn`: 96
+      * `lora_alpha`: 2× rank (384 for attention, 192 for FFN)
+      * `lora_dropout`: 0.05
+      * `target_modules`: `["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]`
+      * `modules_to_save`: `["embed_tokens"]`
+  * **Optimizer Configuration:**
+      * `optimizer`: AdamW (8-bit)
+      * `optimizer_betas`: (0.9, 0.98)
+      * `optimizer_eps`: 1e-5
+      * `learning_rate_peak`: 1e-4
+      * `lr_scheduler`: cosine
+      * `warmup_ratio`: 0.02
+  * **Batch Configuration:**
+      * `micro_batch_size`: 8
+      * `gradient_accumulation_steps`: 4
+      * `effective_batch_size`: 256
+
+#### Training Data
+
+Veena was trained on **proprietary, high-quality datasets** specifically curated for Indian language TTS.
+
+  * **Data Volume:** 15,000+ utterances per speaker (60,000+ total)
+  * **Languages:** Native Hindi and English utterances with code-mixed support
+  * **Speaker Diversity:** 4 professional voice artists with distinct characteristics
+  * **Audio Quality:** Studio-grade recordings at 24kHz sampling rate
+  * **Content Diversity:** Conversational, narrative, expressive, and informational styles
+
+**Note:** The training datasets are proprietary and not publicly available.
+
+## Performance Benchmarks
+
+| Metric                | Value                     |
+| --------------------- | ------------------------- |
+| Latency (H100-80GB)   | \<80ms                     |
+| Latency (A100-40GB)   | \~120ms                    |
+| Latency (RTX 4090)    | \~200ms                    |
+| Real-time Factor      | 0.05x                     |
+| Throughput            | \~170k tokens/s (8×H100)   |
+| Audio Quality (MOS)   | 4.2/5.0                   |
+| Speaker Similarity    | 92%                       |
+| Intelligibility       | 98%                       |
+
+## Risks, Limitations and Biases
+
+  * **Language Support:** Currently supports only Hindi and English. Performance on other Indian languages is not guaranteed.
+  * **Speaker Diversity:** Limited to 4 speaker voices, which may not represent the full diversity of Indian accents and dialects.
+  * **Hardware Requirements:** Requires a GPU for real-time or near-real-time inference. CPU performance will be significantly slower.
+  * **Input Length:** The model is limited to a maximum input length of 2048 tokens.
+  * **Bias:** The model's performance and voice characteristics are a reflection of the proprietary training data. It may exhibit biases present in the data.
+
+## Future Updates
+
+We are actively working on expanding Veena's capabilities:
+
+  * Support for Tamil, Telugu, Bengali, Marathi, and other Indian languages.
+  * Additional speaker voices with regional accents.
+  * Emotion and prosody control tokens.
+  * Streaming inference support.
+  * CPU optimization for edge deployment.
+
+## Citing
+
+If you use Veena in your research or applications, please cite:
+
+```bibtex
+@misc{veena2025,
+  title={Veena: Open Source Text-to-Speech for Indian Languages},
+  author={Maya Research Team},
+  year={2025},
+  publisher={HuggingFace},
+  url={[https://huggingface.co/maya-research/veena-tts](https://huggingface.co/maya-research/veena-tts)}
+}
+```
+
+## Acknowledgments
+
+We thank the open-source community and all contributors who made this project possible. Special thanks to the voice artists who provided high-quality recordings for training.
+
+<!--End Original Model Card-->
+
+---
+
+# <span id="testllm" style="color: #7F7FFF;">🚀 If you find these models useful</span>
+
+Help me test my **AI-Powered Quantum Network Monitor Assistant** with **quantum-ready security checks**:  
+
+👉 [Quantum Network Monitor](https://readyforquantum.com/?assistant=open&utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface_repo_readme)  
+
+
+The full Open Source Code for the Quantum Network Monitor Service available at my github repos ( repos with NetworkMonitor in the name) : [Source Code Quantum Network Monitor](https://github.com/Mungert69). You will also find the code I use to quantize the models if you want to do it yourself [GGUFModelBuilder](https://github.com/Mungert69/GGUFModelBuilder)
+
+💬 **How to test**:  
+ Choose an **AI assistant type**:  
+   - `TurboLLM` (GPT-4.1-mini)  
+   - `HugLLM` (Hugginface Open-source models)  
+   - `TestLLM` (Experimental CPU-only)  
+
+### **What I’m Testing**  
+I’m pushing the limits of **small open-source models for AI network monitoring**, specifically:  
+- **Function calling** against live network services  
+- **How small can a model go** while still handling:  
+  - Automated **Nmap security scans**  
+  - **Quantum-readiness checks**  
+  - **Network Monitoring tasks**  
+
+🟡 **TestLLM** – Current experimental model (llama.cpp on 2 CPU threads on huggingface docker space):  
+- ✅ **Zero-configuration setup**  
+- ⏳ 30s load time (slow inference but **no API costs**) . No token limited as the cost is low.
+- 🔧 **Help wanted!** If you’re into **edge-device AI**, let’s collaborate!  
+
+### **Other Assistants**  
+🟢 **TurboLLM** – Uses **gpt-4.1-mini** :
+- **It performs very well but unfortunatly OpenAI charges per token. For this reason tokens usage is limited. 
+- **Create custom cmd processors to run .net code on Quantum Network Monitor Agents**
+- **Real-time network diagnostics and monitoring**
+- **Security Audits**
+- **Penetration testing** (Nmap/Metasploit)  
+
+🔵 **HugLLM** – Latest Open-source models:  
+- 🌐 Runs on Hugging Face Inference API. Performs pretty well using the lastest models hosted on Novita.
+
+### 💡 **Example commands you could test**:  
+1. `"Give me info on my websites SSL certificate"`  
+2. `"Check if my server is using quantum safe encyption for communication"`  
+3. `"Run a comprehensive security audit on my server"`
+4. '"Create a cmd processor to .. (what ever you want)" Note you need to install a [Quantum Network Monitor Agent](https://readyforquantum.com/Download/?utm_source=huggingface&utm_medium=referral&utm_campaign=huggingface_repo_readme) to run the .net code on. This is a very flexible and powerful feature. Use with caution!
+
+### Final Word
+
+I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is [open source](https://github.com/Mungert69). Feel free to use whatever you find helpful.
+
+If you appreciate the work, please consider [buying me a coffee](https://www.buymeacoffee.com/mahadeva) ☕. Your support helps cover service costs and allows me to raise token limits for everyone.
+
+I'm also open to job opportunities or sponsorship.
+
+Thank you! 😊