imsanjoykb/sqlCoder-Qwen2.5-8bit

🤗 HF Repo | ♾️ Colab

Introducing the latest fine-tuned version of Qwen2.5-Coder-14B-Instruct, specifically tailored for SQL code generation. Built on the robust 14-billion parameter Qwen2.5-Coder architecture, this model leverages advanced configurations like bfloat16 precision and a custom quantization setup, optimized for efficient 4-bit computation. With a maximum context window of 32K tokens, this model supports extensive SQL sequences and complex query generation without compromising accuracy or performance.

Our fine-tuning process has enriched this model with domain-specific SQL patterns and nuanced query constructions, making it exceptionally adept at handling real-world SQL requirements, from query creation to debugging and optimization. By combining Qwen2.5's foundational strengths with targeted training on custom SQL data, this model achieves a powerful balance of general-purpose code understanding and SQL-specific precision, making it an ideal tool for developers and data engineers seeking top-tier SQL generation capabilities.

Inference

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

# Import necessary libraries
from unsloth import FastLanguageModel
import torch

# Define the model name and other parameters
model_name = "imsanjoykb/sqlCoder-Qwen2.5-8bit"
max_seq_length = 2048
dtype = None
load_in_4bit = True

# Load the model and tokenizer from Hugging Face
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=model_name,
    max_seq_length=max_seq_length,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
)

# Enable faster inference
FastLanguageModel.for_inference(model)

# Define the prompt template
odoo_text2sql_prompt = """Below is an instruction describing a task related to generating a SQL query specifically for Odoo's database structure. The input provides relevant context about Odoo models or data fields from {db_schema}. Write a SQL query that fulfills the given task using Odoo's database schema.

### Instruction:
Generate a SQL query in the context of Odoo to {}

### Input:
{}

### Response:
{}
"""

# Optionally, use a TextStreamer for continuous inference
from transformers import TextStreamer

# Prepare the input text for continuous inference
instruction = ""
input_text = "What is the top profitable product?"
output_text = ""

# Tokenize the input text
inputs = tokenizer(
    [
        odoo_text2sql_prompt.format(instruction, input_text, output_text)
    ],
    return_tensors="pt"
).to("cuda")

# Initialize the TextStreamer
text_streamer = TextStreamer(tokenizer)

# Generate the output using the model with TextStreamer
_ = model.generate(**inputs, streamer=text_streamer, max_new_tokens=350)

Model Download

Model	#Total Params	#Active Params	Context Length	Download
sqlCoder-Qwen2.5-8bit	14B	2.4B	128k	🤗 HuggingFace

Uploaded model

• Developed by: Sanjoy Biswas
• License: apache-2.0