NumbersStation

/

nsql-llama-2-7B

like 79

Follow

NumbersStation 26

After fine tuning on custom dataset, the model is not generating the expected output for the exact same query which in my training dataset.

Below query is part of my custom dataset

CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR() -- Using valid SQLite, answer the following questions for the tables provided above. -- sum of call duration week on week ?
)

My query which I have given the training dataset is
SELECT
TO_VARIANT(DATE_TRUNC('MONTH', DATE_TRUNC('WEEK', datetime)))::STRING AS month_start_date,
TO_VARIANT(EXTRACT(WEEK FROM datetime))::STRING AS week_number,
SUM(DURATION) AS monthly_weekly_call_duration
FROM
AppTable
GROUP BY
month_start_date,
week_number
ORDER BY
month_start_date, week_number

Now when I do the inference its giving me the following answer:

text = """
CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR
) -- Using valid SQL, answer the following questions for the tables provided above. -- sum of call duration week on week ?

"""

model_input = tokenizer(text, return_tensors="pt").to("cuda")

generated_ids = model.generate(**model_input, max_new_tokens=100)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

CREATE TABLE APPTABLE (
DATETIME DATE,
DURATION FLOAT,
RESPONSETIME VARCHAR
) -- Using valid SQL, answer the following questions for the tables provided above. -- sum of call duration week on week ?

SELECT SUM(DURATION) FROM APPTABLE;

Output Generated is incorrect. Can you please share some inputs to get a better output as mentioned in my training dataset used for fine tuning.