akameswa/mistral-7b-instruct-code-ties-old

mistral-7b-instruct-code-ties

mistral-7b-instruct-code-ties is a merge of the following models using mergekit:

• akameswa/mistral-7b-instruct-javascript-16bit
• akameswa/mistral-7b-instruct-python-16bit
• akameswa/mistral-7b-instruct-java-16bit
• akameswa/mistral-7b-instruct-javascript-16bit

🧩 Configuration

models:
  - model: akameswa/mistral-7b-instruct-v0.2-bnb-16bit
  - model: akameswa/mistral-7b-instruct-javascript-16bit
    parameters:
      density: 0.85
      weight: 0.25
  - model: akameswa/mistral-7b-instruct-python-16bit
    parameters:
      density: 0.85
      weight: 0.25
  - model: akameswa/mistral-7b-instruct-java-16bit
    parameters:
      density: 0.85
      weight: 0.25
  - model: akameswa/mistral-7b-instruct-javascript-16bit
    parameters:
      density: 0.85
      weight: 0.25
merge_method: ties
base_model: akameswa/mistral-7b-instruct-v0.2-bnb-16bit
parameters:
  normalize: true
dtype: float16

Inference

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "akameswa/mistral-7b-instruct-code-ties",
    max_seq_length = 2048,
)

xlcost_prompt = """Below is a description of a programming task. Write a response that appropriately completes the task based on the given description.

### Description:
{}

### Code:
{}"""

FastLanguageModel.for_inference(model)
inputs = tokenizer(
[
    xlcost_prompt.format(
        "Continue the fibonnaci sequence.",
        "",
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)

• Link to inference notebook