Model Card for Model ID

This model is a fine-tuned version of TinyLlama-1.1B-Chat-v1.0, specialized for question-answering and summarization tasks related to the topic of DNA data storage. It was trained using the PEFT (Parameter-Efficient Fine-Tuning) method with LoRA adapters on a custom dataset tatsu-lab/alpaca.

Model Details

Model Description

This is a fine-tuned language model based on the TinyLlama-1.1B-Chat-v1.0 architecture. The model was trained to improve its ability to understand, summarize, and answer questions from text related to DNA data storage technology. It utilizes LoRA adapters, which makes the fine-tuned checkpoint small and efficient. This model is intended for research and educational purposes to explore the application of LLMs in niche, domain-specific tasks.

Developed by: Abhishek Singh
Model type: Causal language model
Language(s) (NLP): English
License: Apache 2.0 (Inherits the license from the base model)
Finetuned from model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Model Sources [optional]

Repository: https://huggingface.co/learn-abc/tatsu-lab_alpaca-tinyllama-chat-bnb-4bit

Uses

Direct Use

This model is intended to be used for the following purposes:

Summarizing key points from new documents or texts about DNA data storage.
Answering specific questions based on provided context regarding DNA data storage.
Generating short, informative explanations on the topic.

Out-of-Scope Use

This model is not suitable for:

General-purpose chat or conversational tasks on topics outside of DNA data storage.
Generating creative writing or essays.
Factual question-answering on general knowledge, as its knowledge is constrained to the fine-tuning data.

Bias, Risks, and Limitations

This model has several limitations due to its specialized nature and small size:

Domain-Specific Knowledge: The model's knowledge is highly specialized. It may provide incorrect or nonsensical information (hallucinate) when asked about topics outside of DNA data storage.
Potential for Bias: The model inherits the biases of its base model, TinyLlama.
Simplicity: The model is not a substitute for expert advice or comprehensive research. It should be used as a supplementary tool for text analysis.

Recommendations

Users should be aware of the model's limitations and verify any critical information it provides. It is recommended to use the model with a clear, specific prompt that includes relevant context for the best results.

How to Get Started with the Model

You can get started with the model by loading it directly from the Hugging Face Hub using the transformers library.

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "learn-abc/tatsu-lab_alpaca-tinyllama-chat-bnb-4bit"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Example usage
prompt = """### Instruction:
Summarize the key findings from the provided text about DNA data storage.

### Input:
Deoxyribonucleic acid (DNA) has been successfully proposed as an advanced archival storage medium, due to its extraordinary data capacity and robust stability. ... (rest of your text here)

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

Training Details

Training Data

The model was fine-tuned on a custom dataset containing text, summaries, and Q&A pairs related to the topic of DNA data storage. The dataset was formatted into a chat-like template with ### Instruction:, ### Input:, and ### Response: sections.

Training Procedure

1. Preprocessing The training data was pre-processed into a chat template format to prepare it for the model. The tokenizer's pad_token was set to the eos_token to handle variable-length sequences.

2. Training Hyperparameters

Fine-tuning Method: PEFT (LoRA)
Training regime: bf16 mixed precision

Evaluation

No formal evaluation metrics were calculated for this model. Its performance was qualitatively assessed by generating responses to prompts and checking for relevance and accuracy with respect to the fine-tuning data.