tahamajs/bitcoin-analyst-training-archive-llama-3.2-3b-instruct-bitcoin-analyst-perfect_v2

Llama-3.2-3B Instruct - Advanced Bitcoin Analyst

This repository contains a highly specialized version of meta-llama/Llama-3.2-3B-Instruct, expertly fine-tuned to function as a Bitcoin and cryptocurrency market analyst. This model is the result of a "continuation training" process, where an already specialized model was further refined on a targeted dataset.

This model excels at understanding and responding to complex instructions related to blockchain technology, financial markets, and technical/fundamental analysis of cryptocurrencies.

🧠 Training Procedure

The final model was created through a sophisticated multi-stage process designed to build upon and deepen existing knowledge.

Stage 1: Initial Specialization (Adapter Merge)

The process began with the base meta-llama/Llama-3.2-3B-Instruct model. This base was then merged with a previously fine-tuned, high-performing LoRA adapter to create an initial specialized model.

• Initial Adapter: tahamajs/llama-3.2-3b-instruct-bitcoin-analyst-perfect

Stage 2: Continued Fine-Tuning (New LoRA)

A new LoRA adapter was then trained on top of the already-merged model from Stage 1. This continuation training allowed the model to further refine its expertise using a specific dataset, improving its nuance and instruction-following on relevant topics.

• Dataset: tahamajs/bitcoin-llm-finetuning-dataset

Stage 3: Final Merge

The final step was to merge the newly trained adapter from Stage 2 into the model. This repository hosts this fully merged, standalone model, which contains the cumulative knowledge of the base model, the first specialized adapter, and the second round of continuation training.

---

📊 Training Details

Hyperparameters

The second stage of LoRA fine-tuning was performed with the following key hyperparameters:

Parameter	Value
learning_rate	1e-4
num_train_epochs	1
lora_r	16
lora_alpha	32
optimizer	paged_adamw_32bit
precision	bf16

Training Loss

The training loss shows a clear downward trend, indicating that the model was successfully learning from the dataset. The process started with a loss of ~2.18 and converged to a loss in the ~1.4-1.6 range, demonstrating effective knowledge acquisition. The fluctuations are normal during training and reflect the varying difficulty of the data in each batch.

---

🚀 How to Use

This is a fully merged model and can be used directly with the transformers library. For best results, use the Llama 3 chat template to format your prompts.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Use the ID of the repository where this model is hosted
model_id = "tahamajs/llama-3.2-3b-instruct-bitcoin-analyst-perfect_v2"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Use the Llama 3 chat template for instruction-following
messages = [
    {"role": "user", "content": "Analyze the current sentiment around Bitcoin based on the concept of the Fear & Greed Index. What does a high 'Greed' value typically imply for the short-term market?"},
]

# Apply the chat template and tokenize
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Generate a response
outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)

# Decode and print the output
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))

Disclaimer

This model is provided for informational and educational purposes only. It is not financial advice. The outputs are generated by an AI and may contain errors or inaccuracies. Always perform your own due diligence and consult with a qualified financial professional before making any investment decisions.