Financial LLaMA-3.1-8B - Experimental Model

Model Description

This is an experimental fine-tuning attempt of Meta's LLaMA-3.1-8B-Instruct model on financial data using LoRA (Low-Rank Adaptation).

Training Details

Base Model: meta-llama/Llama-3.1-8B-Instruct
Training Method: LoRA fine-tuning
Dataset: Financial news, company fundamentals, market analysis
Training Framework: Transformers + Custom LoRA implementation
Training Environment: Google Colab with GPU

Training Configuration

{{
  "model_name": "meta-llama/Llama-3.1-8B-Instruct",
  "training_type": "lora",
  "lora_r": 8,
  "lora_alpha": 16,
  "learning_rate": 1e-4,
  "num_train_epochs": 2,
  "per_device_train_batch_size": 1,
  "gradient_accumulation_steps": 8
}}

Dataset Information

Data Sources

NewsAPI: Financial news articles (truncated)
AlphaVantage: Company fundamental data
Yahoo Finance: Market analysis data
RSS Feeds: Financial news feeds

Data Statistics

Total Examples: ~93 training examples
Data Types: News articles, fundamentals, market analysis
Quality Issues: Truncated content, mixed formats

File Structure

financial_llama_model_clean/ ├── config.json # Model configuration ├── generation_config.json # Generation settings ├── model-00001-of-00004.safetensors # Model weights (4.9GB) ├── model-00002-of-00004.safetensors # Model weights (5.0GB) ├── model-00003-of-00004.safetensors # Model weights (4.9GB) ├── model-00004-of-00004.safetensors # Model weights (1.3GB) ├── model.safetensors.index.json # Weight mapping ├── lora_weights.pt # LoRA weights (84MB) ├── training_metrics.json # Training results ├── tokenizer.json # Tokenizer ├── tokenizer_config.json # Tokenizer config └── special_tokens_map.json # Special tokens

Research Value

Despite being broken, this model serves as:

Educational Example: Shows what happens with poor training setup
Research Data: Training failure case study
Debugging Reference: Helps understand training pitfalls

Citation

@model{{experimental_financial_llama_2024,
  title={{Financial LLaMA-3.1-8B - Experimental/Broken Model}},
  author={{Your Name}},
  year={{2024}},
  publisher={{HuggingFace}},
  url={{https://huggingface.co/{repo_id}}},
  note={{Experimental model - training failed, for research purposes only}}
}}

Disclaimer

This model is provided for educational and research purposes only. It demonstrates a failed training attempt and should not be used for any practical applications. The training failure provides valuable insights into the importance of proper data preprocessing, parameter tuning, and training monitoring.

Contact

Authors: Karthik Narayan Venkatasubramanian & Gishor Thavakumar
Email: [email protected] | [email protected]
Project: Financial AI Research
Status: Experimental Phase

"""

tgishor
/

financial-llama-8b-experimental