---
license: apache-2.0
tags:
- trl
- sft
library_name: transformers
---

# CatMemo: Fine-Tuning Large Language Models for Financial Applications

## Model Overview
This model, **CatMemo**, is fine-tuned using **Data Fusion** techniques for financial applications. It was developed as part of the FinLLM Challenge Task and focuses on enhancing the performance of large language models in finance-specific tasks such as question answering, document summarization, and sentiment analysis.

### Key Features
- Fine-tuned on financial datasets using **Supervised Fine-Tuning (SFT)** techniques.
- Optimized for **Transfer Reinforcement Learning (TRL)** workflows.
- Specialized for tasks requiring domain-specific context in financial applications.

## Usage
You can use this model with the [Hugging Face Transformers library](https://huggingface.co/docs/transformers/) to perform financial text analysis. Below is a quick example:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load the model and tokenizer
model_name = "zeeshanali01/cryptotunned"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Tokenize input
inputs = tokenizer("What are the key takeaways from the latest earnings report?", return_tensors="pt")

# Generate output
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Training Details
This model was fine-tuned using **Data Fusion** methods on domain-specific financial datasets. The training pipeline includes:
- Preprocessing financial documents and datasets to enhance model understanding.
- Applying **Supervised Fine-Tuning (SFT)** to optimize the model for financial NLP tasks.
- Testing and evaluation on FinLLM benchmark tasks.

## Citation
If you use this model, please cite our work:

```
@inproceedings{cao2024catmemo,
  title={CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications},
  author={Cao, Yupeng and Yao, Zhiyuan and Chen, Zhi and Deng, Zhiyang},
  booktitle={Joint Workshop of the 8th Financial Technology and Natural Language Processing (FinNLP) and the 1st Agent AI for Scenario Planning (AgentScen) in conjunction with IJCAI 2023},
  pages={174},
  year={2024}
}
```

## License
This model is licensed under the Apache 2.0 License. See the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) file for details.

## Acknowledgments
We thank the organizers of the FinLLM Challenge Task for providing the benchmark datasets and tasks used to develop this model.

---
### Model Card Metadata
- **License:** Apache 2.0
- **Tags:** TRL, SFT
- **Library Used:** Transformers