|
--- |
|
license: cc-by-nc-sa-4.0 |
|
language: |
|
- en |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
tags: |
|
- finance |
|
- legal |
|
--- |
|
# Model Card for Model ID |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
RegLLM is LLM model for regulatory compliance. It has been domain adapted by unsupervised pretraining and instruction finetuned for regulatory compliance. |
|
This release focuses on Indian Banking rules and regulations. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
- **Developed by:** [dataeaze systems pvt ltd](https://www.dataeaze.io/) |
|
- **Funded by:** [dataeaze systems pvt ltd](https://www.dataeaze.io/) |
|
- **Shared by:** [dataeaze systems pvt ltd](https://www.dataeaze.io/) |
|
- **Model type:** PhiForCausalLM |
|
- **Language(s) (NLP):** English |
|
- **License:** [cc-by-nc-sa-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en) Model is made available under non-commercial use for research purposes only. For commercial usage please connect at [email protected] |
|
- **Finetuned from model:** [miscrosoft-phi-2](https://huggingface.co/microsoft/phi-2) |
|
|
|
|
|
## Uses |
|
|
|
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
|
### Direct Use |
|
|
|
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
|
|
|
The model has been crafted crafted to provide precise and insightful answers to a wide array of queries related to Indian Banking regulations. |
|
|
|
### Downstream Use |
|
|
|
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
|
|
This model can be used as core component in RegTech application |
|
|
|
### Out-of-Scope Use |
|
|
|
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. --> |
|
|
|
Model has been fine tuned on a specific task of answering questions related to Indian regulatory compliance. |
|
Any use beyond this is not guaranteed to be accurate. |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
<!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
|
- **Bias:** Trained for English language only (as of now). |
|
- **Risk:** Guardrails are reliant on the base models Microsoft Phi-2. Finetuning could impact this behaviour. |
|
- **Limitations:** Intended to be a small model optimised for Indian regulations (as of now). |
|
|
|
|
|
### Recommendations |
|
|
|
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
|
* This model is supposed to be used as an assistive AI technology. Kindly consult and verify with the source documents for decision making. |
|
* This model should be used with grounding on a set of regulatory documents. |
|
|
|
## How to Get Started with the Model |
|
|
|
``` |
|
from transformers import pipeline |
|
import torch |
|
|
|
pipe = pipeline("text-generation", model="dataeaze/dataeaze-RegLLM-microsoft_phi_2-dzcompli", |
|
torch_dtype=torch.bfloat16, |
|
device_map="auto") |
|
|
|
pipe.tokenizer.pad_token = pipe.tokenizer.eos_token |
|
|
|
result = pipe(f"What are the skills that a CCO should have?", |
|
max_new_tokens=256, |
|
do_sample=True, |
|
temperature=0.1, |
|
top_k=50, |
|
top_p=0.95)[0]['generated_text'] |
|
|
|
print(result) |
|
|
|
``` |
|
|
|
## Sample Output |
|
|
|
### Question |
|
What are the skills that a CCO should have? |
|
|
|
### RegLLM respose |
|
|
|
``` |
|
Instruct: What are the skills that a CCO should have? |
|
Output: The skills that a CCO should have include leadership, communication, and a strong understanding of compliance. |
|
They should also be able to work effectively with other departments and have a good track record of compliance. |
|
``` |
|
|
|
### GPT-4 response |
|
<table> |
|
<tr style="border-spacing: 5px;"> |
|
<td> |
|
<img src="chatgpt_cco_1.png" alt="gpt-4-respnse" width="750" /> |
|
</td> |
|
<td> |
|
<img src="chatgpt_cco_2.png" alt="gpt-4-respnse" width="750" /> |
|
</td> |
|
</tr> |
|
</table> |
|
|
|
|
|
### Reference |
|
|
|
For evalating truthfulness / hallucination of this response, refer to RBI notification |
|
[RBI/2022-23/24 |
|
Ref.No.DoS.CO.PPG./SEC.01/11.01.005/2022-23](https://rbidocs.rbi.org.in/rdocs/Notification/PDFs/NT244C25EB0BBB1E4F91AEB101D425EA639A.PDF) (page 8) |
|
|
|
Screenshot below |
|
|
|
<img src="CCO_Skills.png" alt="rbi-gold-answer" width="1000"/> |
|
|
|
|
|
As you can see, RegLLM has identified CCO has identified Chief Compliance Officer, while GPT-4 (Copilot) identifies CCO has Chief Commercial Officer. |
|
Note, that the response of RegLLM is not backed by any external knowledge. |
|
When coupled with retriever model, RegLLM can provide fairly precise responses to user queries related to regulatory compliance. |
|
|
|
Keep watching this space for more updates on the model and evaluations. |
|
|
|
## Model Card Authors |
|
|
|
* Niranjan Kakade |
|
* Atharva Inamdar |
|
* Tony Tom |
|
* Nayan Chheda |
|
* Sourabh Daptardar |
|
|
|
## Model Card Contact |
|
|
|
"dataeaze systems" <[email protected]> |
|
|
