Hugging Face's logo

dataeaze
/
dataeaze-RegLLM-microsoft_phi_2-dzcompli

Text Generation
Transformers
Safetensors
English
phi
finance
legal
custom_code
text-generation-inference
Inference Endpoints
Model card Files Community
dataeaze-RegLLM-microsoft_phi_2-dzcompli / README.md
sourabhd's picture
sourabhd
Update CCO image width
0f5e0b8 verified
preview code
|
raw
history blame
4.95 kB
metadata
license: cc-by-nc-sa-4.0
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - finance
  - legal

Model Card for Model ID

RegLLM is LLM model for regulatory compliance. It has been domain adapted by unsupervised pretraining and instruction finetuned for regulatory compliance. This release focuses on Indian Banking rules and regulations.

Model Details

Model Description

Uses

Direct Use

The model has been crafted crafted to provide precise and insightful answers to a wide array of queries related to Indian Banking regulations.

Downstream Use

This model can be used as core component in RegTech application

Out-of-Scope Use

Model has been fine tuned on a specific task of answering questions related to Indian regulatory compliance. Any use beyond this is not guaranteed to be accurate.

Bias, Risks, and Limitations

  • Bias: Trained for English language only (as of now).
  • Risk: Guardrails are reliant on the base models Microsoft Phi-2. Finetuning could impact this behaviour.
  • Limitations: Intended to be a small model optimised for Indian regulations (as of now).

Recommendations

  • This model is supposed to be used as an assistive AI technology. Kindly consult and verify with the source documents for decision making.
  • This model should be used with grounding on a set of regulatory documents.

How to Get Started with the Model 

from transformers import pipeline
import torch

pipe = pipeline("text-generation", model="dataeaze/dataeaze-RegLLM-microsoft_phi_2-dzcompli",
                torch_dtype=torch.bfloat16,
                device_map="auto")

pipe.tokenizer.pad_token = pipe.tokenizer.eos_token

result = pipe(f"What are the skills that a CCO should have?",
              max_new_tokens=256,
              do_sample=True,
              temperature=0.1,
              top_k=50,
              top_p=0.95)[0]['generated_text']

print(result)

Sample Output

Question

What are the skills that a CCO should have?

RegLLM respose 

Instruct: What are the skills that a CCO should have?
Output: The skills that a CCO should have include leadership, communication, and a strong understanding of compliance.
They should also be able to work effectively with other departments and have a good track record of compliance.

GPT-4 response

gpt-4-respnse gpt-4-respnse

Reference

For evalating truthfulness / hallucination of this response, refer to RBI notification RBI/2022-23/24 Ref.No.DoS.CO.PPG./SEC.01/11.01.005/2022-23 (page 8)

Screenshot below

rbi-gold-answer

As you can see, RegLLM has identified CCO has identified Chief Compliance Officer, while GPT-4 (Copilot) identifies CCO has Chief Commercial Officer. Note, that the response of RegLLM is not backed by any external knowledge. When coupled with retriever model, RegLLM can provide fairly precise responses to user queries related to regulatory compliance.

Keep watching this space for more updates on the model and evaluations.

Model Card Authors

  • Niranjan Kakade
  • Atharva Inamdar
  • Tony Tom
  • Nayan Chheda
  • Sourabh Daptardar

Model Card Contact

"dataeaze systems" [email protected]