Add README.md

README.md

@@ -0,0 +1,28 @@
+# BioGPT INT8 Quantized for Medical Feature Extraction
+
+This is an INT8 quantized version of Microsoft's BioGPT for CPU inference.
+
+## Quick Start
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# Load base model and apply quantization
+tokenizer = AutoTokenizer.from_pretrained("microsoft/biogpt")
+model = AutoModelForCausalLM.from_pretrained("microsoft/biogpt", torch_dtype=torch.float16)
+model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)
+model.eval()
+
+# Use for inference
+prompt = "Extract medical features: Patient is 45-year-old male with fever 101.2F"
+inputs = tokenizer.encode(prompt, return_tensors="pt")
+outputs = model.generate(inputs, max_new_tokens=100)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Model Details
+- Base: microsoft/biogpt
+- Quantization: INT8 dynamic
+- Size: ~85MB (vs 1.56GB original)
+- Optimized for: CPU inference