First README file.
Browse files
README.md
CHANGED
|
@@ -1,3 +1,92 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
base_model:
|
| 6 |
+
- microsoft/deberta-v3-base
|
| 7 |
+
pipeline_tag: zero-shot-classification
|
| 8 |
+
tags:
|
| 9 |
+
- smart
|
| 10 |
+
- city
|
| 11 |
+
- classifier
|
| 12 |
+
- genai
|
| 13 |
+
---
|
| 14 |
+
# GenAI Smart City Classifier (DeBERTa v3 Base Fine-Tune)
|
| 15 |
+
|
| 16 |
+
Binary transformer classifier detecting whether a text describes a Generative AI (GenAI) application in a smart city context.
|
| 17 |
+
|
| 18 |
+
## Labels
|
| 19 |
+
- 0: GenAI used for smart city application
|
| 20 |
+
- 1: Not related
|
| 21 |
+
|
| 22 |
+
`id2label = {0: "GenAI used for smart city application", 1: "Not related"}`
|
| 23 |
+
|
| 24 |
+
## Model Card Summary
|
| 25 |
+
- Base: microsoft/deberta-v3-base
|
| 26 |
+
- Tokenizer: DebertaV2Tokenizer (same as base)
|
| 27 |
+
- Max length used in training batches: 512 (inference examples use 256)
|
| 28 |
+
- Loss: Custom focal loss (γ=2) + label smoothing (0.1)
|
| 29 |
+
- Scheduler: Cosine, warmup 10%
|
| 30 |
+
- Epochs: 8, batch size 8 (train) / 16 (eval)
|
| 31 |
+
- Calibration: Temperature scaling (optimal ≈ 0.602)
|
| 32 |
+
|
| 33 |
+
## Quick Start
|
| 34 |
+
```python
|
| 35 |
+
import torch
|
| 36 |
+
from transformers import DebertaV2Tokenizer, AutoModelForSequenceClassification
|
| 37 |
+
|
| 38 |
+
MODEL_ID = "joaocarlosnb/genai-smartcity-classifier" # replace with actual repo id
|
| 39 |
+
TEMP = 0.602
|
| 40 |
+
id2label = {0: "GenAI used for smart city application", 1: "Not related"}
|
| 41 |
+
|
| 42 |
+
tokenizer = DebertaV2Tokenizer.from_pretrained(MODEL_ID)
|
| 43 |
+
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
|
| 44 |
+
model.eval()
|
| 45 |
+
|
| 46 |
+
def predict(text, max_length=256, apply_temp=True):
|
| 47 |
+
inputs = tokenizer(text, truncation=True, padding="max_length",
|
| 48 |
+
max_length=max_length, return_tensors="pt")
|
| 49 |
+
with torch.no_grad():
|
| 50 |
+
logits = model(**inputs).logits
|
| 51 |
+
if apply_temp:
|
| 52 |
+
logits = logits / TEMP
|
| 53 |
+
probs = torch.softmax(logits, dim=-1)[0]
|
| 54 |
+
top = int(probs.argmax())
|
| 55 |
+
return {
|
| 56 |
+
"label": id2label[top],
|
| 57 |
+
"probabilities": {id2label[i]: float(p) for i, p in enumerate(probs)}
|
| 58 |
+
}
|
| 59 |
+
|
| 60 |
+
print(predict("We apply a diffusion model to simulate traffic for urban planning."))
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
## Installation
|
| 64 |
+
```bash
|
| 65 |
+
pip install transformers torch
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
## Input Guidance
|
| 69 |
+
Short technical sentences or abstract fragments (English). Truncate >512 tokens automatically.
|
| 70 |
+
|
| 71 |
+
## Limitations
|
| 72 |
+
- Binary only (no “mentioned, not used” middle class)
|
| 73 |
+
- English academic / technical domain bias
|
| 74 |
+
- Not evaluated for adversarial or multilingual robustness
|
| 75 |
+
|
| 76 |
+
## Intended Use
|
| 77 |
+
Research, corpus analysis, and exploratory filtering. Human review is recommended before operational deployment.
|
| 78 |
+
|
| 79 |
+
## Dataset
|
| 80 |
+
Training data hosted separately (same namespace). Contains augmented, adaptive, contrastive, and diagnostic subsets.
|
| 81 |
+
|
| 82 |
+
## Reproducibility Notes
|
| 83 |
+
Set `seed=42`. Use DebertaV2Tokenizer with max_length=512 for full retraining.
|
| 84 |
+
|
| 85 |
+
## Citation (Placeholder)
|
| 86 |
+
> Bittencourt, J. C. N., Flores, T. K. S., Jesus, T. C., & Costa, D. G. (2025). On the Role of AI in Building Generative Urban Intelligence. In Review. https://doi.org/10.21203/rs.3.rs-7131966/v1
|
| 87 |
+
>
|
| 88 |
+
## License
|
| 89 |
+
See repository LICENSE (ensure compatibility with upstream model license).
|
| 90 |
+
|
| 91 |
+
## Security
|
| 92 |
+
Do not hard-code Hugging Face tokens. Use environment variable: `export
|