Upload merged LoRA model

README.md

@@ -0,0 +1,42 @@
+
+# CultureCLIP Model (LoRA Merged)
+
+This is a CLIP model fine-tuned with LoRA for cultural understanding and image-text matching. The LoRA weights have been merged into the base model.
+
+## Model Details
+
+- **Base Model**: openai/clip-vit-base-patch32
+- **Task**: Contrastive Image-Text Learning
+- **Framework**: PyTorch
+- **Fine-tuning Approach**: LoRA (Low-Rank Adaptation)
+
+## LoRA Configuration
+- **Rank (r)**: 4
+- **Alpha**: 16
+- **Dropout**: 0.1
+- **Target Modules**: q_proj, v_proj
+- **Task Type**: FEATURE_EXTRACTION
+
+## Usage
+
+```python
+from transformers import CLIPModel, CLIPProcessor
+
+# Load model and processor
+model = CLIPModel.from_pretrained("lukahh/cultureclip_lora_0315_100k_32_07_03")
+processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")  # Use base model's processor
+
+# Process text and images
+inputs = processor(
+    text=["a photo of a cat", "a photo of a dog"],
+    images=image,
+    return_tensors="pt",
+    padding=True
+)
+
+# Get outputs
+outputs = model(**inputs)
+```
+
+## Training Details
+This model was fine-tuned using LoRA and then merged back into the base model. The LoRA approach enables efficient adaptation of the CLIP model while maintaining its core capabilities.

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "openai/clip-vit-base-patch32",
+  "architectures": [
+    "CLIPModel"
+  ],
+  "initializer_factor": 1.0,
+  "logit_scale_init_value": 2.6592,
+  "model_type": "clip",
+  "projection_dim": 512,
+  "text_config": {
+    "bos_token_id": 0,
+    "dropout": 0.0,
+    "eos_token_id": 2,
+    "model_type": "clip_text_model",
+    "torch_dtype": "float32"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "vision_config": {
+    "dropout": 0.0,
+    "model_type": "clip_vision_model",
+    "torch_dtype": "float32"
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc5c2b36fd3ef136fbd745b4c7dc1240e6743cd81a6ff43883281bddb8a10500
+size 605156676