Upload README.md

README.md

@@ -0,0 +1,48 @@
+---
+tags:
+- image-to-text
+- image-captioning
+license: apache-2.0
+language:
+  - zh
+widget:
+- src: https://huggingface.co/Maciel/Muge-Image-Caption/blob/main/%E5%B0%8F%E8%80%B3%E9%92%89.jpg
+  example_title: 小耳钉
+- src: https://huggingface.co/Maciel/Muge-Image-Caption/blob/main/%E5%8D%AB%E8%A1%A3.jpg
+  example_title: 卫衣
+- src: https://huggingface.co/Maciel/Muge-Image-Caption/blob/main/%E9%AB%98%E8%B7%9F%E9%9E%8B.jpg
+  example_title: 高跟鞋
+---
+
+### 功能介绍
+
+该模型功能主要是对图片生成文字描述。模型结构使用Encoder-Decoder结构，其中Encoder端使用BEiT模型，Decoder使用GPT模型。
+
+使用中文Muge数据集训练语料，训练5k步，最终验证集loss为0.3737，rouge1为20.419，rouge2为7.3553，rougeL为17.3753，rougeLsum为17.376。
+
+[Github项目地址](https://github.com/Macielyoung/Chinese-Image-Caption)
+
+
+
+### 如何使用
+
+```python
+from transformers import VisionEncoderDecoderModel, ViTFeatureExtractor, AutoTokenizer
+from PIL import Image
+
+pretrained = "Maciel/Muge-Image-Caption"
+model = VisionEncoderDecoderModel.from_pretrained(pretrained)
+feature_extractor = ViTFeatureExtractor.from_pretrained(pretrained)
+tokenizer = AutoTokenizer.from_pretrained(pretrained)
+
+image_path = "https://huggingface.co/Maciel/Muge-Image-Caption/blob/main/%E9%AB%98%E8%B7%9F%E9%9E%8B.jpg"
+image = Image.open(image_path)
+if image.mode != "RGB":
+		image = image.convert("RGB")
+pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
+
+output_ids = model.generate(pixel_values, **gen_kwargs)
+preds = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
+preds = [pred.strip() for pred in preds]
+print(preds)
+```