soldni commited on
Commit
67f2c20
1 Parent(s): db1daf2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -0
README.md CHANGED
@@ -135,6 +135,58 @@ print(generated_text)
135
 
136
  *Benchmarks: AI2D test, ChartQA test, VQA v2.0 test, DocQA test, InfographicVQA test, TextVQA val, RealWorldQA, MMMU val, MathVista testmini, CountBenchQA, Flickr Count (we collected this new dataset that is significantly harder than CountBenchQA).*
137
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
  ## License and Use
139
 
140
  This model is licensed under Apache 2.0. It is intended for research and educational use.
 
135
 
136
  *Benchmarks: AI2D test, ChartQA test, VQA v2.0 test, DocQA test, InfographicVQA test, TextVQA val, RealWorldQA, MMMU val, MathVista testmini, CountBenchQA, Flickr Count (we collected this new dataset that is significantly harder than CountBenchQA).*
137
 
138
+
139
+ ## FAQ
140
+
141
+ ### I'm getting an error a broadcast error when processing images!
142
+
143
+ Your image might not be in RGB format. You can convert it using the following code snippet:
144
+
145
+ ```python
146
+ from PIL import Image
147
+
148
+ image = Image.open(...)
149
+
150
+ if image.mode != "RGB":
151
+ image = image.convert("RGB")
152
+ ```
153
+
154
+ ### Molmo doesn't work great with transparent images!
155
+
156
+ We received reports that Molmo models might struggle with transparent images.
157
+ For the time being, we recommend adding a white or dark background to your images before passing them to the model. The code snippet below shows how to do this using the Python Imaging Library (PIL):
158
+
159
+ ```python
160
+
161
+ # Load the image
162
+ url = "..."
163
+ image = Image.open(requests.get(url, stream=True).raw)
164
+
165
+ # Convert the image to grayscale to calculate brightness
166
+ gray_image = image.convert('L') # Convert to grayscale
167
+
168
+ # Calculate the average brightness
169
+ stat = ImageStat.Stat(gray_image)
170
+ average_brightness = stat.mean[0] # Get the average value
171
+
172
+ # Define background color based on brightness (threshold can be adjusted)
173
+ bg_color = (0, 0, 0) if average_brightness > 127 else (255, 255, 255)
174
+
175
+ # Create a new image with the same size as the original, filled with the background color
176
+ new_image = Image.new('RGB', image.size, bg_color)
177
+
178
+ # Paste the original image on top of the background (use image as a mask if needed)
179
+ new_image.paste(image, (0, 0), image if image.mode == 'RGBA' else None)
180
+
181
+ # Now you can pass the new_image to Molmo
182
+ processor = AutoProcessor.from_pretrained(
183
+ 'allenai/Molmo-7B-D-0924',
184
+ trust_remote_code=True,
185
+ torch_dtype='auto',
186
+ device_map='auto'
187
+ )
188
+ ```
189
+
190
  ## License and Use
191
 
192
  This model is licensed under Apache 2.0. It is intended for research and educational use.