Body: <p>I'm looking for a captioning model that would be able to describe a group of images in a single sentence. Alternatively, I need a way to conceptually average a group of images before feeding that &quot;concept&quot; (presumably a feature vector) to a regular captioning model.</p>
  <p>Why?</p>
  <p>For training dataset curation. It would be beneficial to test the trained generation model on a prompt that would fit it as a whole instead of selecting captions of single images or trying to find what's common among them. Moreover, this would also allow to produce a single negative prompt to test how model behaves on out of scope prompts.</p>
  <p>What I've done so far:
  I've <a href="https://github.com/Seedmanc/clip-interrogator-average/" rel="nofollow noreferrer">modified</a> an existing CLIP+BLIP interrogator to work with image sets. However, while CLIP captioning allows averaging image features before using them to choose the best caption, it's far less accurate than captions produced by BLIP, which only works with single images. I need a model that would take in feature vectors like CLIP so I can preprocess them.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-03T12:28:08.073Z
Comments.Text: >ask on fitting stackexchange >get ignored >ask on SO >get closed
Comments.UserDisplayName: Seedmanc
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-03T22:56:24.643Z
Comments.Text: [ai.se] says *"Q&A for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment"* you could try asking on [stats.se] which is the engineering-minded sibling site, or [datascience.se]. be sure to check out each site's "on-topic" article in the [help/on-topic]. the big index: https://stackexchange.com/sites#questionsperday
Comments.UserDisplayName: Christoph Rackwitz
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-10T14:10:35.510Z
Comments.Text: This is the fourth time I'm asking this question in different SEs and every time I'm either ignored entirely or my question is closed without any help. I'm disappointed in this site, I've done my research, I built a prototype, everything checks out and yet I'm being given the middle finger.
Comments.UserDisplayName: Seedmanc
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-17T12:41:15.113Z
Comments.Text: @Seedmanc Your question has not been closed here, but anyways there are never guarantees of getting an answer. People might simply not know what you are asking.
Comments.UserDisplayName: Dr. Snoopy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-17T12:53:50.093Z
Comments.Text: see also: https://stats.stackexchange.com/questions/657030/captioning-an-average-of-image-set -- OP asked this twice on SO. -- I guess it's a matter of feeling entitled to something the community didn't promise.
Comments.UserDisplayName: Christoph Rackwitz
Tags: <machine-learning><computer-vision><image-recognition>
Title: Captioning an average of image set (instead of individial pictures)
UserDisplayName: Seedmanc
ThreadText: # Captioning an average of image set (instead of individial pictures)
  
  (Asked by: username_0 on 2024-10-16)
  
  I'm looking for a captioning model that would be able to describe a group of images in a single sentence. Alternatively, I need a way to conceptually average a group of images before feeding that "concept" (presumably a feature vector) to a regular captioning model.
  
  Why?
  
  For training dataset curation. It would be beneficial to test the trained generation model on a prompt that would fit it as a whole instead of selecting captions of single images or trying to find what's common among them. Moreover, this would also allow to produce a single negative prompt to test how model behaves on out of scope prompts.
  
  What I've done so far:
  I've [modified](https://github.com/Seedmanc/clip-interrogator-average/) an existing CLIP+BLIP interrogator to work with image sets. However, while CLIP captioning allows averaging image features before using them to choose the best caption, it's far less accurate than captions produced by BLIP, which only works with single images. I need a model that would take in feature vectors like CLIP so I can preprocess them.
  
  **Tags:** machine-learning, computer-vision, image-recognition
  > **Comments:**
  > **username_0**: >ask on fitting stackexchange >get ignored >ask on SO >get closed  
  > **username_1**: [ai.se] says *"Q&A for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment"* you could try asking on [stats.se] which is the engineering-minded sibling site, or [datascience.se]. be sure to check out each site's "on-topic" article in the [help/on-topic]. the big index: https://stackexchange.com/sites#questionsperday  
  > **username_0**: This is the fourth time I'm asking this question in different SEs and every time I'm either ignored entirely or my question is closed without any help. I'm disappointed in this site, I've done my research, I built a prototype, everything checks out and yet I'm being given the middle finger.  
  > **username_2**: @Seedmanc Your question has not been closed here, but anyways there are never guarantees of getting an answer. People might simply not know what you are asking.  
  > **username_1**: see also: https://stats.stackexchange.com/questions/657030/captioning-an-average-of-image-set -- OP asked this twice on SO. -- I guess it's a matter of feeling entitled to something the community didn't promise.  
  
  ---
  
  ## 0 answers
  
  