Answers.Body: <p>You can take the problem from a text point of view (especially if you think about Vision Transformer).</p>
  <p>First, I assume you cut out the images to avoid having white background in all images at the same place. This is good to avoid having larger image than necessary. And there is no information there.</p>
  <p>Now, having a white background means you would have sentence with repeated tokens but at random location in your sentence.</p>
  <p>First things, you could get a smarter tokenizer (since the vocabulary of your sentences is quite repetitive :D ). It means you would probably need to finetune or even retrain your embedding model (which would be necessary if you have very strange data whatever).</p>
  <p>You could also imagine to get a masked attention (ignoring the &quot;white-background&quot; token which carry no information by themself) getting inspiration from <a href="https://openaccess.thecvf.com/content/CVPR2022/papers/Cheng_Masked-Attention_Mask_Transformer_for_Universal_Image_Segmentation_CVPR_2022_paper.pdf" rel="nofollow noreferrer">this article</a> for example (but i guess many other).</p>
  <p>Hope it helps to nurture your curiosity.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-22T15:23:11.313Z
Answers.LastActivityDate: 2024-11-22T15:23:11.313Z
Answers.UserDisplayName: Paucodeici
Body: <p>I’m a third-year student working on recommender systems. Lately, I’ve been studying image preprocessing in this area. For image data, there are ViT (vision transformer), MAE, or other kinds of vision encoders that generate embedding for an image. I’m wondering if my image data all have monotonic backgrounds (ex. white background), is there a way that this would make generating embedding easier? Most of the papers I found deal with noisy backgrounds, rather than “clean” ones.</p>
  <p>Please help, thanks.</p>
  
Tags: <computer-vision><embeddings>
Title: Is there a method that similar background in computer vision woulf help the generation of embeddings?
UserDisplayName: I_really_love_toefl
ThreadText: # Is there a method that similar background in computer vision woulf help the generation of embeddings?
  
  (Asked by: username_0 on 2024-11-22)
  
  I’m a third-year student working on recommender systems. Lately, I’ve been studying image preprocessing in this area. For image data, there are ViT (vision transformer), MAE, or other kinds of vision encoders that generate embedding for an image. I’m wondering if my image data all have monotonic backgrounds (ex. white background), is there a way that this would make generating embedding easier? Most of the papers I found deal with noisy backgrounds, rather than “clean” ones.
  
  Please help, thanks.
  
  **Tags:** computer-vision, embeddings
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-22. Score: 1
  
  You can take the problem from a text point of view (especially if you think about Vision Transformer).
  
  First, I assume you cut out the images to avoid having white background in all images at the same place. This is good to avoid having larger image than necessary. And there is no information there.
  
  Now, having a white background means you would have sentence with repeated tokens but at random location in your sentence.
  
  First things, you could get a smarter tokenizer (since the vocabulary of your sentences is quite repetitive :D ). It means you would probably need to finetune or even retrain your embedding model (which would be necessary if you have very strange data whatever).
  
  You could also imagine to get a masked attention (ignoring the "white-background" token which carry no information by themself) getting inspiration from [this article](https://openaccess.thecvf.com/content/CVPR2022/papers/Cheng_Masked-Attention_Mask_Transformer_for_Universal_Image_Segmentation_CVPR_2022_paper.pdf) for example (but i guess many other).
  
  Hope it helps to nurture your curiosity.
  
  
  ---
  
  