Yes Tried Trainer instead,
The CLIPModel forward throws an exception where we are trying to get CLIPVisionEmbeddings from pixel_values.
8 frames
/usr/local/lib/python3.7/dist-packages/transformers/models/clip/modeling_clip.py in forward(self, pixel_values)
    138         class_embeds = self.class_embedding.expand(batch_size, 1, -1)
    139         embeddings = torch.cat([class_embeds, patch_embeds], dim=1)
--> 140         embeddings = embeddings + self.position_embedding(self.position_ids)
    141         return embeddings
    142 
RuntimeError: The size of tensor a (50) must match the size of tensor b (145) at non-singleton dimension 1