I also think that it might make more sense to finetune the official CLIP weights! Applying for the data might take a few days though so if this project is going to be selected, we might want to take that into account!