You can still use your dataset for metric learning or embedding finetuning. You can use the backbone that you want (also efficientnet).
See:

  
      

      keras.io
  

  
    

Keras documentation: Image similarity estimation using a Siamese Network with... 29



  

  
    
    
  

  



  
      

      keras.io
  

  
    

Keras documentation: Metric learning for image similarity search 26