readme: ---
  license: mit
  ---
  
  ## Overview
  
  This model is based on [CLIP](https://openai.com/blog/clip) model and test on four kinds of animal datasets and ten kinds of animal datasets. CLIP model is a zero-shot pre-trained model so we don't need train model. We just input possible classes and image dataset to use model. Possible classes can be defined by yourself, it can be dataset labels or other description.
  
  ## Text-image matching 
  
  ### Model Input
  
  ```python
  Class = ["dog", "cat", "rabbit","squirrel"]
  image = preprocess(Image.open("/content/drive/MyDrive/Transformer_CLIP/Golden_Retriever.jpeg")).unsqueeze(0).to(device)
  text = clip.tokenize(Class).to(device)
  ```
  
  ### Model Process
  
  ```python
  with torch.no_grad():
      image_features = model.encode_image(image)
      text_features = model.encode_text(text)
      
      logits_per_image, logits_per_text = model(image, text)
      probs = logits_per_image.softmax(dim=-1).cpu().numpy()
  
  res = 0
  pos = -1
  for j in range(len(probs[0])):
    if probs[0][j] > res:
      res = probs[0][j]
      pos = j
  
  
  print("The options available are: " + str(Class))
  for i in range(len(probs[0])):
    print("The probability of " + str(Class[i]) + " is " + str(probs[0][i]))
  
  print("Model thinks this photo is most likely a " + Class[pos])
  ```
  
  ## Animal Classification
  
  ### Demo code (Python Notebook)
  https://huggingface.co/Haofeng/CLIP_animal_classification/blob/main/CLIP_classfication.ipynb
  https://huggingface.co/Haofeng/CLIP_animal_classification/blob/main/CLIP_classfication10.ipynb
  
  ### Dataset reference
  https://www.kaggle.com/datasets/alessiocorrado99/animals10
  https://www.kaggle.com/datasets/ayushv322/animal-classification