readme: ---
  tags:
  - image-to-text
  - image-captioning
  - endpoints-template
  license: bsd-3-clause
  library_name: generic
  ---
  
  # Fork of [salesforce/BLIP](https://github.com/salesforce/BLIP) for a `image-captioning` task on 🤗Inference endpoint.
  
  This repository implements a `custom` task for `image-captioning` for 🤗 Inference Endpoints. The code for the customized pipeline is in the [pipeline.py](https://huggingface.co/florentgbelidji/blip_captioning/blob/main/pipeline.py).
  To use deploy this model a an Inference Endpoint you have to select `Custom` as task to use the `pipeline.py` file. -> _double check if it is selected_
  ### expected Request payload
  ```json
  {
    "image": "/9j/4AAQSkZJRgABAQEBLAEsAAD/2wBDAAMCAgICAgMC....", // base64 image as bytes
  }
  ```
  below is an example on how to run a request using Python and `requests`.
  ## Run Request 
  1. prepare an image. 
  ```bash
  !wget https://huggingface.co/datasets/mishig/sample_images/resolve/main/palace.jpg
  ```
  2.run request
  
  ```python
  import json
  from typing import List
  import requests as r
  import base64
  
  ENDPOINT_URL = ""
  HF_TOKEN = ""
  
  def predict(path_to_image: str = None):
      with open(path_to_image, "rb") as i:
          image = i.read()
      payload = {
          "inputs": [image],
          "parameters": {
                     "do_sample": True,
                     "top_p":0.9,
                     "min_length":5,
                     "max_length":20
          }
      }
      response = r.post(
          ENDPOINT_URL, headers={"Authorization": f"Bearer {HF_TOKEN}"}, json=payload
      )
      return response.json()
  prediction = predict(
      path_to_image="palace.jpg"
  )
  
  ```
  Example parameters depending on the decoding strategy:
  
  1. Beam search
  
  ``` 
          "parameters": {
                     "num_beams":5,
                     "max_length":20
          }
  ```
  
  2. Nucleus sampling
  
  ``` 
          "parameters": {
                     "num_beams":1,
                     "max_length":20,
                     "do_sample": True,
                     "top_k":50,
                     "top_p":0.95
          }
  ```
  
  3. Contrastive search
  
  ``` 
          "parameters": {
                     "penalty_alpha":0.6,
                     "top_k":4
                     "max_length":512
          }
  ```
  
  See [generate()](https://huggingface.co/docs/transformers/v4.25.1/en/main_classes/text_generation#transformers.GenerationMixin.generate) doc for additional detail
  
  
  expected output
  ```python
  ['buckingham palace with flower beds and red flowers']
  ```
  