Yes, we talked about this for a more articulate example at:

  
    
    
    Applying Grad-Cam to a model that was converted to Tensorflow Lite General Discussion
  
  
    You can also refer to the following colab for getting the gradients: