Thanks @jplu
By considering your suggestion by setting input shape, I was able to do inference on TFlite models converted from Huggingface.
Here is my Notebook 69 for someone who is facing such an issue.