Get FLOP count for a model
# 🚀 Feature request

Is there a way to calculate the number of FLOPs (an approximate is also fine) from a HuggingFace model?

## Motivation

I'm trying to look at the variation of FLOPs as sequence length varies for a BERT base model. Unfortunately, I could not find it and it is a super useful metric. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>Me too...
UP.<|||||>Is there any updates? Thanks!<|||||>maybe use pytorch profiler?