Open AI have a report on training FLOPs of various models:

  
      
      OpenAI – 16 May 18
  
  
    

AI and Compute 166

We’re releasing an analysis showing that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time (by comparison, Moore’s Law had a 2-year doubling period).


  
  
    
    
  
  


and they used the following equation to calculate training FLOPs:
(add-multiplies per forward pass) * (2 FLOPs/add-multiply) * (3 for forward and backward pass) * (number of examples in dataset) * (number of epochs)