github.com/tensorflow/addons
  

  
    
  
	  
  

  
    
      'Profiler Report' - like PyTorch-Lighting in TF.Keras interface.  2
    

    
      
        opened 
        
          
        
        Mar 24, 2021
      
      


      
        
          
          innat
        
      
    

    
        discussion needed
        Feature Request
    
  


  
    **Describe the feature and the current behavior/state.**

I found the [profile…r report](https://pytorch-lightning.readthedocs.io/en/0.7.1/profiler.html) produced in the pytorch-lighting (`pt`) model training is very convenient to debug training. AFAIK, `tf` has also a profiler which can be viewed via `tensor board`. In `pt`, it has 

```
Profiler Report

Action                  |  Mean duration (s)    |  Total time (s)
-----------------------------------------------------------------
on_epoch_start          |  5.993e-06            |  5.993e-06
get_train_batch         |  0.0087412            |  16.398
on_batch_start          |  5.0865e-06           |  0.0095372
model_forward           |  0.0017818            |  3.3408
model_backward          |  0.0018283            |  3.4282
on_after_backward       |  4.2862e-06           |  0.0080366
optimizer_step          |  0.0011072            |  2.0759
on_batch_end            |  4.5202e-06           |  0.0084753
on_epoch_end            |  3.919e-06            |  3.919e-06
on_train_end            |  5.449e-06            |  5.449e-06
```

Numbers are random of course. We can easily accomplish `on_epoch_start/end`, `on_batch_start/end` timing. But not sure about others (mostly). I don't know yet if there is any such mechanism exist already in `tf. keras`. I think it would be nice to have it. 

**Relevant information**
- Are you willing to contribute it (yes/no): no
*If you wish to contribute, then read the requirements for new contributions in [`CONTRIBUTING.md`](https://github.com/tensorflow/addons/blob/master/CONTRIBUTING.md#requirements-for-new-contributions-to-the-repository)*
- Are you willing to maintain it going forward? (yes/no): no
- Is there a relevant academic paper? (if so, where): no
- Does the relevant academic paper exceed 50 citations? (yes/no): no
- Is there already an implementation in another framework? (if so, where):
- Was it part of tf. contrib? (if so, where): yes, https://pytorch-lightning.readthedocs.io/en/0.7.1/profiler.html

**Which API type would this fall under (layer, metric, optimizer, etc.)**
 `tfa.callbacks` I guess. 

**Who will benefit from this feature?**
`tf.keras` users. 

**Any other info.**