For that I think that you need to wait for:

  

      github.com
  

  
    tensorflow/community/blob/master/rfcs/20201201-cpp-gradients.md 17


    # C++ Gradients 

| Status        | Accepted       |
:-------------- |:---------------------------------------------------- |
| **RFC #**     | [335](https://github.com/tensorflow/community/pull/335) |
| **Author(s)** | Saurabh Saxena (Google) |
| **Sponsor**   | Rohan Jain (Google)                 |
| **Updated**   | 2020-12-01                                           |


## Objective

We propose performing gradient computation entirely in C++. We mainly focus on reverse-mode autodiff here and leave forward-mode AD as future work. A C API for this implementation is also left as future work but we imagine it to be a straightforward wrapper over the provided C++ APIs. The APIs discussed here are experimental and hence do not have any backwards compatibility guarantees yet.


## Motivation

**Performance**: Running op gradient functions in Python means we incur python overheads for a transformation pass that could entirely be done in C/C++. This most severely affects eager mode training of compute-light pipelines as we show in the performance impact section.

**Support C++ training**: Lack of autodiff support in C/C++ could be detrimental to high performance training pipelines at Google scale. This has often come up in discussions internally.



  This file has been truncated. show original