You can create custom layers in python:
  
      

      keras.io
  

  
    

Keras documentation: Making new layers and models via subclassing 2



  

  
    
    
  

  



  
      

      keras.io
  

  
    

Keras documentation: Simple custom layer example: Antirectifier 3



  

  
    
    
  

  


The problem is that sometime, when you combine different TF native ops in the implementation, the performance could be too low (memory/flops) to reach an efficient training/inference as some part of your specific codepath could be not explicitly optimized by the TF compiler stack (e.g. missing ops fusion, etc…).
Generally uou can try to benchmark your specific implementation using also tf.function 1 but if you canno’t directly interact/contribute to the compiler stack to improve that specific codepath you will probably need to write C++/CUDA custom op/kernel to support your python custom layer.
See also:
  

      github.com
  

  
    tensorflow/community/blob/master/rfcs/20190814-kernel-and-op-registration.md 4


    # Kernel and Op Implementation and Registration API

| Status        | Accepted                                             |
:-------------- |:---------------------------------------------------- |
| **Author(s)** | James Ring (sjr@google.com).                                      |
| **Sponsor**   | Günhan Gülsoy (gunan@google.com)                                  |
| **Updated**   | 2020-06-02                                                        |

## Objective

Tensorflow (TF) currently provides a C++ API for implementing kernels and ops.
The Voltron project aims to create a modular/plugin-based TF implementation with
API and ABI surfaces. Plugins will be able to create and register custom kernel
and op implementations.

In order to provide a stable ABI, the Voltron team has chosen to provide C APIs
to plugin authors. This document introduces the C API for op and kernel
registration. For authors who wish to continue using C++ to interface with
TensorFlow, an ABI-stable C++ header-only API is provided.




  This file has been truncated. show original