Have you tried with

  

      github.com/onnx/onnx-tensorflow
  

  
    
  
	  
  

  
    
      ONNX->"TensorFlow Lite micro" fundamentally broken because of transpose operations 4
    

    
      
        opened 
        
          
        
        Feb 12, 2021
      
      


      
        
          
          maartenvds
        
      
    

    
    
  


  
    **Problem**

In my usecase, I need to port a model (only containing conv, relu… and maxpool layers) from PyTorch to TensorFlow Lite micro, using the following conversions: PyTorch->ONNX->TensorFLow->Tensorflow Lite micro. The microcontroller kernels of TFLite however only support NHWC format and there is no support for a transpose layer (see supported layers [here](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/micro/all_ops_resolver.cc)). In CPU mode, onnx-tf inserts transpose operations before and after every conv layer so the conv layers can work with NHWC format. However, the rest of the network is kept in NCHW format. What I need is a network with only NHWC tensors for all layers without any transpose operations (also input/output layers should have NHWC format).

It would be nice if onnx-tf had a mode where you can completely convert a model that is only using NCHW format to a model that is only using NHWC format instead of creating hybrids as is done by the current implementation.
I know this is currently not supported and there is probably not an easy solution for this. But is want to indicate that a conversion from ONNX->TFLite micro is currently fundamentally broken for any given model because of the transpose ops.

We could also fix the problem at the TFLite micro side, by implementing a transpose op, but that is a more wasteful solution given the unnecessary overhead of transpose operations on an already very constrained platform. Fixing the issue in TensorFlow by rewriting the graph seems a bit too complex to do and error prone to future changes.

I therefore kindly ask to consider implementing an additional mode in the converter if possible. I'm willing to contribute if necessary.

**Python, ONNX, ONNX-TF, Tensorflow version**

 - Python version: 3.6.5
 - ONNX version: 1.8.0
 - ONNX-TF version: 1.7.0 (master branch, commit: db092105ceebe076610a1b27c1fc1553978c17cd)
 - Tensorflow version: 2.3.1