I would try with an increased representative dataset.
Also, when you are using the integer quantized model, it’s important to account for the scale and offset as shown here:

  
      

      TensorFlow
  

  
    

Post-training integer quantization  |  TensorFlow Lite 8



  

  
    
    
  

  


BatchNormalization should not be an issue here.



 Olivier_Staehli:

To run it on the Coral USB Accelerator I need to quantize it and convert it to a TensorFlow Lite model (and then compile it again with the Edge TPU compiler). But the prediction is massively worse after quantizing from 32bit float to 8bit int. The problem seems to be the batch normalization keras.layers.BatchNormalization().


Batch Normalization is well-supported actually. Refer here:

  
      

      TensorFlow
  

  
    

TensorFlow Lite and TensorFlow operator compatibility 10



  

  
    
    
  

  


Also, please try with the most stable version of the libraries that you are using.