You can find model memory with:

  
    
    
    How to find out keras model memory size? General Discussion
  
  
    Hello! 
I am doing a school work and I need to find out keras model memory size so I could compare different models. It is supposed to be composed of weights/parameters and model itself. It was given that model.summary() should contain all the information. From there I see that layer info and output shapes with the number of parameters. 
I understand parameters as they are just a numbers and so, number of parameters * 4B most likely will give how much room parameters take. But I know that more i…
  


You can try with smaller model and explore some model optimizations:

  
      

      TensorFlow
  

  
    

TensorFlow Model Optimization 3

  A suite of tools for optimizing ML models for deployment and execution. Improve performance and efficiency, reduce latency for inference at the edge.


  

  
    
    
  

  


You can also play with the TF-TRT memory managment params:
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#mem-manage 3
  
      

      NVIDIA Developer Forums – 11 Jul 21
  

  
    

Same memory usage for fp16 and int8 1

  Hello,  I wanted to benchmark depth estimation model on Jetson Xavier NX in terms of speed and memory usage. For that purpose I have converted pytorch model to ONNX format and than I have created TensorRT engines with fp32, fp16 and int8 precisions....

  
    Reading time: 1 mins 🕑
      Likes: 1 ❤