TF Lite includes a Model Optimization Toolkit for helping you trade off size, latency, accuracy, etc. There’s also this model optimization guide 1.
Typically you do things like quantization (e.g. converting 32-bit floats into something with less precision), model pruning (e.g. trimming the parts unused for inference) and clustering (e.g. reducing the number of unique weights by sharing when approximately equal). The toolkit and guide can help you with all of these.