The Dynamic Quantization on an LSTM Word Languange Model 2 or the Static Quantization with Eager Mode 1 tutorials might be a good starter.
The first tutorial seems to use a custom model, which seems to fit your use case.