Hi,
You may find this blog post interesting: https://towardsdatascience.com/the-power-of-constrained-language-models-cf63b65a035d 2.
It shows how to customize the beam search objective of Transformer-based methods.