I found that the best way to do this is by directly calling the model with the necessary inputs rather than using the generate method, and to build logic around this that checks the number of a particular token in the resulting sequence and stops once its reached.