joeddav:

mixed precision


I noticed using the zero-shot-classification pipeline that loading the model (i.e. this line:  classifier = pipeline(“zero-shot-classification”, device=0)) takes about 60 seconds, but that inference afterward is quite fast.  Is there a way to speed up the model/tokenizer loading process?  Thanks!