@Taylor I have tried your demo, it is pretty amazing.
The performance can be better, since your model contains variable length input, the model need to compile the op shader each inference if the shapes changes.
You can see that for the exact same input string, the first inference could take up to 6sec, which the following inference only takes 600ms.
We also have a flag you can try where we parameterize the shape for the op shader, it should significant improve the first inference.  Can you try to add following flag to your demo.
tf.env().set('WEBGL_USE_SHAPES_UNIFORMS', true);