Welcome to the forum and thank you for sharing your TensorFlow.js demo! Always great to see what folk are up to.  I actually did something similar a few years back (but was not TensorFlow.js) where I recorded a GIF of myself and was able to make it look like you were playing a real person - you may be able to update your version to do something similar too:
Rock Paper Scissors - Machine Learning Style using Tensor Flow

You can see in the bottom left the live view in the last second when you show your hands, and the large GIF is the “computer”. My version here used websockets to transmit the final frame to web server for classification which is certainly less elegant than a pure TensorFlow.js solution, but it did not exist back when I made this 
Look forward to seeing how your project (or future projects) evolve! Thanks for being part of the #madeWithTFJS community!