The keypoints are simply the x,y co-ordinates found in the image you sent for classification. If you want to do any extra processing then that would be up to you.


It seems you have 2 different sized things - the image and the canvas.  You can solve this by simply taking the x,y co-ordinates that are returned and divide the x by the original image width, and divide the y by the original image height. You then have a ratio you can use to find it’s new position of a larger version of the image (eg the canvas). Simply then multiply the new canvas.width with the ratio for the x to find the new relative x, and then do the same for height too.


From your static image I do not see any jitter and I can not replicate this on my side. It may be your camera/lighting causing lack of accuracy in prediction. If that does not solve then you could investigate some smoothing algorithm eg moving average or something to reduce jitter at cost of latency.