I just found this thread recently.  I think I am dealing with similar performance issue of deploying object detection model on CPU.
I am running Faster RCNN object detection model built on pytorch. The hardware is an AWS EC2 instance (T2.large with Intel Xeon, clock speed of 3.3 GHz). I am getting inference time of ~3 sec per frame. This seems to be similar to the performance the original poster (bonzogondo) mentioned above.
While I understand GPU is necessary for training a model, but I thought CPU is adequate for running a model in inference mode, (as I have seen real time object detection system running on raspberry pi or android phone) . Is this performance issue specific to Faster RCNN?
Sorry abt the newbie question. Thank you for any advice and insight.