@smth Thank you for your solution for improvement. I had the same problem with OP and this largely improved inference speed on the Xeon server that I’m using. However, it is still significantly slower than on a Mac. Are there any other methods that you can think of that can make the gap smaller? Thanks!