Just tried some more benchmarks on various processors – the speed definitely does not scale linearly with number of CPUs – I found actually that one of my machines that 12x CPUs outperformed one with 20x CPUs, possibly because it is a couple years newer, or has higher clock speed. And all the inference times are pretty close – within 250ms of eachother, even when comparing the 12x CPU machine with the 36x CPU. I wonder if for some reason the speed is more related to clock GHZ instead of number of CPUs.