I am gonna suggest using 2 threads. The octacore on pixel6 has 2 X1 2 A76 and 4 A55. 4 A55 are pretty slow. So you may get better speedup with 2 core. A few follow up questions:

Is your model quantized?
If not quantized are you using optimize_for_mobile to transform your model?