I think increasing the # of threads helps you only as much as the # of cores matches that. Particularly more modern phones architecture is more heterogenous where you have some variation of small, medium and large cores. What is the phone you are using? Also what is the model? If you are spending large part of the runtime outside of say qnnpack backed operators then also utilization might be low.
I would say simpleperf would be a good tool to get handle on this. Are you familiar with that?