I would check, if this method is the bottleneck for your current approach and then check alternatives.
While a large number of classes might slow down the method, it isn’t necessarily the bottleneck, so your optimizations might not help.