There are always going to be outlying cases, whatever strategy you use.  I suppose I could fix this one by making the threshold depend on either average comment karma or total karma.  But as I said, my inclination now is just to stop displaying points on comment threads.