I disagree.  Being able to downvote comments to -4 (and no farther) seems like an excellent system to me, as it allows us to punish comments that are detrimental to discussion while preventing comments from being so downvoted that they have no hope of redemption (in the case of controversial but contributory comments).  I'm curious - if downvoting were done away with, how would you suggest that we flag comments which are contrary to the spirit of HN?