Question: Is there way in jetpack(mozilla's extension development framework) , to find the the visited links in a page?

BTW: jquery's a:visited works in firebug , but not in jetpack. i'm guessing
it's part of jetpack's security.

Answer: 
I had a similar problem and you cannot avoid having overlapped icons regardless the method to mark some icons as high-priority.

What I did(it might not be easy to apply in your case - in my case the map was rendered in a desktop application, with much more control above the rendering process) was to sort based on priority, and paint only markers which are not overlapping - showing also a message like "XXX overlapping markers removed". This way the user doesn't become overwhelmed with information and he still sees the most important information.

I hope this helps.
Not sure I understand completely, but perhaps you can assign each layer a "neighborhood density," based on the inverse of the average distance from each point to its closest neighbor. For a particular zoom level, you would calculate a maximum density that is comfortably viewable and use that as a threshold.