Is there an app to ping IP's periodically to check latency? I'm sure I came across a really nice app that would ping a list of sites or IP addresses every x mins/hours and send alerts when it could not ping.

But I can't find the app back, anyone know of any good ones?
How about a web service that will do this for you?

Check out host-tracker.com.  It will ping an IP address (or http-get or traceroute) for you from a collection of sensor sites.  It looks like it will email you error notifications.

I haven't used the site, so can't speak to how well it works.