Is your SSH password revealed when you attempt to connect to the wrong server?
I highly recommend to move off password-based SSH at all in favor of SSH keys or (even better, certificates).Here are some projects to help you roll out new infrastructure without SSH passwords:https://github.com/gravitational/teleport ( I work on this one with our team) and 
https://github.com/Netflix/bless (heard great things about it, especially if you are using AWS)