Even when he swam thousands of miles out of the Pacific and got to a place called Cape Corrientes -LRB- that was when he was coming back from Gough 's Island -RRB- , he found a few hundred mangy seals on a rock and they told him that men came there too .