Question: If I connect with Java to MySQL on my localhost server, I access instantaneously.  

But if I connect outside of the localhost, from a network PC (192.168.1.100), it is very slow (4-5 seconds).  

And, if I connect from a public IP to my MY SQL server, it is also very slow (6 seconds or more).

Answer: 
Because your computer needs time to send packets to external servers and they need time to send packets back. It's called network latency, and is not an issue with Java specifically, but a general network issue.
Network latency plus connection creation time would be my guess. I don't know what else you have between the client machine and the MySQL server.