Security by obscurity is underrated
Just as a side note, moving your ssh server port from 22 to whatever may make your server unreachable under strict firewalls. If you are allowed to connect to some whitelisted ports then it's highly unlikely that port  64323 will be allowed.